Improving Performance Assessment for an Executive Coaching Company

Client Background

COACH A is a global executive coaching company that supports organizational transformation through dialogue. Headquartered in Tokyo, with offices in New York, Shanghai, Hong Kong, and Bangkok, the company offers multilingual services and is accredited by the International Coaching Federation (ICF).*

‍

Since its founding, COACH A has focused on elevating coaching quality and nurturing internal talent. Over the past 25 years, the company has built a comprehensive in-house training and capability evaluation system. This system integrates COACH A’s expertise with insights from leading international coaches and senior professionals within the organization.

‍

COACH A’s dedicated R&D division, the Coaching Research Institute, is a global pioneer in measuring coaching outcomes and applying those findings to continually refine its services.

‍

*ICF (International Coaching Federation): The world’s largest nonprofit professional coaching organization.

‍

Challenge

As part of its ongoing efforts to improve coaching quality, COACH A relies on evaluations conducted by specially trained assessors as one of the key elements in coach development. With client consent, recorded sessions are reviewed by multiple evaluators using the ICF Core Competencies* and PCC Markers** as criteria.

‍

However, manually evaluating every session requires a significant amount of time and effort. To address this challenge and further accelerate coach development, COACH A began developing an AI-powered evaluation tool. Recursive was chosen as the development partner for its advanced AI capabilities and ability to deliver flexible, rapid customization.

‍

*ICF Core Competencies: A set of professional coaching standards defined by ICF.

**PCC Markers (Professional Certified Coach Markers): Behavioral indicators that illustrate the ICF Core Competencies at a concrete, behavioral level, designed by ICF to evaluate whether coaching is being conducted in alignment with the Core Competencies.

‍

Recursive’s Solution

To support COACH A’s goals, Recursive developed a custom AI agent that automates the initial assessment of coaching sessions obtained with explicit client consent for the purpose of “improving coaching quality.”

‍

The AI evaluates each session based on the ICF Core Competencies, which cover four key areas: setting the foundation, co-creating the relationship, communicating effectively, and cultivating learning and growth. These are further broken down into specific behavioral indicators defined by PCC Markers.

‍

In addition to the content of the conversation, the AI also analyzes meta-communication elements such as tone, pacing, pauses, and the balance of speaking time between coach and client.

‍

How It Works

The evaluation engine is powered by large language models (LLMs) that compare session transcripts to PCC Marker definitions. Both are converted into numerical vectors, and the AI calculates their similarity. A similarity score of around 80% or higher is used as the benchmark for a passing judgment.

‍

The model also incorporates additional instructions (e.g., such as PCC Marker explanations) to enhance judgment accuracy.

‍

Feedback Format

For each session, the AI provides:

‍

• A pass/fail judgment for each competency
• An overall pass rate
• A written explanation for each evaluation
• Highlighted portions of the transcript that justify its decisions

‍

The system also flags sessions that may need closer human review, for example, when certain competencies are less visible or when response patterns are unusual. 

‍

Privacy and Interface

To protect client privacy, the AI does not process raw audio. Instead, it refers only to analysis outputs generated by an external audio processing tool already in use at COACH A.

‍

The initial version of the tool was tested through a web interface. Future updates, including desktop or mobile formats, will be considered based on operational needs.