How to Choose a Leadership Assessment: Criteria That Matter

DorWay

Sandor Kovacs

February 19, 2026

Editorial photograph illustrating: Criteria For Selecting Leadership Evaluation Methods
How to choose a leadership assessment: define the decision, apply selection criteria such as validity, reliability, and fairness, match methods to purpose, and pilot first.

To choose a leadership assessment, start with the decision it will support (selection, promotion, or development), then judge each option on validity for that purpose, reliability, fairness, practicality, and whether it leads to action. An assessment that is excellent for development can be a poor basis for a promotion decision, and the reverse is also true.

Most selection mistakes happen because buyers compare features and price before they agree on purpose. The criteria below help you compare options on what matters.

Step 1: Define the purpose before you compare anything

Write one sentence: “We will use the results to decide ____.” The answer changes what you need.

  • Selection or promotion: you need strong evidence that scores predict performance in the target role, consistent administration, and documented fairness.
  • Development: you need insight the leader will accept and act on, a skilled debrief, and a path to follow-up.
  • Program evaluation: you need measures that are stable and repeatable so change over time is visible.
  • Succession and talent review: you need comparability across many leaders and a view of future potential, such as a leadership potential assessment.

Step 2: Apply the core selection criteria

Criterion What it means Question to ask the vendor
Validity The assessment measures what it claims and predicts what you care about What evidence links scores to leadership performance for roles like ours?
Reliability Results are consistent across time and raters What are the reliability figures, and how were they calculated?
Fairness Scores do not disadvantage groups for reasons unrelated to the job Has adverse impact been studied, and with what results?
Relevance Content matches your leadership model and context Can it map to our competencies, or will we have to translate?
Practicality Time, cost, and administration fit your capacity How long does it take per leader and per rater? Who administers it?
Acceptance Leaders and raters see it as credible How do participants typically react, and what do debriefs involve?
Actionability Results point to specific changes What does a leader do the day after receiving the report?

For high-stakes uses, the Society for Industrial and Organizational Psychology’s Principles for the Validation and Use of Personnel Selection Procedures is the standard professional reference. If an assessment will influence who gets hired or promoted, ask vendors how their validation work aligns with it.

Step 3: Match the method to the purpose

Different methods suit different decisions:

  • Multi-rater feedback is strong for development and weak for selection, because ratings reflect relationships and raters soften scores when stakes rise.
  • Simulations and assessment centers are strong for selection and promotion into senior roles, at higher cost.
  • Psychometric instruments add consistency and are useful context, but should only drive decisions when validated for that use.
  • Self assessments are best for reflection. A large meta-analysis found that self-ratings are more lenient than supervisor ratings and agree with them only modestly.
  • Structured behavioral interviews work for both selection and development when interviewers are trained and questions are consistent.

Our guide to leadership assessment tools compares specific instruments in more depth.

Step 4: Check what the assessment cannot see

Every method has blind spots. Ask what yours will miss. A common one: most assessments describe traits, styles, or competency ratings, but very few look at how a leader handles commitments in practice. DorWay sees organizations as a Network of Agreements™, and many of the problems an assessment is meant to prevent, such as missed results, conflict, and low trust, trace back to agreements that were never specific enough to keep. The PACT™ Leadership Assessment was designed to cover that gap and pairs well with a broader instrument.

Step 5: Pilot before you scale

  1. Run the assessment with a small, varied group of leaders.
  2. Collect reactions from participants and raters.
  3. Check whether the results matched what managers already knew, and where they added something new.
  4. Confirm that debriefs led to specific development commitments.
  5. Only then roll out more widely, with a plan for re-assessment.

Red flags when evaluating vendors

  • Claims of predictive power with no published or shareable evidence.
  • One instrument presented as right for every purpose.
  • No guidance on how results should, and should not, be used.
  • Reports that rank leaders without explaining what to do next.

Once you have chosen, connect the assessment to outcome measures so you can see whether it helped. Our guides on measuring leadership impact and how to measure leadership effectiveness cover that step.

FAQ

How do I choose the right leadership assessment?

Define the decision it will support, then compare options on validity for that purpose, reliability, fairness, relevance to your leadership model, practicality, participant acceptance, and how clearly results lead to action. Pilot before scaling.

What makes a leadership assessment valid?

Evidence that it measures what it claims and that scores relate to outcomes you care about, such as job performance, for people in roles similar to yours. Validity is specific to a purpose, so ask for evidence tied to your intended use.

Should the same assessment be used for development and promotion?

Usually not. When feedback is tied to promotion, raters tend to be less candid. Many organizations use multi-rater tools for development and separate, validated methods for selection.

What is the difference between reliability and validity in leadership assessment?

Reliability means results are consistent across time and raters. Validity means results measure the right thing and predict what matters. An assessment can be reliable without being valid, but it cannot be valid without being reliable.

Tracy Vasaturo

Head of Sales

Tracy Vasaturo has spent the better part of two decades in rooms where the stakes are high and the sell is real. Medical devices. SaaS. ERP. High-ticket coaching programs. Live events. She has sold millions across her career — but the throughline has always been the same: finding the bridge between a technology or a transformation and the person who needs it. She’s sold to Fortune 500 executives and to entrepreneurs rebuilding their lives from scratch. She’s worked corporate sales floors and grassroots stages.

She understands that people don’t just buy a product — they buy into a belief that something can change.

At DorWay, Tracy works with individuals, entrepreneurs, executives, and organizational teams who are ready to lead and operate differently. If you’re exploring whether DorWay is right for you, there’s a good chance Tracy is who you’ll talk to first. That’s very much by design.

Thyme Francis

Chief Client Experience and Creative Officer | Co-Founder
Thyme Francis is the architect of how every client experiences DorWay, from the first moment of contact through the full arc of their transformation journey.
 

Her background spans creative direction and financial services, an unusual combination that turns out to be exactly the right one for her role. The creative lens means she understands how people experience brands, communications, and environments emotionally, not just functionally. The financial services background means she understands precision, accountability, and the weight of decisions that actually matter.

What Thyme does at DorWay is ensure that the quality of the client experience matches the quality of the transformation the programs are designed to create. That every touchpoint (every communication, every program delivery, every interaction with the DorWay team) reflects the same integrity and intentionality that DorWay teaches.

She brings to every client relationship the same thing she brings to every aspect of her work: genuine attention to the specific person in front of her. Their story. Their goals. What they actually need versus what they think they need. The ability to hold both the detail and the larger vision simultaneously.

In an organization built on the conviction that leadership is fundamentally relational, Thyme ensures that DorWay practices that conviction in every interaction it has.