You can't delete bias from a human judgment, but you can reduce and contain it by design. An awareness workshop or a nicer form won't do that. Removing bias from reviews means engineering the process so the rating reflects the work instead of who reviewed it, how recently, or which client they liked. You fix the criteria before you review, judge concrete evidence against a shared bar, cover the whole period instead of the last project, gather more than one input weighted by who saw the work, calibrate to catch outlier raters, then run a final language check on the written review. AI can flag vague or loaded wording, and a human makes the call. Aim for a more reliable, defensible judgment. Don't expect a "debiased" person or perfect objectivity.
How much does the rater contaminate the result? When researchers decomposed 4,492 managers' ratings, about 62% of the variance came from the rater's idiosyncrasies and only ~21% from the person's actual performance (Scullen, Mount & Goff, Journal of Applied Psychology, 2000). So most of a single score is the rater, and containing bias is how you get the work back into the picture.
Which biases distort project-based reviews most?
The ones that thrive when a person is staffed across several engagements and rated by whichever leads happened to run them. Five do most of the damage:
- Recency. The most recent engagement weighs too heavily in the overall rating. A strong or weak finish near the cycle close swamps ten months that came before (Steiner & Rain, Journal of Applied Psychology, 1989), and end-of-cycle recall turns into "what did they do lately?"
- Halo (the "favourite client" glow). One strong overall impression colours every separate judgment. The original "halo error" study found that supposedly independent ratings of a person's distinct qualities were implausibly correlated (Thorndike, Journal of Applied Psychology, 1920). The lead who loved working with someone rates them high on everything.
- The idiosyncratic rater effect. Much of any score reflects the rater's own perspective and leniency: ~62% of rating variance is rater idiosyncrasy versus ~21% actual performance (Scullen, Mount & Goff, 2000). One reviewer is a noisy instrument, and the interrater reliability of supervisory ratings is only about .52 (Viswesvaran, Ones & Schmidt, Journal of Applied Psychology, 1996).
- Similarity and affinity bias. Reviewers favour people who work, communicate, or problem-solve the way they do, and you can easily mistake that for "good judgment" or "culture fit".
- Leniency and central tendency. Some leads rate everyone high to avoid hard conversations. Others cluster everyone in the middle. Either way the rating stops discriminating between real differences in the work.
Why don't awareness training or a better form fix it?
Bias lives in the judgment, where a template can't reach it, and telling people to "just be objective" barely moves ratings. Worse, it can backfire: when an organisation's values stressed meritocracy, managers gave male employees bonuses about $46 higher on average than identically-performing women ($418.80 vs $372.40), the "paradox of meritocracy" (Castilla & Benard, Administrative Science Quarterly, 2010). Believing you're impartial gives you permission to act on bias.
The durable lever is redesigning the process. Unconscious bias is hard and costly to train away (diversity workshops show limited and sometimes counter-productive effects), so the reliable move is to de-bias the organisation instead of the individual (Bohnet, What Works, 2016). Besides directional bias, you're fighting noise: random variability between equally competent raters judging the same work. Structured, criterion-first, independent judgment ("decision hygiene") reduces both (Kahneman, Sibony & Sunstein, Noise, 2021). A nicer form leaves all of that untouched.
How do you design the process to contain bias?
Put the guardrails before and around the judgment, where they can catch it. Four do the heavy lifting:
- Fix the criteria before you review. Define, in observable behaviours, what "good" looks like at each level, and write it down before the cycle (CIPD, Performance management). Once you set the bar in advance, it can't shift to fit the person in front of you.
- Judge evidence against the shared bar: gather first, score second. Collect concrete examples tied to the criteria ("renegotiated scope when the client moved the goalposts"), then rate. The moment a proposed score is visible, everyone anchors to it. Independent inputs dilute a single rater's slant.
- Cover the whole period and more than one viewpoint. Running notes across engagements beat end-of-cycle recall (the antidote to recency). Inputs from the different leads and peers who saw the work, weighted by proximity rather than the org chart, dilute any one rater's idiosyncrasy.
- Calibrate to catch outliers. Compare ratings across reviewers and levels to surface the lenient lead, the harsh one, and the idiosyncratic rater before anyone makes a decision. Use calibration as a noise-and-bias check. It is no tool for massaging numbers or filling a quota.
How do you de-bias the written review, and where does AI help?
Swap vague or loaded language for evidence, then check the wording before it ships. In the written review, bias gets encoded into the record: personality words ("abrasive", "not partner material", "a joy to work with") instead of behaviours, different vocabulary for the same act depending on who did it, praise that's all warmth and no substance. Make every claim point to a concrete example against the shared bar, then read the final text for words that judge the person rather than describe the work.
AI earns its place here as a human-in-the-loop check on wording and consistency, and never as the thing that assigns the rating. A model is good at flagging vague adjectives, loaded language, and uneven tone across a batch of reviews so a human can rewrite them against the evidence. It can't judge a person's year fairly or answer for the verdict, and if you outsource the rating to it, you hide the bias behind a confident sentence. Use it to tighten the language. Keep the judgment with the people who own it.
How do you check whether your reviews are actually fair?
Audit the outputs, because you can't trust a process you can't see. After the cycle, look at rating patterns by reviewer, by level, and by group. If the same leads always run high or low, if one client team always scores above the rest, or if a particular group is consistently rated down, your guardrails aren't holding (CIPD, Performance management: Could do better?). Verify that the process is fair, transparent and inclusive with the numbers, instead of assuming it because you wrote a competency framework.
Keep the whole thing developmental, too. Feedback raised performance on average across 607 studies, but over a third of interventions made performance worse (Kluger & DeNisi, Psychological Bulletin, 1996), and after a century of research there's little evidence appraisal on its own improves anything (DeNisi & Murphy, Journal of Applied Psychology, 2017). If a "cleaner" rating leaves people demotivated, you've gained nothing. De-biasing has to make the judgment more accurate and more useful.
Why does removing bias matter more for agencies and consulting boutiques?
In a project-based firm, no single manager watched someone all year. People are staffed across engagements and rated by whichever leads happened to run them, so the last client, the most-visible project, or the partner's favourite tends to crowd out everything else, and a promotion or partner-track decision rides on it. Here the 62% rater-idiosyncrasy finding (Scullen, Mount & Goff, 2000) decides whether a promotion case is defensible or political.
So de-biasing belongs in the structure of the cycle instead of a one-off training. A shared bar lets different engagement leads mean the same thing. Running notes and weighted inputs stop a brilliant final sprint or a single difficult client from defining the cycle. Calibration across leads catches the reviewer whose ratings say more about them than about the people they rated. And because the result decides who makes the partner track, defensible guardrails are also how a boutique keeps the senior talent it can least afford to lose. People read an opaque or unfair verdict as exactly that, and they leave over it. Build the guardrails light enough to run between billable hours, or they won't run at all.
A quick self-check: how bias-resistant is your review process?
Score one point per "yes". Six or seven and your process contains bias. Four or fewer and the rating is still mostly the rater.
- What "good" looks like at each level is written down in observable behaviours before the cycle starts.
- Reviewers gather concrete examples first and score against the shared bar second.
- The rating draws on the whole period (running notes) instead of the last engagement alone.
- More than one lead/peer input feeds the rating, weighted by who saw the work.
- Ratings are calibrated across reviewers to catch the lenient, harsh, and idiosyncratic ones.
- The written review is checked for vague or loaded language (AI can flag it, and a human rewrites).
- After the cycle you audit rating patterns by reviewer, level, and group, and act on what you find.
FAQ
Can you completely remove bias from performance reviews?
No, and you should be suspicious of any process that claims to. Bias is built into human judgment. The realistic, honest goal is to reduce and contain it so the rating reflects the work rather than the rater. Most of a single score is the rater (about 62% rater idiosyncrasy vs ~21% performance, Scullen, Mount & Goff, 2000), so your lever is structure: criteria, evidence, multiple inputs, calibration and audit. Nobody can promise objectivity.
Doesn't unconscious-bias training fix this?
Mostly not. Training people to "be objective" barely moves ratings, and framing the firm as a pure meritocracy can increase biased decisions (Castilla & Benard, 2010). Unconscious bias is hard and costly to train away, so the reliable fix is to redesign the process rather than the person (Bohnet, What Works, 2016). Use training to explain the guardrails. It can't be the guardrail itself.
What's the single biggest bias in project-based reviews?
Usually recency combined with the idiosyncratic rater effect: the last engagement dominates (Steiner & Rain, 1989) and a big share of the score is the particular lead who wrote it (Scullen, Mount & Goff, 2000). Running notes across the whole period and inputs from more than one lead, weighted by proximity, are the most direct counters.
Is it safe to use AI to remove bias from reviews?
Only as a human-in-the-loop assistant on the wording. Never let it act as the judge. AI is useful for flagging vague adjectives, loaded language, and inconsistent tone across a batch of reviews so a person can rewrite them against the evidence. If a model assigns the rating, it hides the bias behind fluent text and removes accountability. Keep the judgment with the people who own it.
How do we know if our de-biasing is actually working?
Audit the outputs. Look at rating distributions by reviewer, level, and group after each cycle. If the same names always run high or low, or one group is consistently rated down, the guardrails aren't holding (CIPD). Keep it developmental too. A cleaner rating that leaves people worse off is no progress, since over a third of feedback interventions reduce performance when handled badly (Kluger & DeNisi, 1996).