Racial and Ethnic Bias in Letters of Recommendation in Academic Medicine: A Systematic Review

Purpose: Letters of recommendations (LORs) are key components of academic medicine applications. Given that bias against students and trainees underrepresented in medicine (UIM) has been demonstrated across assessment, achievement, and advancement domains, the authors reviewed studies on LORs to assess racial, ethnic, and UIM differences in LORs. Standardized LORs (SLORs), an increasingly common form of LORs, were also assessed for racial and ethnic differences.

Method: A systematic review was conducted for English-language studies that assessed racial or ethnic differences in LORs in academic medicine published from database inception to July 16, 2023. Studies evaluating SLORs underwent data abstraction to evaluate their impact on the given race or ethnicity comparison and outcome variables.

Results: Twenty-three studies describing 19,012 applicants and 41,925 LORs were included. Nineteen studies (82.6%) assessed LORs for residency, 4 (17.4%) assessed LORs for fellowship, and none evaluated employment or promotion. Fifteen of 17 studies (88.2%) assessing linguistic differences reported a significant difference in a particular race or ethnicity comparison. Of the 7 studies assessing agentic language (e.g., "strong," "confident"), 1 study found fewer agentic terms used for Black and Latinx applicants, and 1 study reported higher agency scores for Asian applicants and applicants of races other than White. There were mixed results for the use of communal and grindstone language in UIM and non-UIM comparisons. Among 6 studies, 4 (66.7%) reported that standout language (e.g., "exceptional," "outstanding") was less likely to be ascribed to UIM applicants. Doubt-raising language was more frequently used for UIM trainees. When SLORs and unstructured LORs were compared, fewer linguistic differences were found in SLORs.

Conclusions: There is a moderate bias against UIM candidates in the domains of linguistic differences, doubt-raising language, and topics discussed in LORs, which has implications for perceptions of competence and ability in the high-stakes residency and fellowship application process.