AI and Equity: Ensuring Your AI Tools Don't Perpetuate Bias

For mission-driven nonprofits, equity isn't optional. It's foundational. But AI adoption creates a hidden risk: automating the biases embedded in historical data, scaling them, and making them harder to see.

An equity-centered approach to AI means being deliberate about which tools you use, how you implement them, and what outcomes you monitor.

How AI Perpetuates Bias: Three Mechanisms

1. Biased Training Data

AI learns from the past. If your historical data reflects inequality, the AI will replicate it.

Example: Your nonprofit runs mentorship programs. Historically, you've matched volunteers with youth, and those matches have disproportionately favored youth from higher-income neighborhoods (because those communities had more volunteer capacity). If you train an AI model on this data and ask it to predict "good matches," it learns the same pattern: match people with youth from affluent areas. The AI has automated historical inequity.

Another example: A healthcare nonprofit uses AI to identify uninsured patients for enrollment in support programs. The training data comes from past enrollment records. Historically, certain communities faced language barriers and were underrepresented in enrollment. The AI learns to deprioritize people in those communities, perpetuating the disparity.

2. Proxy Variables

Sometimes the biased variable isn't directly in the data, but you can infer it from other variables.

Example: You're using AI to target donors for a capital campaign. The AI considers: zip code, wealth indicators, age, education, prior giving. You didn't include race. But zip code is often a proxy for race in America—neighborhoods are segregated, and the AI learns to target more affluent, whiter areas. The tool has effectivelylearned a racial bias without race being an explicit variable.

How to identify proxy variables: Ask: "What historical group is this variable correlating with? If I segment the results by race/ethnicity/income, do certain groups get systematically different outcomes?"

3. Optimizing the Wrong Metric

Sometimes bias comes from what you're asking the AI to optimize for.

Example: You use AI to identify "high-risk" youth for early intervention. You optimize for "likelihood of dropping out of school." But school dropout rates correlate with poverty and over-policing. By optimizing for this metric, you're disproportionately flagging low-income youth and youth of color as "high-risk," regardless of their actual support needs.

Better framing: Optimize for "who would benefit most from mentorship" rather than "who is most at-risk." The outcome is more equitable.

Auditing Your AI Tools for Bias: A Practical Framework

You don't need a data scientist to audit for bias. You need curiosity and basic analytical thinking.

Step 1: Understand the Training Data

Ask the tool vendor or developer:

• What data trained this model? Is it publicly available?
• How representative is that data? Was it collected from a specific geography, demographic, or time period?
• Are there known biases in the training data?
• Have you stress-tested the model for bias? What did you find?

Not all vendors will answer transparently. Those who refuse to are a red flag. Reputable vendors should be able to discuss known limitations.

Step 2: Segment Results by Key Demographics

After implementing an AI tool, look at the outputs by race, ethnicity, gender, income, geography, and any other relevant demographic.

Questions to ask:

• Are outcomes different for different groups? If so, why?
• Is the AI allocating resources, opportunities, or recommendations equitably?
• Who benefits from this AI? Who is disadvantaged?

Example analysis:

Volunteer Matching AI Results (by demographics):

Success (matched to volunteer):
  White volunteers: 89%
  Black volunteers: 72%
  Latinx volunteers: 78%

Finding: Black volunteers are being matched at significantly lower rates.
This suggests the AI is biased against them.
Action: Stop using this tool until it's audited and fixed.

Step 3: Test Edge Cases

Run the same query with different demographic characteristics and see if results change.

Example: A fundraising tool scores prospects for major gift capacity. Input two identical profiles—identical income, wealth, prior giving—but change the name to signal different ethnicity. Does the score change? If so, the tool is biased.

Step 4: Measure Real-World Outcomes

This is the ultimate test. Does using this AI actually improve outcomes for your mission?

Example: You use AI to identify families eligible for a housing assistance program. Track outcomes: Are families being matched appropriately? Are certain groups disproportionately denied assistance? Are there differences in housing stability six months later?

If the AI improves outcomes for everyone and doesn't create new disparities, it's working. If it helps some groups but harms others, it's perpetuating inequality.

Building Equity Into Your AI From the Start

1. Be Intentional About Your Training Data

If you're building a custom AI model (most nonprofits aren't, but some do), source diverse training data. Don't just use your historical records—that perpetuates past biases. Seek data that represents the full diversity of your community.

2. Define Equity-Centered Outcomes

Before implementing AI, define success in equity terms. Not just "improve efficiency" but "improve efficiency while reducing disparities for underserved communities."

Example objectives:

• Increase service access for communities historically underrepresented in our programs by 20%
• Ensure all demographic groups experience equal outcomes from our AI-driven mentorship matching
• Identify barriers faced by marginalized communities and use AI to address them, not entrench them

3. Diversify Your AI Committee

When evaluating and implementing AI, include voices from your community. Staff and volunteers from historically marginalized backgrounds bring perspective that homogeneous teams miss.

Ask them: "Does this feel fair? Does this align with our equity values? What am I not seeing?"

4. Monitor Continuously

Bias doesn't disappear after you deploy an AI tool. It evolves. Set a routine (quarterly minimum) to review outcomes by demographics and ask whether the tool is still advancing your equity goals.

When to Say No to AI

Some AI applications are too risky for equity-centered work:

• Black-box predictive models with no explainability: If you can't understand why the AI made a decision, you can't audit for bias.
• Tools built on biased training data that hasn't been corrected: If the vendor knows about bias and hasn't fixed it, don't use it.
• High-stakes decisions made entirely by AI: Determining program eligibility, allocating resources, making funding decisions—these should always have human review.
• Tools that require data from marginalized communities without benefit: If you're asking vulnerable people to provide data for an AI tool and they don't directly benefit, reconsider.

Communicating About AI and Equity to Your Board and Staff

Use this framing:

"Our community trusts us to use resources and opportunities fairly. When we adopt AI, we need to ensure it supports that promise, not undermines it. That's why we're being intentional about tool selection, monitoring outcomes, and being willing to stop using something if it creates disparities. Here's how we're doing that."

Most board members understand equity as a mission imperative. Frame AI governance as a way to protect that commitment.

Red Flags You Should Have Caught Earlier

If any of these are true, audit immediately:

• ☐ You implemented an AI tool but have never checked outcomes by demographics
• ☐ The tool makes recommendations that humans usually follow without questioning
• ☐ You can't explain how the AI reached a specific decision
• ☐ The tool doesn't have a human override or appeal process
• ☐ Certain demographic groups are consistently advantaged or disadvantaged by the tool
• ☐ You implemented the tool without consulting people from affected communities
• ☐ The vendor couldn't answer basic questions about bias when you asked