Training

Ever wondered why AI seems to know so much about some topics but almost nothing about others?

That’s because AI models only know what they’ve been taught - and what gets included in that teaching reflects human choices about what is important and what isn’t.

Today’s generative AI models cant explore the world or gather new information on their own. They only work with what they have seen during development - a process called ‘training’. AI training data comes from a vast variety of human-made content: books, websites, articles, forums, and social media posts that were collected by the companies building these systems.

But this data isn’t neutral. It reflects the choices, values, and power structures of the people and systems that produced it. Some languages and cultures are overrepresented whilst others barely appear. Some regions dominate the conversation. Some topics are filtered out by design, by law, or by organisational policy. Others are simply missing because no one wrote them down, or because they were excluded from the digital record.

This means the model’s world is fundamentally uneven. It knows a lot about some things and very little about others. And that imbalance has real consequences for how these systems understand and represent the world.

What Shapes AI’s Knowledge

The data that trains AI systems carries the fingerprints of human culture and power relations:

Whose voices are included? A model might know extensive details about Silicon Valley startup culture but very little about rural farming communities, not because one is more important, but because one group produces more digital content that gets included in training datasets.

Whose histories are told? AI might reflect dominant narratives about historical events, gender roles, or cultural practices, not because it believes them, but because they were repeated most often in its sources.

What kinds of knowledge are considered valid? For example, a model might know more about Western academic philosophy than Indigenous knowledge systems, not because one is more valuable, but because one is more present in digitised, accessible formats.

Understanding this helps us see AI responses for what they are: not a mirror of the world, but a reflection of the data these systems were given - shaped by culture, power, and what got left out.