Morality was forged in the marketplace

What happens to right and wrong when AI no longer needs us to trade with each other?

On Monday I typed my name at the bottom of a one-page statement titled ‘We Must Act Now’, joining Erik Brynjolfsson, Daron Acemoglu, Joseph Stiglitz, Ben Bernanke and dozens of others – among them more than a dozen Nobel laureates. The statement warns that AI may become radically more powerful over the next ten years, driving a transformation of the economy ‘larger than the Industrial Revolution, but unfolding over a vastly shorter time frame’. It calls on economists to help build the incentives, guardrails and institutions to steer the technology well.

Most of my co-signatories worry about jobs and wages. Those are real worries, and I share them. But signing from the southern tip of Africa, I found myself worrying about something stranger: what AI might do to morality itself.

Let me explain.

Where do our morals come from? The Sunday-school answer is: from above. The answer emerging from economics, psychology and evolutionary biology is: from below – from the daily business of making a living together. In two recent papers, the Harvard economist Benjamin Enke reviews a decade of research on what he calls moral boundaries. The foundational idea is that morality is functional. In his words, it ‘evolved as a form of “psychological and biological police” to enforce prosocial behavior and cooperation, including in economic production and exchange’. Groups whose members kept promises, punished cheats and helped their partners outproduced groups whose members did not; our consciences are the enforcement technology that survived.

If morality is a tool for cooperation, it should differ across societies according to who people need to cooperate with. And it does. Everyone loves their family more than a faraway stranger; the moral question is how quickly that love falls away with distance. Economists call people whose care declines slowly universalists, and those whose care is concentrated on kin and clan particularists. Neither is more moral; they simply spend the same moral budget differently. As Enke quips, universalists are great strangers to encounter, while particularists are great friends to have (choose your dinner guests accordingly).

What moves the slope? Need. In societies built on tight clans – where your harvest, your safety and your marriage all depend on kin – loyalty pays handsomely and trusting outsiders pays nothing, and the moral system matches: fierce in-group devotion, revenge, shame. Where people make their living trading with strangers, the moral system tilts towards impartial fairness, internalised guilt and trust in outsiders.

The evidence for this is remarkably wide. Enke coded the folklore of nearly 1,000 preindustrial societies and found that those with markets and money told morally universalist stories. In Ethiopia, Devesh Rustagi compared villages by their distance to historical marketplaces: villagers who live closer – and so trade more often with people they do not know – cooperate more readily with strangers in experiments. In Greenland, traditional hunters cheat anonymous strangers in the same games; their market-employed neighbours do not. And survey data from sixty countries show that cohorts who lived longer under democracy – that great machine for impersonal cooperation – are more universalist than their compatriots who did not.

The pattern is consistent: when making a living requires trusting strangers, we learn to value them. When it does not, we do not.

For most of recorded history the direction of travel was one way. Markets expanded, cities grew, institutions scaled, and the circle of people we needed – and therefore the circle of people we valued – widened from the clan to the village to the nation and beyond. The philosopher Peter Singer called it the expanding circle. An economist would call it a repricing: strangers became useful, and morality adjusted.

Now for the question that one-page statement forces on us. What follows is not a forecast. It is what happens inside a model economists already believe – Guido Tabellini’s 2008 model of how parents decide whether to raise universalist children – when you ask it one new question: what if the strangers stop being human?

In Tabellini’s model, parents pass on values by asking, in effect, two things. Who will my child mostly deal with? And will dealing with distant strangers pay off? If the answers are ‘strangers’ and ‘yes’, they raise universalists. If the answers are ‘kin’ and ‘no’, they teach loyalty and caution. Simulate the model from 1800 and it reproduces the moral history sketched above: as markets and democracies raise the returns to impersonal cooperation, universalism climbs, generation by generation.

Then let AI arrive, and the path forks on a single parameter – whether the technology raises or lowers the returns to impersonal cooperation between humans.

In one branch, AI is the new trade route. It translates languages, verifies reputations, shrinks the cost of dealing with someone you have never met, and pulls more of humanity into productive exchange with itself. The returns to trusting strangers rise, and the circle keeps widening. In the other branch, machines take over the exchange itself. Firms produce with fewer people; algorithms mediate what workers once negotiated; the stranger across the market square becomes economically irrelevant. Nothing dramatic happens at first. But in the model, parents notice that trusting outsiders no longer pays, and invest their children’s moral budget closer to home. The circle sags, one generation at a time.

Of course, a model is not the world. Morality has flywheels the simulation ignores: religions, constitutions, schools, the sheer human pleasure of company. Values move over generations, not quarters, which leaves time to steer. That is precisely why the statement says ‘now’.

But the mechanism is anything but hypothetical, and we need not wait for 2100 to observe it. Cue South Africa. A third of South Africans who want to work cannot find it; among the young, the share is closer to half. That is millions of people whom the production side of the economy, in the coldest sense of the phrase, does not need. The functional theory of morality predicts what should follow – trust retreating, circles tightening, the stranger recast as a threat – and two decades of stagnation, eroding social trust and periodic xenophobic violence read uncomfortably like the prediction. Benjamin Friedman made the general point in The Moral Consequences of Economic Growth: broadly shared growth buys tolerance and generosity; stagnation calls in the loan. On this one dimension, South Africa may already be a preview of the AI future the statement warns about.

I remain optimistic, as always. Perhaps the machines will release us from the dreary office and its spreadsheets into the social spaces where we cooperate more, not less. We’ll do Hyrox. We’ll go to yoga. We’ll play padel, sing in choirs, run parkruns, braai with the neighbours we currently only greet over the wall. We’ll have more time to watch World Cup fixtures, even if they expand the tournament to 64 teams!

We have flattened that curve before. Marketplaces did it. Constitutions did it. A dominee or two probably helped. The wide circle we now take for granted was built, transaction by transaction, because we needed each other. The machines will decide a great deal in the coming decades. Whether we keep needing each other is still ours to choose.