Why South Africa needs a data observatory

Economic growth requires better information about what's actually happening

In 1602, Dutch merchants pooled their capital to form the Vereenigde Oostindische Compagnie – the VOC. What made the venture revolutionary was not the ships or the spice. It was the information: prices in distant ports, harvest conditions, currency movements, political risks. As the historian Steven Marks has argued, the profit motive is ancient – but it was the expanding capacity to gather and analyse information that turned trade into capitalism. Every great leap – double-entry bookkeeping, the printing press, the telegraph, the internet – was, at root, an information revolution.

Large language models are the latest turn in this long story. ChatGPT, Gemini, Claude – these tools democratise the kind of analysis that was once the preserve of expensive professionals. A small business owner in Soweto can now interrogate a data set, draft a contract, or evaluate a market opportunity in ways that previously required a consultant. The arbitrage gap – between those who have information and those who do not – narrows dramatically. That is, historically speaking, how economies grow.

The point? The very technology that accelerates economic activity also makes that activity harder to count. And counting it accurately has never been more important for South Africa.

Consider a teacher in Limpopo preparing her matric class for the final examinations. A year ago she would have directed struggling students to a tutoring centre at R500 a month, or bought supplementary workbooks from a commercial publisher. Both transactions appear neatly in GDP. Now she uses Claude to generate personalised practice problems, mark essays with detailed feedback, and produce revision notes tailored to each student’s weak points. Her students’ results improve. But the tutoring centre loses customers, the publisher sells fewer workbooks, and measured GDP in education services may actually fall. The real output has increased. The national accounts cannot see it.

This example captures a broader problem. As economies mature, services – finance, software, consulting, healthcare, education – account for a larger share of output. Services are notoriously difficult to measure, and AI is about to make it much worse. When a single subscription can replace a paralegal, a copywriter, and a data analyst simultaneously, the standard framework of counting market transactions breaks down.

A broken estimate of economic output can really hurt an economy, especially one that stands at a fork in the road. The RMB/BER Business Confidence Index, published a few weeks ago, shows that confidence has climbed to levels last seen around the pre-2008 period, with the post-COVID rebound the only comparable jump. But such confidence is not reflected in GDP statistics – or projections. Why?

In a new working paper, Helanya Fourie and I examine the revision history of South Africa’s GDP and find that StatsSA tends to understate booms and overstate downturns. We also show that sectoral revisions are far noisier – agriculture can see a single quarter’s growth revised by more than ten percentage points as harvest data trickles in. Or consider the load-shedding example. Over 6,500 productive hours lost in 2023 must surely have imposed a sharp fall in output. Yet real GDP edged up by 0.7 per cent. When electricity supply returned to uninterrupted in 2024, growth stayed at 0.6 per cent rather than rebounding. The mismatch should invite deeper scrutiny. It also does not help that the demographic benchmarks underpinning these estimates are themselves unreliable – and the troubled 2022 census raises exactly that concern.

For firms deciding where to invest, for the Reserve Bank setting interest rates, and for a government defending its fiscal credibility, these are not abstract statistical quibbles. Get the numbers wrong, and policy is set in the dark.

The good news is that a remarkable toolkit has emerged. Satellite imagery now tracks agricultural output in near real time; vegetation indices explain crop variation that conventional surveys miss. Night-time lights, measured from space, correlate with economic activity and reveal sub-national patterns where ground-level data is sparse – invaluable for an economy with a large informal sector.

Consider the map below. It shows average annualised GDP per capita growth for South Africa (and Lesotho and Eswatini) between 2012 and 2022. The data, from Estaban Rossi-Hansberg and Jialing Zhang’s paper ‘Local GDP Estimates Around the World’, include information on population, vegetation, nighttime light emissions, urban areas, cropland, forest, water, snow & ice, ruggedness, national GDP per capita, and CO₂ emissions from manufacturing, heavy industry, and transportation. For South Africa, it reveals something quite interesting: growth in places we might typically not associate with improvements in GDP per capita: the mining hub of the Kalahari, the Breede River valley, the northern border of South Africa, the foothills of the Drakensberg. KZN and Gauteng (and Lesotho) seems to have had very little per capita growth.

Rossi-Hansberg and Zhang obviously used data available for the entire world, but one could think of a far more innovative measure. Mobile-phone metadata and payment flows map economic engagement at fine spatial scales. Online prices scraped from retailer websites produce daily inflation indices that capture dynamics quarterly surveys cannot. Textual data from earnings calls, news, and job advertisements yield forward-looking indicators of sentiment and labour demand. And AI itself – the same technology disrupting our measurement frameworks – offers powerful nowcasting models that combine these disparate streams into composite indicators updated weekly or even daily.

What we propose is a real-time economic barometer: a composite index that draws on electricity generation, financial transactions, mobility data, satellite imagery, online prices, and textual indicators. Not a replacement for GDP, but a complement that captures shocks and structural change between quarterly releases.

Building a data observatory to produce this economic barometer requires collaboration between Stats SA, the Reserve Bank, universities, and the private firms whose data feeds would form the backbone of the system. It requires investment in public-sector data science. And it requires governance – data-sharing protocols, privacy safeguards, a trusted intermediary – to ensure credibility and transparency.

None of this is fanciful. The UK’s Office for National Statistics already publishes experimental high-frequency indicators. South Africa has the institutional foundations and the technical talent. What it also has – critically – is the urgent need: we cannot afford to have poor GDP numbers slow investment decisions when the reality is very different.

The VOC merchants understood something that still holds: those who see the economy most clearly are the ones best placed to prosper in it. From the printing press to the large language model, that principle has not changed. South Africa can build the tools to see clearly again.