Why Model Collapse should concern us all.
Scary but true. AI Enterprises are running out of training data for their foundation models like Gen AI. There’s a shortage of this costly asset according to Scale AI CEO, Alexander Wang, speaking recently at the Cisco AI Summit.
For global majority groups, if the bias already existing in AI is a concern, the future could be more bleak. But that’s not just the concern that will affect everyone.
Firstly, where to get new diverse data to help train existing LLMs. That’s an issue. This is less about hallucinations and more about a default exclusion of knowledge, because its unobtainable. Some groups are going after the notes of academics, and experts.
Ergo, not by any stretch is all human knowledge available by AI, yet experts by 2030 would want us to believe so. If it isn’t in AI it doesn’t exist.
One solution is that this will force multiple LLMs to flourish offering more bespoke personal knowledge from specifically ingested data. This in turn will create knowledge silos.
You’ve heard the stats: if one in four people on this planet in 2050 will be from Africa (see Guardian Newspaper). How will LLMs will be geared to assist them? Herein, not only lies a huge business opportunity, but a means to correct the present for future generations.
Secondly, the pressing problem of Model Collapse. Imagine you’ve left the acquisition of global knowledge to AI. Our reliance is so much that its answers that you and I use is regurgitated into posts and blogs, that are picked up again by Gen AIs.
It becomes an endless cycle of feeding itself, as human knowledge grows out of their purview. There may be a new dance style out of Africa, started in a remote town, that surpasses Afro Beat, yet Gen AI doesn’t pick this up.
Modern economist have long aired views, now being considered by the Trump administration, about how a country’s wealth and performance can be measured beyond GDP. Yet if this isn’t the norm in LLMs, it’ll keep spurting out answers it considers true, and invariably it will be its own.
The model is collapsing; it’s a regression to the mean.
What might be other concerns? If Artificial General Intelligence becomes the norm, the pandora’s box that opens is a concern that will be more difficult to right itself once it’s on the market. I write about this in the forthcoming edition of Representology.
