Artificial intelligence obviously cannot learn continuously. Rather, the training of an AI always has to start from the beginning if new data is to be added. The backgrounds.
AI models cannot learn over time like humans can. This is the result of a current one studywhich appeared in the science magazine Nature was published. Accordingly, AI models can no longer update themselves and learn from new data after their initial training phase.
Correctly means: If new data is to be added, training must start from scratch with the larger data set. This is forcing technology companies to invest billions of dollars to further develop their models.
AI: Training with new data starts from scratch
Most AI systems are so-called neural networks whose functionality is modeled on the human brain. They have artificial neurons to process information.
In order for the interaction between humans and artificial intelligence to work, an AI must first be trained with data. It can then use this information to respond to text input, as is the case with ChatGPT, for example.
However, according to the researchers, once the neurons complete the initial training, they are no longer able to update and learn from new data. This means that most AI models have to be retrained from scratch when new data is to be added.
A team of researchers from the University of Alberta in Canada has therefore investigated whether artificial intelligence can be adapted so that a system continuously learns. They first noticed that the artificial neurons reach their limits relatively quickly. Study leader Shibhansh Dohare said: “If you think of it like your brain, then 90 percent of the neurons are dead. There just isn’t enough left to learn.”
Artificial Intelligence: Performance decreases with increasing training
The researchers first tested AI using a database consisting of 14 million simple images. But instead of training the system once and then testing it, they retrained it after each pair of images. The result: After several thousand training sessions, the model got worse and worse. More and more neurons appeared “dead”.
However, according to the scientists, there may be a way around the problem. They developed an algorithm that randomly turned on some neurons after each training session. This reduced the performance to a lesser extent. Shibhansh Dohare says: “If a [Neuron] died, let’s just revive it. Now it’s able to learn again.”
According to the researchers, the algorithm is promising, but needs to be tested with larger AI systems before it can be ensured that it works.
“A continuous learning solution is literally a billion-dollar question,” Dohare said. “A true, comprehensive solution that would allow you to continually update a model would significantly reduce the cost of training those models.”
Also interesting:
- AI changes its own code to launch itself
- AI image generator: Start-up from Freiburg is better than Midjouney and OpenAI
- AI search engine: What is Perplexity AI?
- Study reveals: AI language models cannot develop a life of their own
The article Study: AI cannot continuously learn – training always starts from zero by Fabian Peters appeared first on BASIC thinking. Follow us too Facebook, Twitter and Instagram.
As a tech industry expert, I find this study to be quite concerning and challenging for the field of artificial intelligence (AI). The concept of continuous learning is crucial for AI systems to adapt and improve over time, and the fact that they are unable to do so effectively means that there are limitations to their capabilities.
The idea that AI training always starts from zero implies that there is a lack of persistence in learning and retaining information, which is a significant obstacle to achieving true artificial intelligence. It also highlights the need for more research and development in the area of continuous learning algorithms and techniques.
This study underscores the importance of addressing the limitations of current AI systems and finding ways to overcome them in order to advance the field and unlock the full potential of artificial intelligence. As a tech industry expert, I believe that this research should serve as a call to action for the industry to invest more resources and effort into solving the challenges of continuous learning in AI.
Credits