Last year, MIT developed an AI/ML algorithm capable of learning and adapting to new information on the fly, not just during the initial training phase. These “fluid” neural networks (in Bruce Lee’s sense) literally play 4D chess—models that require time-series data to operate—which makes them ideal for use in time-sensitive tasks such as pacemaker monitoring, weather forecasting, investment forecasting, or autonomous vehicle navigation. But the problem is that data throughput has become a bottleneck, and scaling these systems has become prohibitively expensive, computationally.
On Tuesday, MIT researchers announced that they have developed a solution to this limitation, not by expanding the data pipeline, but by solving a differential equation that has baffled mathematicians since 1907. Specifically, the team solved “the differential equation underlying the interaction between the two. neurons through synapses… to open up a new kind of fast and efficient artificial intelligence algorithms.
“The New Machine Learning Models We Call “CfC” [closed-form Continuous-time] to replace the differential equation that defines the neuron’s computation with a closed-form approximation, preserving the beautiful properties of fluid networks without the need for numerical integration,” MIT professor and CSAIL director Daniela Rus said in a Tuesday press release. “CfC models are causal, compact, explainable, and efficiently trainable and predictive They pave the way to reliable machine learning for safety-critical applications.
For those of us without a PhD in really hard math, differential equations are formulas that can describe the state of a system at various discrete points or stages throughout a process. For example, if you have a robot arm moving from point A to B, you can use a differential equation to find out where it is between two points in space at any point in the process. However, solving these equations for each step quickly becomes computationally expensive as well. MIT’s “closed-form” solution solves this problem by functionally modeling the entire system description in a single computational step. As the MIT team explains:
Imagine if you had a complete neural network receiving driving input from a car-mounted camera. The network is trained to generate results such as the steering angle of the car. In 2020, the team solved this using fluid neural networks with 19 nodes, so that 19 neurons plus a small sensing module could drive the car. A differential equation describes each node of this system. With a closed-form solution, if you replace it in this network, it would give accurate behavior because it is a good approximation of the actual dynamics of the system. This allows them to solve the problem with even fewer neurons, which means it would be faster and computationally cheaper.
By solving this equation at the neural level, the team hopes that they will be able to create models of the human brain that measure millions of neural connections, something that is not possible today. The team also notes that this CfC model may be able to learn visual learning in one environment and apply it to a completely new situation without additional work called out-of-distribution generalization. Current generation models really can’t do this, and this will prove to be a major step towards tomorrow’s generalized AI systems.
All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories contain affiliate links. If you purchase something through one of these links, we may earn an affiliate commission. All prices are correct at time of publication.
