Emerging Technology

On night and weekends, I spend time in the garage working on an alternative machine learning approach for use in artificial intelligence.

Despite tremendous progress in AI/ML, current technologies based on end-to-end deep learning turn out to be quite limited in the extent to which they generalize to new contexts or situations. Because of their design, they fail to handle real dynamical interactive effects present in the world. In the case of vision, models are challenged by shadows, motion blur, reflections, and even lighting changes, which can throw them off quite easily. But this limitation does not apply just to vision — it applies to every modality as well as multi-modal, motor and cognitive phenomena.

With the recent impressive performance of large language models (LLMs) many have begun looking to these to form the core of models with hopefully better generalization. However this direction is also a dead end. Language is not a “generalization” of the world — it is an abstraction that is lower dimensional, strictly linear, and also low-fidelity. Too low fidelity for genuine understanding of the worlds interdependent effects. Written language — on which LLMs are based — is even more lossy, as it loses prosody, emphasis, and rate.

The way the vision problem is being defined by most practitioners is highly removed from the vision problem solved by real brain-like systems. Vision in the world is not about pooling across pixel texture patterns. The vision problem is about learning to decompose, to source separate — multiple overlapping factors that influence the way things appear. It's being able to recognize and understand that the shadow from a gate on a dog didn't actually turn it into a striped tiger. Arguably, the capability to decompose reality underlies general intelligence.

To learn more about our approach and its results see our paper Unsupervised Learning from Continuous Video in a Scalable Predictive Recurrent Network (https://arxiv.org/abs/1607.06854).