Hybrid AI simplified.

A simplified explanation of how hybrid AI robot could work

I have a mobile platform that can move a camera about an office environment.  Images from the camera are sent to an image classifier based around a artificial neural network (ANN).  The ANN can produce a list of the n most likely object classifications in the input image sorted be decreasing order if likelihood.

So one day the camera is aimed at my desk and the ANN says it sees a “whale”.  This is clearly wrong.  The solution is to retrain the network with more labeled images or maybe to redesign the network to be multiple networks, one network for each environment such as “office”, “city street” or  “marine” and then tell the robot where it is.  Or maybe the current environment becomes in input to a single ANN?   In either case correcting the error involves retaining and hours of number crunching.

But what we’d like is to simply tell the robot (using something close to English) “Whales are marine mammals.  Marine mammals are only found in marine environments.  My office is not a marine environment.”  Humans can learn this way, by reading or being told.

If camera data are processed serially first by the ANN and then by the rule based logic system, then objects classified by the ANN can be culled and re-sorted using explicit rules so the end result might be that “Sharpie Pen” is the most likely object and not “Whale”.  I think humans do something like this.  We use both a kind of visual memory that might be much like a neural network but this is strongly modified by our knowledge and experience.

A more complex arraignment is like the above serial processing but in addition we feed the output of the logic programming system back to the ANN and create a feedback loop.  The feedback is used train the network using reenforcement learning.

The feedback loop approximates what humans do.  When we learn something new we might be told explicitly and then over time internalize this so we no longer have to consciously apply the rules.   An example is a music student is told that “a major chord has a root note, a major third above this root, and a perfect fifth above this root note.”  At first, applying this rule is rather slow and laborious but over time she no longer has to “think” and sees the three notes on the page as a major chord.  Using a rule base to provide feedback to an ANN may model this type of learning.   In this type of learning a symbolic rule is pushed into the realm of perception.

Another use of a rule based system within a hybrid AI is to implement memory.  When an object is classified the details along with some metadata can be written as a fact in the rule base.  The facts record the history of the robot’s observations and can be either directly queried or they can be used to trigger actions or behaviors.    We can then ask the system something like “where did you see a coffee mug?”

Lastly there is the possibility that multiple ANNs are connected so that their output layer is routed to the logic programming system.  Perhaps one ANN processes images, another processes text and so on.

Building a system such as described above is an ambitious task.  Research will start by attempting to discover a general way of interfacing existing connectionist and symbolic platforms and then take incremental steps.