Multi Layered Networks

The Hopfield network has no internal structure with respect to connections and is homogenous.

However NNs are usually structured often by connecting several layers of neurons together .

The first layer is usually reserved for input patterns (as we have seen in the Perceptron). Every neuron of this layer sends out connections: to every neuron of the next layer. This continues until the last layer has been reached whose activity pattern constitues the output of the NN.

NB Each individual layer can perform a partial transformation of the activity pattern of the preceding layer .

For the Perceptron we saw that a serious limitation of possible transformations between input and output occurs (remember linear seperability of input classes was required).

Hence we would like to overcome the limits of the Perceptron by connecting several layers in series and concatenating their transformations.

In contrast to Perceptron and Hopfield net a layered feed-forward network contains hidden units that are not directly connected to input or output lines. Therefore the activity state of these neurons cannot be directly affected by the outside world but can only be influenced indirectly through the internal circuitry of the network. The Perceptron learning proof cannot be extended to the case of thesee networks.

In these cases the output units are only indirectly affected by input signals and it is not clear which of the weights are responsible for the bad result and how they are to be changed.

This is known as the credit assignment problem. and led to the demise of NNs in the 60s. The Backpropagation of errors algorithm provided an elegant solution to this problem.

--