Kohonen Networks

The objective of a Kohonen network is to map input vectors (patterns) of arbitrary dimension N onto a discrete map with 1 or 2 dimensions. Patterns close to one another in the input space should be close to one another in the map: they should be topologically ordered. A Kohonen network is composed of a grid of output units and N input units. The input pattern is fed to each output unit. The input lines to each output unit are weighted. These weights are initialised to small random numbers.

Learning in Kohonen Networks

The learning process goes as roughly as follows:

• initialise the weights for each output unit
• loop until weight changes are negligible
• • for each input pattern
  • • present the input pattern
    • find the winning output unit
    • find all units in the neighbourhood of the winner
    • update the weight vectors for all those units
  • reduce the size of neighbourboods if required

The winning output unit is simply the unit with the weight vector that has the smallest Euclidean distance to the input pattern. The neighbourhood of a unit is defined as all units within some distance of that unit on the map (not in weight space). In the demonstration below all the neighbourhoods are square. If the size of the neighbourhood is 1 then all units no more than 1 either horizontally or vertically from any unit fall within its neighbourhood. The weights of every unit in the neighbourhood of the winning unit (including the winning unit itself) are updated using

This will move each unit in the neighbourhood closer to the input pattern. As time progresses the learning rate and the neighbourhood size are reduced. If the parameters are well chosen the final network should capture the natural clusters in the input data.