6. Hopfield Network model of associative memory¶
See Chapter 17 Section 2 for an introduction to Hopfield networks.
hopfield_network.hopfield module contains all code required for this exercise.
At the beginning of your exercise solutions, import the contained functions by running
from neurodynex.phase_plane_analysis.fitzhugh_nagumo import *
You can then simply run the exercise functions by executing the functions, e.g.
get_trajectory() get_fixed_point() plot_flow()
6.1. Introduction: Hopfield-networks¶
This exercise uses a model in which neurons are pixels and take the values of -1 (off) or +1 (on). The network can store a certain number of pixel patterns, which is to be investigated in this exercise. During a retrieval phase, the network is started with some initial configuration and the network dynamics evolves towards the stored pattern (attractor) which is closest to the initial configuration.
The dynamics is that of equation:
In the Hopfield model each neuron is connected to every other neuron (full connectivity). The connection matrix is
where N is the number of neurons, \(p_i^\mu\) is the value of neuron \(i\) in pattern number \(\mu\) and the sum runs over all patterns from \(\mu=1\) to \(\mu=P\). This is a simple correlation based learning rule (Hebbian learning). Since it is not a iterative rule it is sometimes called one-shot learning. The learning rule works best if the patterns that are to be stored are random patterns with equal probability for on (+1) and off (-1). In a large networks (N to infinity) the number of random patterns that can be stored is approximately 0.14 times N.
6.2. Exercise: 4x4 Hopfield-network¶
This exercise deals not only with Python functions, but with Python classes and objects. The class
HopfieldNetwork implements a Hopfield network. To run the exercises you will have to instantiate the network:
from neurodynex.hopfield_network.hopfield import HopfieldNetwork n = HopfieldNetwork(4) # instantiates a new HopfieldNetwork
documentation for the HopfieldNetwork class to see all methods you can use on a instantiated HopfieldNetwork.
6.2.1. Storing patterns¶
Create an instance of the
HopfieldNetwork with N=4. Use the
make_pattern method to store a pattern (default is one random pattern with half of its pixels on) and test whether it can be retrieved with the
n.run() # Note: this will fail with a RuntimeError if no patterns have been stored before
run method, by defaults, runs the dynamics for the first pattern with no pixel flipped.
6.2.2. Question: Capacity of the 4x4 network¶
What is the experimental maximum number of random patterns the 4x4 network is able to memorize?
Store more and more random patterns and test retrieval of some of them. The first few patterns should be stored perfectly, but then the performance gets worse.
Does this correspond to the theoretical maximum number of random patterns the network should be able to memorize?
6.3. Exercise: 10x10 Hopfield-network¶
6.3.1. Question: Capacity of the 10x10 network¶
Increase the network size to 10x10 and repeat the steps of the previous exercise.
6.3.2. Question: Error correction¶
Instatiate a network and store a finite number of random patterns, e.g. 8.
How many wrong pixels can the network tolerate in the initial state, such that it still settles into the correct pattern?
See the documentation for the
run method to see how to control which percentage of pixels is flipped.