# 6. Hopfield Network model of associative memory¶

**Book chapters**

See Chapter 17 Section 2 for an introduction to Hopfield networks.

**Python classes**

The `hopfield_network.hopfield`

module contains all code required for this exercise.
At the beginning of your exercise solutions, import the contained functions by running

```
from neurodynex.phase_plane_analysis.fitzhugh_nagumo import *
```

You can then simply run the exercise functions by executing the functions, e.g.

```
get_trajectory()
get_fixed_point()
plot_flow()
```

## 6.1. Introduction: Hopfield-networks¶

This exercise uses a model in which neurons are pixels and take the values of -1 (*off*) or +1 (*on*). The network can store a certain number of pixel patterns, which is to be investigated in this exercise. During a retrieval phase, the network is started with some initial configuration and the network dynamics evolves towards the stored pattern (attractor) which is closest to the initial configuration.

The dynamics is that of equation:

In the Hopfield model each neuron is connected to every other neuron (full connectivity). The connection matrix is

where N is the number of neurons, \(p_i^\mu\) is the value of neuron \(i\) in pattern number \(\mu\) and the sum runs over all patterns from \(\mu=1\) to \(\mu=P\). This is a simple correlation based learning rule (Hebbian learning). Since it is not a iterative rule it is sometimes called one-shot learning. The learning rule works best if the patterns that are to be stored are random patterns with equal probability for on (+1) and off (-1). In a large networks (N to infinity) the number of random patterns that can be stored is approximately 0.14 times N.

## 6.2. Exercise: 4x4 Hopfield-network¶

This exercise deals not only with Python functions, but with Python classes and objects. The class `HopfieldNetwork`

implements a Hopfield network. To run the exercises you will have to instantiate the network:

```
from neurodynex.hopfield_network.hopfield import HopfieldNetwork
n = HopfieldNetwork(4) # instantiates a new HopfieldNetwork
```

Note

See the `documentation for the HopfieldNetwork class`

to see all methods you can use on a instantiated HopfieldNetwork.

### 6.2.1. Storing patterns¶

Create an instance of the `HopfieldNetwork`

with N=4. Use the `make_pattern`

method to store a pattern (default is one random pattern with half of its pixels *on*) and test whether it can be retrieved with the `run`

method:

```
n.run() # Note: this will fail with a RuntimeError if no patterns have been stored before
```

The `run`

method, by defaults, runs the dynamics for the first pattern with no pixel flipped.

### 6.2.2. Question: Capacity of the 4x4 network¶

What is the experimental maximum number of random patterns the 4x4 network is able to memorize?

Store more and more random patterns and test retrieval of some of them. The first few patterns should be stored perfectly, but then the performance gets worse.

Does this correspond to the theoretical maximum number of random patterns the network should be able to memorize?

## 6.3. Exercise: 10x10 Hopfield-network¶

### 6.3.1. Question: Capacity of the 10x10 network¶

Increase the network size to 10x10 and repeat the steps of the previous exercise.

### 6.3.2. Question: Error correction¶

Instatiate a network and store a finite number of random patterns, e.g. 8.

How many wrong pixels can the network tolerate in the initial state, such that it still settles into the correct pattern?

Note

See the documentation for the `run method`

to see how to control which percentage of pixels is flipped.

### 6.3.3. Question: Storing alphabet letters¶

Try to store alphabetic characters as the relevant patterns. How good is the retrieval of patterns? What is the reason?

Note

See the documentation for the `make_pattern method`

on how to store alphabet characters.

## 6.4. Exercise: Bonus¶

Try one of the preceding points in bigger networks.

Try downloading the source code for the network, and modify it by adding a smooth transfer function *g* to the neurons. A short introducion on how to run the downloaded file can be found here.