Generating Coherent Patterns of Activity from Chaotic Neural Networks

Sussillo and Abbott improved the echo-state network (ESN) with FORCE algorithm, which is more robust again noise and perturbations. They also provided the analysis of train dynamics and reverse-engineering of network dynamics for reservoir network generating coherent patterns.

• Due to the chaotic nature, conventional BP fails in training RNN. Reservoir computing, using the echo-state property of RNNs, trains the output weight without manipulating the random reservoir current connectivity, and is capable to performing different tasks.

• Compared with ESN (Jaeger 2004), FORCE algorithm is more robust against perturbations, and is more capable of multi-task continual learning.

• The training dynamics and property of well-trained network were investigated.

  • During training, weights along eigenvectors with larger eigenvalues converge first, and the network encode task ability with multiple eigen-modes.

  • Well trained networks have fixed projections along first few eigenvectors and accompanied with flexible projections alone rest of eigenvectors.