The Initial version of CLP in CPU backend

[elvinhajizada]

Issue Number: #706

Objective of pull request: As a user, I want to be able to use the CLP algorithm in Lava, to learn continually from a stream of data in Loihi chip.

Pull request checklist

Your PR fulfills the following requirements:

• [x] Issue created that explains the change and why it's needed
• [x] Tests are part of the PR (for bug fixes / features)
• [ ] Docs reviewed and added / updated if needed (for bug fixes / features)
• [x] PR conforms to Coding Conventions
• [x] PR applys BSD 3-clause or LGPL2.1+ Licenses to all code files
• [x] Lint (flakeheaven lint src/lava tests/) and (bandit -r src/lava/.) pass locally
• [x] Build tests (pytest) passes locally

Pull request type

Please check your PR type:

• [ ] Bugfix
• [x] Feature
• [ ] Code style update (formatting, renaming)
• [ ] Refactoring (no functional changes, no api changes)
• [ ] Build related changes
• [ ] Documentation changes
• [ ] Other (please describe):

What is the current behavior?

• We don't have any continual learning algorithm in Lava

What is the new behavior?

This PR is the first step in the full implementation of CLP. With this version of CLP, three base processes are implemented in the CPU backend and have been tested in terms of the following behaviors:

• PrototypeLIF process generates the correct bAP signal if it receives a 3rd-factor input
• PrototypeLIF process overwrites y1 trace with the value of the received 3rd-factor input, every time such an input is injected
• The novelty can be detected from the coincidence between the presence of input and the absence of a subsequent output
• Readout process outputs the predicted label
• Readout process labels unlabeled winner neurons with user-provided label
• Readout process checks the correctness of the predicted label against the user-provided actual label

In addition, several integration tests are written to test if LearningDense -> PrototypeLIF -> NoveltyDetection intengration works properly

Does this introduce a breaking change?

• [ ] Yes
• [x] No

Supplemental information

[elvinhajizada]

All check have passes

[elvinhajizada]

Integration tests are added

[elvinhajizada]

Just glanced over it, will review in more detail after some questions.

1. Why did you call it prototype_lif? It seems like a 3F learning LIF neuron. Also, is this usable in general? If yes, we should not hide it inside clp folder.
2. What does nsm mean? The process is called Readout, I could not figure out what nsm stands for.
3. I saw print statements in the PyNoveltyDetectorModel. I assume they are left overs from the development?

@PhilippPlank

1. Prototype LIF right now looks like a 3F learning LIF, because it is the first, simple of version. In upcoming releases we will add more features to this neuron model that will make it different from LIF (adaptive learning rate, adaptive threshold etc.) So right now it is just setting the interface that will be filled in later releases.
2. NSM means "Neural State Machines". I planned it as a folder that will host possibly multiple processes. Right now it has only Readout process, because in this release of CLP, we just reading out network prediction. But later, we will need to receive user labels and create supervisory signals to inject into the network. Based on these we will perform both supervised and unsupervised learning. For that purpose we will need other CPU processes as interface between user and network in Loihi 2. So NSM is a general name for bunch of future processes that will control the learning, prediction, labeling states of the system.
3. Yes, right I will remove those. Thanks for pointing out.