Rethink SP local inhibition

Oblynx / HierarchicalTemporalMemory.jl

A simple, high-level Julia implementation of Numenta HTM algorithms

https://oblynx.github.io/HierarchicalTemporalMemory.jl

MIT License

21 stars 3 forks source link

Rethink SP local inhibition #48

Open Oblynx opened 4 years ago

Oblynx commented 4 years ago

The SP tiebreaker is necessary to enforce strict sparsity control, but it has undesirable sideffects in other properties:

sp(z) becomes stochastic (random seed from hash(z)?)
noise robustness suffers and doesn't improve a lot with training

This points to a need to rethink inhibition, and not do it by comparing against a threshold, but more low-level, by ordering indices and picking the highest activations.

Oblynx commented 3 years ago

Implemented in this commit, but only for global inhibition.