Boosting as a separate class

breznak commented 5 years ago

I want to separate boosting from SpatialPooler into a standalone class.

What is Boosting

Roughly speaking, boosting is a process of monitoring activation frequency (activity) of columns, and (artificially) supporting~boosting those who lack behind. This theoretically helps better utilize resources by allowing the columns that do not participate (others do their job better) to find theirs niche.

Why?

better logical organization of the code (variables, methods only for boosting won't clutter SP's implementation)
easier to develop and use own, new Boosting methods
- easier to improve existing method
boosting can be easily added to other classes as needed (currently only for SP)
better unit testing

Design

this is an early draft, but:

abstract class Boosting {
  Boosting(...)
  void applyBoost(SDR &columns) const;
  void get*/set* //from current SP
}

BoostingLocal : Boosting
BoostingGlobal: Boosting //2 current implementations from SP

We may want to break the SP's API interface by removing the public set/get methods related to boosting, or we can keep them as wrappers.

Alternative

An interesting alternative would be (optionally) implementing boosting into Connections. That way, it can be used for any (SP, TM, CP) class using connections easily.

Question is, at what level? Currently boosting works with Columns. In Connections it could be either on activity of Segments, or Cells. Connections don't have a concept of Columns. Or can we extend Connections to understand Collumns, that could be also useful.

TL;DR:

[ ] ok to implement Boosting as a separate class?
[ ] should Connections support concept of Columns?
[ ] can boosting be applied directly in Connections (to Cells, Segments,..)?

ctrl-z-9000-times commented 5 years ago

ok to implement Boosting as a separate class?

Something to consider: the Spatial Pooler has several boosting methods:

Homeostatic boosting (this is what I think you're talking about)
minPctOverlapDutyCycles - find cells which don't activate enough and increase permanences of all synapses in their proximal segment.
raisePermanencesToThreshold - ensures that every cell has enough connected synapses to activate.

Why are there three boosting methods? Because none of them are very good...

The homeostatic boosting equation isn't strong enough. As a cells activation frequency approaches 0, this approaches a fixed constant. If that constant isn't large enough then this boosting will be ineffective. The same limitation also applies when the activation frequency approaches 1. The logarithmic boosting equation has an asymptote & zero crossing so it does not suffer from this limitation.
minPctOverlapDutyCycles works fine, but doesn't kick in until the cell has clearly failed.
raisePermanencesToThreshold doesn't kick in until the cell has clearly failed. I have doubts about this one since with only this mechanism a failing cell would need 100% overlap to activate.

can boosting be applied directly in Connections (to Cells, Segments,..)?

The Temporal Memory also uses the connections, so we need the option to disable boosting for that. Also as it's written now, the Connections doesn't know when a cell or segment activates.

should Connections support concept of Columns?

IMO no. I think that the Connections should model the segments on a dendrite, but that how the dendrite responds & how the cell incorperates the input should not be in the connections class. Someday I hope to use the Connections class to model the basal ganglia & thalamus, which don't have mini-columns.

breznak commented 5 years ago

Thanks for comment, more follows:

Something to consider: the Spatial Pooler has several boosting methods: Homeostatic boosting (this is what I think you're talking about)

I'm not sure what you mean by this one.

minPctOverlapDutyCycles - find cells which don't activate enough and increase permanences of all synapses in their proximal segment. (*2)

I mean this by "boosting" here. Monitors for cells that fail desired activation frequency.

raisePermanencesToThreshold - ensures that every cell has enough connected synapses to activate.

correct, this is other boost option, and is already provided by Connections.

The Temporal Memory also uses the connections, so we need the option to disable boosting for that. Also as it's written now, the Connections doesn't know when a cell or segment activates.

Boosting would be made available by connections, same as raisePermanencesToThreshold is already there. It's on the user (SP, TM..) to use the boosting if desired.

Ad "Connections does not know..cell activates" , Boosting class would take care of that.

Why are there three boosting methods? Because none of them are very good...

but that decision/evaluation is not a task of this PR, the PR would be about separation of the logic (of *2) boosting method. If we later decide to alter/remove the method, the easier it would be.

actually, pivoting 180degrees, I'd personally prefer to get rid of the current model of Boosting.

We could implement a concept of "energy starvation" for Cells/ Segments (something like synapse decay which exists(ed?)), and delete under-used cells. This might even help #267 . The second part would be spawning a new Cells (segment) (randomly, in place of "hotspots",..?) IMHO this would be a more plausible model of "activation-frequency-boosting", whith some benefits.

OT:

[on Connections modelling Columns] Someday I hope to use the Connections class to model the basal ganglia & thalamus, which don't have mini-columns.

ok, Thalamus does not. But all current users, SP, TM, CP do have concept of Columns, so I guess having it in Conn would make their implementation easier.

we could have optional concept of Columns in Connections, if that can be achieved without negatively affecting the performance
or do a new derived class Columns: Connections which would be used by SP,TM as needed. (I'll start a separate thread for this, if we want to discuss )

breznak commented 5 years ago

PS: even with

We could implement a concept of "energy starvation" for Cells/ Segments ...[and remove Boosting]

I would like to keep the functionality of boosting there, for other, more high-level purposes. Such as manula/emotions/anomaly/... guided boosting.

dkeeney commented 5 years ago

I don't have much to add when it comes to the algorithms. I am concerned however that we might deviate too much from the Numenta flagship SP and TM pair of algorithms as documented in their papers. They seem to be what everything is comparing their implementations to.

I personally would rather see a separate(s) class that you can do your best to improve on the SP TM pair in much the same way that BacktrackingTM is an alternative implementation of the TM. By doing that we still have the basic SP & TM as a standard and you are free to do whatever you want without having to worry about the Numenta API.

ctrl-z-9000-times commented 5 years ago

all current users, SP, TM, CP do have concept of Columns

Actually only the TM has a concept of mini-columns. The SP's output can be used as mini-columns but the SP itself is agnostic to how its output is used. The CP does not have mini-columns at all, the name is misleading.

ctrl-z-9000-times commented 5 years ago

Homeostatic boosting

I'm not sure what you mean by this one.

To quote the documentation:

SP parameter boostStrength:

A number greater or equal than 0, used to control boosting strength. No boosting is applied if it is set to 0. The strength of boosting increases as a function of boostStrength. Boosting encourages columns to have similar activeDutyCycles as their neighbors, which will lead to more efficient use of columns. However, too much boosting may also lead to instability of SP outputs.

This is what I referred to as homeostatic boosting.

SP parameter minPctOverlapDutyCycles:

A number between 0 and 1.0, used to set a floor on how often a column should have at least stimulusThreshold active inputs. Periodically, each column looks at the overlap duty cycle of all other column within its inhibition radius and sets its own internal minimal acceptable duty cycle to: minPctDutyCycleBeforeInh * max(other columns' duty cycles). On each iteration, any column whose overlap duty cycle falls below this computed value will get all of its permanence values boosted up by synPermActiveInc. Raising all permanences in response to a sub-par duty cycle before inhibition allows a cell to search for new inputs when either its previously learned inputs are no longer ever active, or when the vast majority of them have been "hijacked" by other columns.

breznak commented 5 years ago

I am concerned however that we might deviate too much from the Numenta flagship SP and TM pair of algorithms as documented in their papers. They seem to be what everything is comparing their implementations to.

This proposal is merely about code reorganization, better modularity. The implementation/functionality will remain exactly the same as numenta's/current. Being modular/having a separate class for Boosting will make it easier to implement the new methods with smaller changes (no need to rewrite the whole SP, just subclass Boosting, etc)

breznak commented 5 years ago

This is what I referred to as homeostatic boosting.

oh, sorry, I've confused activeDutyCycles (homeo, boostStrength), and overlap duty cycles

htm-community / htm.core