Anomaly detection improvements

[breznak]

Organized workflow for anomaly related issues/enhancements.

• [ ] Swarming optimization of Anomaly
  • [X] Expose anomaly params to OPF for swarming
  • [ ] New Anomaly metric to drive optimization during swarming
  • [ ] Pass extra information to Metric (CLAModel, Anomaly, ..)
  • [ ] minimize= optimization criterion/Metric can be set as Metric instance/ID, not only constructed from string
  • [ ] Tool for anomaly supervised learning/parameter optimization
  • [ ] Anomaly changes TP outputType
• [ ] Improvements
  • [x] New anomaly mode: binary discretization
  • [ ] New anomaly computation - taking unexpected observations into account
  • [ ] Multistep Anomaly predictions
• [ ] Problems
  • [ ] Handle missing anomaly spikes
  • [ ] Boosting creates artificial anomaly spikes
  • [ ] Serialization of AnomalyLikelihood looses state
  • [ ] Anomaly Score jumps in compute() method
  • [ ] Validate Anomaly ComputeStrict (XOR) method

[cogmission]

May I ask if swarming has been refactored out of the OPF? Is it now a stand-alone function?

[breznak]

it's not. There's an issue for that, but without much progress yet I think..

[rhyolight]

It's a big job.

[rhyolight]

One thing we might be missing here is incorporating the anomaly likelihood into the model itself (like the anomaly score).

If a user creates a model and passes data into it, there is a period of time where the anomaly likelihood is always 0.5 until it gathers enough data to start it's windowed calculation. If this model is serialized and restarted, this same learning period applies again. Ideally, this resurrected model would retain the anomaly likelihood data so it could continue where it left off instead of starting with a bunch of 0.5 values upon resurrection.

[breznak]

@rhyolight good point. We might open an issue for that, it would fall to the anomaly_likelihood's responsibility to handle that during de/serialization. Actually, I'm not sure if it's not already handeled? As the prev values are stored in a [] and that would be serialized fine. Same goes for MovingAverage class.

[rhyolight]

It is not currently handled, as I noticed it occurred recently for @anubhavchaturvedi.

[breznak]

https://github.com/numenta/nupic/issues/1908