search

ezmsg-org / ezmsg

Pure-Python DAG-based high-performance SHM-backed pub-sub and multi-processing pattern
https://ezmsg.readthedocs.io/en/latest/
MIT License
9 stars 5 forks source link

Adds AdaptiveStandardScaler #67

Closed cboulay closed 5 months ago

cboulay commented 7 months ago

The implementation is based on River's AdaptiveStandardScaler. Two implementations are provided and tested. The first, scaler, uses river directly (import river is hidden inside scaler function). The second, scaler_np, uses only numpy.

scaler is about twice as fast for single-channel data but scaler_np is at least twice as faster for multi-channel data.

Either implementation is faster than the current EWM (at least in a 62-channel 500 Hz pipeline I was testing) and requires less memory.

There are two tests. The first tests both functions using known input and output data from river docstrings. The second tests a system created with the AdaptiveStandardScaler unit created from gen_to_unit.

cboulay commented 5 months ago

Shouldn't be merged as-is because it uses gen_to_unit. See #85 .