We'd prefer them to be evaluated in rolling-window mode. Based on a quick glance at River's documentation, this will most likely mean having to write that ourselves.
Can we implement that via a wrapper class over the metrics offered by River?
if by "Attack Detection Rate" we mean $\frac{TP}{P}$ (where $P = TP+FN$ is the number of actually positive examples), recall is obviously also available — we just need to ensure the class is "cast" into the binary attack/benign
if by "False Alarm Rate" we mean $\frac{FP}{PP}$ (where $PP$ is "predicted positive"), that would be $1 - precision$?
We agreed to use the following:
We'd prefer them to be evaluated in rolling-window mode. Based on a quick glance at River's documentation, this will most likely mean having to write that ourselves.
Can we implement that via a wrapper class over the metrics offered by River?