mitre / menelaus

Online and batch-based concept and data drift detection algorithms to monitor and maintain ML performance.

https://menelaus.readthedocs.io/en/latest/

Apache License 2.0

66 stars 7 forks source link

Open Anmol-Srivastava opened 1 year ago

Anmol-Srivastava commented 1 year ago

As experimental code, this serves as an outline of the NLP framework rather than final versions. Closes #152

Added

transforms (Representation removed) related to alibi-detect example
basic Detector and KS test Alarm
updated wilds_datasets.ipynb example to reflect testing on civil-comments dataset

design of the KS Alarm scheme -- test occurs feature-wise, but how should drift alarm?

tms-bananaquit commented 1 year ago

Will need to add citations and update references for alibi-detect's paper and/or documentation before merging the experimental branch.

Anmol-Srivastava commented 1 year ago

This is essentially ready to be merged in -- citations are added and the code is stable enough.

This also now closes #153 #154 and can enable proper integration into menelaus

Anmol-Srivastava commented 1 year ago

Now also closes #164