elastic / ml-cpp

Machine learning C++ code
Other
149 stars 62 forks source link

[ML] Enable time-limited eager adjustment to time shift events like daylight saving #2691

Open valeriy42 opened 1 month ago

valeriy42 commented 1 month ago

Problem

In many countries around the world, a time shift related to Daylight Saving Time (DST) causes users to change their behavior with respect to UTC. While we continue to improve the machine learning algorithms associated with detecting these shifts, it still takes some time to gather evidence that supports the time shift hypothesis. The anomaly detection job may take a few days to adjust to the change. During this time, anomaly detection may produce false positive or false negative alerts.

Enhancement

We allow users to enable a mode for more eagerly accepting the time shift hypothesis (using a new detection rule). This will reduce the burden of evidence the algorithm expects to see in order to accept the hypothesis that there has been a time shift in user behavior. Once the time shift hypothesis has been accepted or the allowed time has elapsed, this eager mode is automatically disabled.