opensearch-project / anomaly-detection

Identify atypical data and receive automatic notifications
https://opensearch.org/docs/latest/monitoring-plugins/ad/index/
Apache License 2.0
67 stars 73 forks source link

Not blocking detector creation on unknown feature validation error #1366

Closed amitgalitz closed 1 week ago

amitgalitz commented 1 week ago

Description

Currently during detector creation we check for a few different things in relation to the configured feature. If aggregation is valid, if we have at least one feature, that we don't aggregate over a non numerical field with something like average. However we currently fail on any seen exception, we want to change this that if there is an unknown exception that we don't completely block detector creation. This is due to the fact that we might get things like timeout or search cancellation that are related to the cluster, not cause the configuration is wrong. Users should be able to still create the detector and handle any cluster or settings changes they need to later on if it effects the actual detector creation.

Related Issues

Resolves #1365

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 80.04%. Comparing base (5ec87b2) to head (a0aa89f). Report is 1 commits behind head on main.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/opensearch-project/anomaly-detection/pull/1366/graphs/tree.svg?width=650&height=150&src=pr&token=ZPONorT0bX&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project)](https://app.codecov.io/gh/opensearch-project/anomaly-detection/pull/1366?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) ```diff @@ Coverage Diff @@ ## main #1366 +/- ## ============================================ + Coverage 80.02% 80.04% +0.01% - Complexity 5664 5674 +10 ============================================ Files 533 533 Lines 23434 23438 +4 Branches 2334 2334 ============================================ + Hits 18753 18760 +7 - Misses 3572 3578 +6 + Partials 1109 1100 -9 ``` | [Flag](https://app.codecov.io/gh/opensearch-project/anomaly-detection/pull/1366/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | Coverage Δ | | |---|---|---| | [plugin](https://app.codecov.io/gh/opensearch-project/anomaly-detection/pull/1366/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | `80.04% <100.00%> (+0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files with missing lines](https://app.codecov.io/gh/opensearch-project/anomaly-detection/pull/1366?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | Coverage Δ | | |---|---|---| | [.../rest/handler/AbstractTimeSeriesActionHandler.java](https://app.codecov.io/gh/opensearch-project/anomaly-detection/pull/1366?src=pr&el=tree&filepath=src%2Fmain%2Fjava%2Forg%2Fopensearch%2Ftimeseries%2Frest%2Fhandler%2FAbstractTimeSeriesActionHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project#diff-c3JjL21haW4vamF2YS9vcmcvb3BlbnNlYXJjaC90aW1lc2VyaWVzL3Jlc3QvaGFuZGxlci9BYnN0cmFjdFRpbWVTZXJpZXNBY3Rpb25IYW5kbGVyLmphdmE=) | `95.07% <100.00%> (+0.04%)` | :arrow_up: | ... and [11 files with indirect coverage changes](https://app.codecov.io/gh/opensearch-project/anomaly-detection/pull/1366/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project)
opensearch-trigger-bot[bot] commented 1 week ago

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/anomaly-detection/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/anomaly-detection/backport-2.x
# Create a new branch
git switch --create backport/backport-1366-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4c545ab27d78be4859afde612719a4367c342f9e
# Push it to GitHub
git push --set-upstream origin backport/backport-1366-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/anomaly-detection/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-1366-to-2.x.