search

AICoE / prometheus-anomaly-detector

A newer more updated version of the prometheus anomaly detector (https://github.com/AICoE/prometheus-anomaly-detector-legacy)
GNU General Public License v3.0
597 stars 151 forks source link

IndexError: list index out of range #187

Closed psujit775 closed 1 year ago

psujit775 commented 2 years ago

The model is trained and it starts to expose metrics but after some time it starts to give 500: Internal Server Error.

Using official docker image: quay.io/aicoe/prometheus-anomaly-detector:latest

Logs:

2022-08-31 23:24:22,062:INFO:__main__: Initializing Tornado Web App
2022-08-31 23:24:22,770:INFO:__main__: Will retrain model every 720 minutes
2022-08-31 23:24:45,189:ERROR:tornado.application: Uncaught exception GET /metrics (xxx.xxx.xxx.xxx)
HTTPServerRequest(protocol='http', host='xxx.xxx.xxx.xxx:8080', method='GET', uri='/metrics', version='HTTP/1.1', remote_ip='xxx.xxx.xxx.xxx')
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/tornado/web.py", line 1704, in _execute
    result = await result
  File "app.py", line 80, in get
    pc.get_current_metric_value(
IndexError: list index out of range
sesheta commented 1 year ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

sesheta commented 1 year ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

sesheta commented 1 year ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

/close

sesheta commented 1 year ago

@sesheta: Closing this issue.

In response to [this](https://github.com/AICoE/prometheus-anomaly-detector/issues/187#issuecomment-1407548498): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.