chaos-genius / chaos_genius

ML powered analytics engine for outlier detection and root cause analysis.
https://www.chaosgenius.io
MIT License
731 stars 83 forks source link

Error in Analytics: KeyError: "['ds'] not in index" #1189

Open Vchekryzhov opened 1 year ago

Vchekryzhov commented 1 year ago

Error Report

I ran the docker compose file and added the metric from Creating your first KPI video guide. After I Configure Anomaly Detector For Average Order Value i get this error

An error was caught when running analytics. Exception message: KeyError: "['ds'] not in index"

Environment

Additional context

Additional information

Exception traceback ``` Traceback (most recent call last): File "/usr/src/app/chaos_genius/core/anomaly/controller.py", line 552, in _run_anomaly_for_series ) = self._detect_anomaly( File "/usr/src/app/chaos_genius/core/anomaly/controller.py", line 222, in _detect_anomaly return ProcessAnomalyDetection( File "/usr/src/app/chaos_genius/core/anomaly/processor.py", line 90, in predict anomaly_df = self._predict(model) File "/usr/src/app/chaos_genius/core/anomaly/processor.py", line 143, in _predict prediction = model.predict( File "/usr/src/app/chaos_genius/core/anomaly/models/standard_deviation_model.py", line 76, in predict df_anomaly = self._detect_anomalies(df) File "/usr/src/app/chaos_genius/core/anomaly/models/standard_deviation_model.py", line 82, in _detect_anomalies forecasted = forecast[["ds", "yhat", "yhat_lower", "yhat_upper", "y"]].copy() File "/usr/local/lib/python3.8/dist-packages/pandas/core/frame.py", line 3511, in __getitem__ indexer = self.columns._get_indexer_strict(key, "columns")[1] File "/usr/local/lib/python3.8/dist-packages/pandas/core/indexes/base.py", line 5796, in _get_indexer_strict self._raise_if_missing(keyarr, indexer, axis_name) File "/usr/local/lib/python3.8/dist-packages/pandas/core/indexes/base.py", line 5859, in _raise_if_missing raise KeyError(f"{not_found} not in index") KeyError: "['ds'] not in index" ```
KPI Info ``` {'active': True, 'aggregation': 'sum', 'anomaly_frequency': None, 'anomaly_params': {'anomaly_period': 30, 'frequency': 'D', 'model_name': 'StandardDeviationModel', 'sensitivity': 'high'}, 'count_column': None, 'created_at': datetime.datetime(2023, 2, 13, 0, 30, 59, 947482), 'data_source': 2, 'datetime_column': 'datetime_col', 'dimensions': ['Country', 'DayOfWeek'], 'id': 1, 'is_certified': False, 'is_static': False, 'kpi_query': 'select \n' ' "Country",\n' ' "DayOfWeek",\n' ' "ItemTotalPrice",\n' ' CAST(date AS TIMESTAMP) as datetime_col,\n' ' CAST("CustomerID" AS text) as "CustomerID_string"\n' 'from ecom_retail_data', 'kpi_type': 'query', 'metric': 'ItemTotalPrice', 'name': 'Average order value', 'run_anomaly': True, 'scheduler_params': {'anomaly_status': 'failed', 'last_scheduled_time_anomaly': '2023-02-13T00:42:30.560471', 'last_scheduled_time_rca': '2023-02-13T00:35:26.898343', 'rca_status': 'completed', 'rca_time': '11:00:00', 'scheduler_frequency': 'D', 'time': '11:00:00'}, 'schema_name': '', 'static_params': None, 'table_name': '', 'timezone_aware': False} ```
Data Source Info ``` {'active': True, 'connection_status': 'connected', 'connection_type': 'Postgres', 'created_at': datetime.datetime(2023, 2, 13, 0, 25, 44, 624992), 'database_timezone': 'UTC', 'id': 2, 'is_third_party': False, 'last_sync': datetime.datetime(2023, 2, 13, 0, 26, 6, 706465), 'name': 'chaosgenius-public', 'sync_status': 'Completed'} ```
github-actions[bot] commented 1 year ago

Hello there, thanks for opening your first issue. We welcome you to the community!

rafaeldelrey commented 1 year ago

Got the same error here

Samyak2 commented 1 year ago

This is a bug in the standard deviation model (see #1152). The other models should work as usual - we recommend Prophet for most cases.