uber / orbit

A Python package for Bayesian forecasting with object-oriented design and probabilistic models under the hood.
https://orbit-ml.readthedocs.io/en/stable/
Other
1.86k stars 132 forks source link

[BUG] Predictions are outside of the prediction interval (the latter seems completely nonsense) #781

Closed Garrus990 closed 9 months ago

Garrus990 commented 2 years ago

Describe the bug When predicting certain time series, the prediction interval do not comprise the predictions.

To Reproduce

from io import StringIO

import pandas as pd
from orbit.models.ets import ETS as OrbitETS

ets_model = OrbitETS(seasonality=5, date_col="timestamp", response_col="ts", prediction_percentiles=[0.025, 0.975], verbose=False)

time_series = pd.read_csv(StringIO(
"""
timestamp,ts
2022-02-22,67.884
2022-02-23,70.376
2022-02-24,71.068
2022-02-25,64.137
2022-02-28,111.97265186124515
2022-03-01,51.199
2022-03-02,47.002
2022-03-03,36.58464786170519
2022-03-04,71.69753606323292
2022-03-07,86.16861299741936
2022-03-08,46.295
2022-03-09,53.366
2022-03-10,47.399
2022-03-11,48.081
2022-03-14,59.042
2022-03-15,44.809
2022-03-16,39.025
2022-03-17,39.6
2022-03-18,57.467
2022-03-21,41.418
2022-03-22,34.193
2022-03-23,32.653
2022-03-24,51.985
2022-03-25,36.748
2022-03-28,59.131
2022-03-29,54.224
2022-03-30,58.378
2022-03-31,84.93108940541931
2022-04-01,95.96319542678368
"""),
parse_dates=["timestamp"])

ets_model.fit(time_series.iloc[:-4])
ets_model.predict(time_series.iloc[-4:][["timestamp"]])

Expected behavior As expected, the last line returns a data frame with four columns, but the values in prediction column are outside of the prediction intervals (columns prediction_0.025 and prediction_0.975). The prediction intervals seem to be considerably off.

Screenshots image

Environment (please complete the following information):

Additional context The problem doesn't happen in every case. But I've encountered a number of cases when it occurs. More frequently than I'd like to believe that it is entirely on me :)

edwinnglabs commented 2 years ago

@Garrus990 In orbit, it is using percentile scale like the np.percentile. So in you case, you should specify prediction_percentiles = [2.5, 97.5] instead.