Closed yuhuishi-convect closed 8 months ago
Hey @yuhuishi-convect, thanks for the excellent report. The problem here is that some series are too short for the horizon. In this specific case the error is raised for the unique id 'Litchi(Indian)'
which has only 41 points, when we need at least 102 (2 train + 100 for validation).
We're going to add a more descriptive error message for this case but it'll still raise an error, the solution would be to either drop the series that are too short or set a shorter horizon. Please let us know if this helps.
Hey @chandrevdw31. Is it the same problem that I described (not enough datapoints for the cross validation settings)?
@jmoralez hi José, the last error above is unrelated to statsforecast at all, thanks for the quick answer though!
Thanks for the heads up @paxcema!
For reference, we recently added an earlier validation for the original issue here (in #610), which will be raised sooner and hopefully be clearer.
@yuhuishi-convect Please let us know if you need any more help.
@jmoralez I tried to reduce the periods from 100 to 20. Then I got this error:
Ah yes, I recently got that error too. It's coming from here https://github.com/Nixtla/statsforecast/blob/12e2dda56ace711383ccd66528bc1648c9fd8d3a/statsforecast/arima.py#L1816
because the series length is used to define the max terms for the order and np.sum returns an np.int, not a python int. I'll create a PR to fix this.
In the meantime, can you try wrapping that in an int
to see if it solves your problem? This should be the diff:
diff --git a/statsforecast/arima.py b/statsforecast/arima.py
index ccb67f0..06929dc 100644
--- a/statsforecast/arima.py
+++ b/statsforecast/arima.py
@@ -1813,7 +1813,7 @@ def auto_arima_f(
nonmissing_idxs = np.where(~missing)[0]
firstnonmiss = nonmissing_idxs.min()
lastnonmiss = nonmissing_idxs.max()
- series_len = np.sum(~missing[firstnonmiss:lastnonmiss])
+ series_len = int(np.sum(~missing[firstnonmiss:lastnonmiss]))
x = x[firstnonmiss:]
if xreg is not None:
if xreg.dtype not in (np.float32, np.float64):
This is with respect to the current main so the line numbers may be different, but it's at the very start of the auto_arima_f
function.
Sorry for the troubles.
@jmoralez should I just upgrade to a new version of statsforecast and try it against the dataset?
Since this is not released to pypi, so maybe I need to pip install
against the github repo directly to get the fixed version?
We'll make a release in the following days, but in the meantime installing from github should work. Please let us know if you get the chance to try the fix and run into any more issues.
I believe this has been fixed, feel free to reopen if you run into this issue.
What happened + What you expected to happen
Encounter wrong shape when running cross validation using AutoARIMA
Versions / Dependencies
Reproduction script
test data: nixtla_debug.csv
Issue Severity
High: It blocks me from completing my task.