Question: Is there a ".forward()" like method in NeuralForecast?

[almostintuitive]

Hi! Thanks for the great library!:) We're trying to integrate it into fold as an available model, NeuralForecast was our first choice for time series DL models.

We fundamentally are trying to do time series cross validation, fitting a model on an expanding window basis, then predicting a single next timestamp, then continuing to train the model on that new data, then predicting again (only one step ahead). We figured that we can't use horizon 1 for some reason (it fails, at least the N-HiTS model) - we can deal with that!

But, more importantly we can't really see an easy way we could continue training a model on new data, similarly to Statsforecast's forward() method: https://github.com/Nixtla/statsforecast/blob/affc9894077edb1b12a7afc964d0df7527e1ddda/statsforecast/models.py#L908

Is there anything like this we could use? Thank you!

We're so looking forward benchmarking neuralforecast models!

[kdgutier]

Hey @almostintuitive,

Thanks for using NeuralForecast. We already have a main class with NeuralForecast.cross_validation method. Which seems to me is what you are looking for. Here is its code and an usage example:

• https://nixtla.github.io/neuralforecast/core.html#neuralforecast.cross_validation
• https://nixtla.github.io/neuralforecast/examples/longhorizon_with_nhits.html

[almostintuitive]

Yes, thank you! Unfortunately, in this case, for reproducability, etc., we need to do the cross validation ourselves. This way we can get many differnet TS libraries under the same umbrella API. Similarly to https://github.com/microprediction/timemachines/, but with support for mini-batch learning, as well as using a more pythonic API. We could easily wrap Nixtla's statsforecast models: https://github.com/dream-faster/fold-models/blob/main/src/fold_models/statsforecast.py

But we're struggling a bit with neuralforecast.

In Nerualforecast.cross_validation(), it looks like time series is split into n folds, then there's a model trained to predict for each fold, with a horizon of size of fold. Please correct me, if that's not the case!

Our usecase is single step-ahead forecasting, so we'd like to be able to re-use the model (update its weights) as we move forward time, don't train it from scratch for each fold (since we have a lot of folds), especially if its costly.

We limit the usecase to single step-ahead forecasting, as we'd like to benchmark model against each other when we have a chance to update with the latest incoming data.

[almostintuitive]

Any guidance around how we could do this would be much appreciated! We heard that there's a plan for an interface that unifies Nixtla models, I guess this is very much in line with what would be useful for us. But we're also happy to get involved and contribute, if this is a functionality that you think is possible with small modifications.

[cchallu]

Hi @almostintuitive. Indeed, our cross_validation does not retrain the model for every window. In our experience, this is the most common approach. Retraining for every new data point will be extremely expensive, and the difference between models will be very small. Retraining models is definitely a common practice, but have you considered doing it every longer intervals? For example if you have hourly data, retrain every day, week, or even month. Also, note that in the cross_validation method, the latest data is used as the input for making forecasts, so even without retraining, the latest data is being used.

If you need to retrain, I can suggest you a very simple loop. Here is the pseudocode:

Y_df = read_data() # Initial training data

model = NHITS(...)
nf = NeuralForecast(models,...)
nf.fit(Y_df) # Initial training

forecasts = []
for t=1 to T:
  forecast_t = nf.predict() # Predicts next h steps after training data
  Y_df = update_data(t)     # Update Y_df with new data point
  nf.fit(Y_df)              # You can reduce number of iterations overriding model.max_steps.
  forecasts.append(forecasts_t)

The NHITS model should work with a horizon of 1, I will check if there is a bug with that. Hope this helps!

[almostintuitive]

thank you very much for the prompt response, let me process this and get back as soon as possible!

[almostintuitive]

Thank you again, I think this is enough information for me to get this working, so I'll close this ticket!:)

[chaoticAttractoor]

Hey there,

I'm looking to fine tune an existing model. You mention in this code chunk you can over-ride max steps. How would I got about doing this?

`Y_df = read_data() # Initial training data

model = NHITS(...) nf = NeuralForecast(models,...) nf.fit(Y_df) # Initial training

forecasts = [] for t=1 to T: forecast_t = nf.predict() # Predicts next h steps after training data Y_df = update_data(t) # Update Y_df with new data point nf.fit(Y_df) # You can reduce number of iterations overriding model.max_steps. forecasts.append(forecasts_t)`

[jmoralez]

Hey @chaoticAttractoor, I think he meant something like nf.models[0].max_steps = some_value. If you have multiple models you have to override that attribute for all of them by iterating over the nf.models list.

[cchallu]

It is a little more complex than that because you have to modify the trainer kwargs. Here is the code we usually use for this:

def _set_trainer_kwargs(nf, max_steps, early_stop_patience_steps):
    # Max steps, validation steps and check_val_every_n_epoch
    trainer_kwargs = {**{"max_steps": max_steps}}
    if "max_epochs" in trainer_kwargs.keys():
        raise Exception("max_epochs is deprecated, use max_steps instead.")
    # Callbacks
    if trainer_kwargs.get("callbacks", None) is None:
        callbacks = [TQDMProgressBar()]
        # Early stopping
        if early_stop_patience_steps > 0:
            callbacks += [
                EarlyStopping(
                    monitor="ptl/val_loss", patience=early_stop_patience_steps
                )
            ]
        trainer_kwargs["callbacks"] = callbacks
    # Add GPU accelerator if available
    if trainer_kwargs.get("accelerator", None) is None:
        if torch.cuda.is_available():
            trainer_kwargs["accelerator"] = "gpu"
    if trainer_kwargs.get("devices", None) is None:
        if torch.cuda.is_available():
            trainer_kwargs["devices"] = -1
    # Avoid saturating local memory, disabled fit model checkpoints
    enable_check = trainer_kwargs.get("enable_checkpointing", None)
    if enable_check is None or enable_check is True:
        trainer_kwargs["enable_checkpointing"] = False
        trainer_kwargs["logger"] = False
    nf.models[0].trainer_kwargs = trainer_kwargs

[chaoticAttractoor]

Thanks guys!