Almost no convergence on Bitcoin Price Data

[Karlheinzniebuhr]

Two questions

• I tried training on BTC data but the model doesn't learn much (MSE=7233). Is there a fundamental mistake in my approach or is nhits not the right tool for this task? By comparison, LSTM achieved a MSE of 0.24655 on the same dataset.

• What does y_hat and y_true mean exactly and how are they calculated? What is their function since they always seem to have the same values?

Model is set to horizon=1 as we need to predict one t+1 interval, 5m interval in this case. I assume this is correct model = nf.auto.NHITS(horizon=1)

Here is the entire notebook with my code

[kdgutier]

Hi @Karlheinzniebuhr, responding the comments:

1. N-HiTS is specialized on long-horizon forecast, if you forecast one step ahead you are basically using N-BEATS.
2. 0.01229 seconds per iteration of hyperopt makes me think that the model is not training enough, I would increase the gradient steps: model.space['max_steps'] = hp.choice('max_steps', [1000])
3. Are you certain that the MSE you are using is not normalized. An MSE of 0.24 seems to me more of a percentage deviation than a quadratic deviation from the original signal.

Hope this helps

[Karlheinzniebuhr]

Indeed scaling solved it. The MSE is now even better than LSTM. How can I predict the next interval with the model? I want to do something like this (Arima example).

model.predict(start=btc_1h.index.min(),end=btc_1h.index.max())

or like this

model.forecast()

Also you recommended N-BEATS, I tried it with the same configuration but N-HITS is slightly better. Could you briefly explain the difference and when to use one over the other?

[kdgutier]

Change the horizon of the model, and parse it like you did before. model = nf.auto.NHITS(horizon=n_time_out)

For the moment we don't have the ability to filter the prediction with a min/max parameters. We are about to release a predict method that returns all the forecast for a given dataframe.

[cchallu]

Hi @Karlheinzniebuhr,

Both N-BEATS and N-HITS have a forecast method, it works as your ARIMA example.

[mergenthaler]

@Karlheinzniebuhr, have you tried this Auto Arima for benchmarking? Is very fast.

[Karlheinzniebuhr]

Ok forecast works fine, last question. What is the best-practice to forecast 1 step at each iteration? I tried appending the prediction f1 to the main dataframe and using f2 as input for the next prediction, this works but is there a better way? I'm aware I can change nf.auto.NHITS(horizon=n_time_out) to a higher number but that's not what we need. We always predict 1 value ahead only.

[Karlheinzniebuhr]

@Karlheinzniebuhr, have you tried this Auto Arima for benchmarking? Is very fast.

I was playing with statsforecast but haven't used it as it wasn't an improvement over ANNs for our case. Neuralforecast might just change that

[kdgutier]

Predicting and then using the prediction for another prediction is not ideal.

N-BEATS and a lot of modern neural forecast methods operate with direct multi-step strategy. If you know in advance the forecast horizon you create all the predictions for the horizon rather than creating the predictions recursively.

If you are interested in the topic see this paper A Bias and Variance Analysis for Multistep-Ahead Time Series Forecasting.