search

jdb78 / pytorch-forecasting

Time series forecasting with PyTorch
https://pytorch-forecasting.readthedocs.io/
MIT License
3.87k stars 611 forks source link

# plot 10 examples error in Evaluate Results #481

Closed leechaelim closed 3 years ago

leechaelim commented 3 years ago

Hi, Thanks for this library!!! I want to predict the market price(Open price) of a stock. I'm trying to apply my data to the N-beats tutorial. but, I can't get 10 plots in the result evaluation step.

I executed code 

for idx in range(10):  # plot 10 examples
    best_model.plot_prediction(x, raw_predictions, idx=idx, add_loss_to_title=True);

Actual behavior

image

Why is there only one plot? And can you tell me why the smape error increases abnormally??

Compare below with baseline error: tensor(14734.1045) Baseline error was: tensor(0.2449)

here is my data and Colab notebook.

Nbeats_Open.zip

jdb78 commented 3 years ago

Do you only have one time series in x by any chance? Do you have an executable colab notebook link to let me look into the error. SMAPE cannot exceed 200 by definition.

leechaelim commented 3 years ago

yes..there is one time series in x

x

here is my colab notebook. https://colab.research.google.com/drive/1zpGWkSnq_LMuPbvAdbTfifMC90WGBcmt

Nbeats_Open.zip

캡처 In the tutorial, the error calculation was done like a picture. But it's different from the baseline_error calculation method.

  1. (actuals - predictions).abs().mean()

  2. SMAPE()(predictions, actuals)

1 and 2 have different results. What's right?

Thank you for your comment!

jdb78 commented 3 years ago

SMAPE is defined as ((y - y_hat).abs()/(y.abs() + y_hat.abs())).mean(), so it is not surprising that 1 (MAE) is not equal to 2 (SMAPE).

leechaelim commented 3 years ago

thank you!!!! I solved this issue Thanks for your comments :) 👍