Poor result on my dataset

[afshinebtia]

Thanks again for your great work! I applied your pre-trained model to my dataset, which has a frequency of 5 minutes (freq = 5T). Unfortunately, the prediction results are quite poor. Can you please confirm whether I need to fine-tune the model or if there might be issues with my implementation?

[ashok-arjun]

Hi! Thank you for the issue!

Did you use the Gluon dataset loader? In case you designed your own dataset loader, there might have been something to verify there. We'll be adding a tutorial soon with options to load datasets in several formats (such as CSV).

In case you're sure that your dataset loader works fine, it is possible that the model's zero-shot prediction is bad on this dataset. We're also preparing fine-tuning scripts and will update soon; I suggest you try this out with the fine-tuning script once released. I'll get back here once it is released, to update you.

Thanks!

[afshinebtia]

Thanks for your response.

Here is the way that I have prepared my dataset. Could you please confirm that the poor results is due to the issue of loading the dataset or the model needs to be fine-tuned?

# Parameters
context_length = 32
prediction_length = 24 
freq = "5T"  

df = pd.read_csv('.\Test.csv', parse_dates=['time'], index_col='time')

# Ensure the index is in the correct period type matching the frequency
df.index = df.index.to_period(freq)

# Convert DataFrame to time series
time_series = df['p'].values

# Assuming a single time series for simplicity; for multiple series, adjust accordingly
start = df.index[0]

train_ds = ListDataset(
    [{'target': time_series[:-prediction_length], 'start': start}],
    freq=freq
)

test_ds = ListDataset(
    [{'target': time_series, 'start': start}],
    freq=freq
)

[ashok-arjun]

The data loading looks OK to me.

Your data looks very similar to the Requests dataset (of a similar 1 minute frequency). We found that finetuning improves forecasts a lot there: see the paper (https://arxiv.org/pdf/2310.08278.pdf) Figure 8 in Page 19 - which is the zero-shot forecast, compared to Figure 11 in Page 20 (the forecast of the finetuned model).

I'm expecting your forecasts to improve a lot with finetuning.

Apologies; we're still cleaning, verifying and preparing the scripts for release. I'll tag you here once we release it. Thanks!

[afshinebtia]

Thanks for your response. That would be great to be able to fine-tune the model and forecast the data via the fine-tuned model.

[ashok-arjun]

Apologies for the delay with the finetuning release. I'm planning to release it within 2 weeks.

[afshinebtia]

Apologies for the delay with the finetuning release. I'm planning to release it within 2 weeks.

That would be great!

[ashok-arjun]

Hi @afshinebtia Can you please try finetuning now? We have added a list of instructions and best practices in the README that you could follow.

[ashok-arjun]

@afshinebtia Is this resolved?

[afshinebtia]

Hello @ashok-arjun.

I apologize for my delayed response. Yes, I truly appreciate your valuable work and your efforts to follow up on my issue.