Clarification on the Evaluation example

[zqiao11]

Hi. I am a bit confused about the Evaluation example. Does that example use the pretrained Moirai from hugging face or the Moirai finetuned on etth1 from the previous Fine-tuning example for evaluation?

Based on the config file, I think it is the pretrained model rather than the finetuned one?

If I want to test the performance of the finetuned model, do I need to modify checkpoint_path to the local finetuned checkpoint's path?

Thanks in advance.

[gorold]

yup, you need to specify the checkpoint path to be your local finetuned model. the yaml file should look like this

_target_: uni2ts.model.moirai.MoiraiForecast.load_from_checkpoint
checkpoint_path: /my/model/path/here.ckpt
num_samples: 100
patch_size: ???
context_length: ???

[zqiao11]

Thank you for your answers. Just to clarify one more thing: In this case, we are loading a checkpoint of a MoiraiFintune object into a MoiraiForecast object. Could this lead to errors?

[gorold]

nope

[zqiao11]

Thanks, I doublechecked and now it is clear to me.

Just a kind suggestion: the context "Following up on the fine-tuning example, we can now perform evaluation on the test split by running the following script:" in the Evaluation example is a bit confusing. One may think it is using finetuned model for evaluation. You can consider revising it.