Hello, I encountered many issues while fine-tuning with my own data, possibly related to the configuration file of the dataset. Is there a description available for the dataset configuration file?

[potoyeee]

Especially regarding the validation set, removing it would allow training to commence. thank u

[liu-jc]

Hi @potoyeee , could you provide the error message here and the configuration file?

[potoyeee]

I follow the tutorial to create my own dataset.

hydra.errors.MissingConfigException: In 'default': Could not find 'data/price24'

Available options in 'data': etth1

[potoyeee]

This kind of error will occur when I wear the corresponding yaml.

Original Traceback (most recent call last): File "/home/ye/anaconda3/envs/darts_val/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop data = fetcher.fetch(index) File "/home/ye/anaconda3/envs/darts_val/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ye/anaconda3/envs/darts_val/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ye/anaconda3/envs/darts_val/lib/python3.10/site-packages/torch/utils/data/dataset.py", line 302, in getitem return self.datasets[dataset_idx][sample_idx] File "/media/ye/CA8A5611E3656A9F/uni2ts/src/uni2ts/data/dataset.py", line 73, in getitem return self.transform(self._flatten_data(self._get_data(idx))) File "/media/ye/CA8A5611E3656A9F/uni2ts/src/uni2ts/transform/_base.py", line 57, in call data_entry = t(data_entry) File "/media/ye/CA8A5611E3656A9F/uni2ts/src/uni2ts/transform/crop.py", line 117, in call a, b = self._get_boundaries(data_entry) File "/media/ye/CA8A5611E3656A9F/uni2ts/src/uni2ts/transform/crop.py", line 142, in _get_boundaries assert time >= b > a >= 0

[potoyeee]

How do I set the "finetune/val_data/xxx.yaml"

[gorold]

You need to create a YAML file similar to this, pointing to your own dataset. Depending on the generated file name, you should change the dataset field to dataset: price24_eval.

[potoyeee]

Thank you for your answer. It has a dimension error after setting up the validation set, which may be due to a problem with yaml file of the validation set. Ican finetune without validation set,loss decline but the performance of the test set is not as good as zero-shot, do you have any suggestions?

[gorold]

Hihi, it's overfitting. You could try training for fewer iterations, use early stopping, adding dropout, increasing weight decay, or collecting more data.

Regarding the validation set, I can't help you if you do not provide more details. If you follow the instructions carefully, you should be able to set up the validation set successfully. You could read the code implementation and the hydra docs for more information to debug.