SalesforceAIResearch / uni2ts

Unified Training of Universal Time Series Forecasting Transformers
Apache License 2.0
868 stars 94 forks source link

Cannot Concatenate Two Datasets for Finetuning Moirai using uni2ts.data.builder.ConcatDatasetBuilder #114

Closed Darshansingh11 closed 2 months ago

Darshansingh11 commented 2 months ago

Describe the bug I am currently finetuning the Moirai model, and it works as expected when using a single dataset. However, when I attempt to finetune using two datasets, I encounter an error. I have created a YAML file located at finetune/data/combined.yaml with the following contents:

_target_: uni2ts.data.builder.ConcatDatasetBuilder
_args_:
 -_target_: uni2ts.data.builder.simple.SimpleDatasetBuilder
  dataset: dataset1
 -_target_: uni2ts.data.builder.simple.SimpleDatasetBuilder
  dataset: dataset2

Individual finetuning with dataset1 and dataset2 works without issues.

To Reproduce Take any two CSV files and execute:

python -m uni2ts.data.builder.simple dataset1 dataset1.csv --dataset_type wide
python -m uni2ts.data.builder.simple dataset2 dataset2.csv --dataset_type wide

Next use the above combined.yaml file to provide datapath during finetuning as follows: python -m cli.train -cp conf/finetune run_name=test model="moirai_1.0_R_large" data=combined.yaml

Expected behavior To finetune without giving errors

Error message or code output

Error in call to target 'uni2ts.data.builder._base.ConcatDatasetBuilder': 
AssertionError ('All builders must be instances of DatasetBuilder') 
full key: data

Environment

Could you please help me with this @gorold @liu-jc @chenghaoliu89 ? Thanks

Darshansingh11 commented 2 months ago

Removing the @dataclass decorator from line 181 of uni2ts/data/builder/simple.py (and adding a simple __init__()) fixes this issue.