search

Lightning-Universe / lightning-transformers

Flexible components pairing 🤗 Transformers with :zap: Pytorch Lightning
https://lightning-transformers.readthedocs.io
Apache License 2.0
610 stars 77 forks source link

Couldn't run Tasks with Custom Data Files #124

Closed kaushikb11 closed 3 years ago

kaushikb11 commented 3 years ago

🐛 Bug

Reference docs.

Error trying to access dataset.train_file: node `dataset` is not a container and thus cannot contain `train_file`
    full_key: dataset.train_file
    object_type=dict

To Reproduce

python train.py +task=nlp/language_modeling dataset.train_file=train.csv dataset.validation_file=valid.csv

One more example:

sh-4.2$ python train.py +task=nlp/multiple_choice +dataset=nlp/multiple_choice/race dataset.train_file=train.json dataset.validation_file=valid.json
Could not override 'dataset.train_file'.
To append to your config use +dataset.train_file=train.json
Key 'train_file' is not in struct
    full_key: dataset.train_file
    object_type=dict