search

huggingface / optimum-neuron

Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.
Apache License 2.0
177 stars 53 forks source link

Unable to use jsonl files for fine-tuning #380

Open vivekgangasani opened 6 months ago

vivekgangasani commented 6 months ago

I am trying to use a jsonl file for fine tuning with optimum neuron using a trainium instance. I get this error:

File "/usr/local/lib/python3.10/site-packages/optimum/neuron/hf_argparser.py", line 44, in parse_args_into_dataclasses

assert extension in ["csv", "json", "txt"], "train_file should be a csv, a json or a txt file." outputs = super().parse_args_into_dataclasses(*args, kwargs)AssertionError : File "/usr/local/lib/python3.10/site-packages/transformers/hf_argparser.py", line 338, in parse_args_into_dataclasses train_file should be a csv, a json or a txt file. assert extension in ["csv", "json", "txt"], "train_file should be a csv, a json or a txt file." AssertionError: train_file should be a csv, a json or a txt file. main() File "/opt/ml/code/run_clm.py", line 241, in main main() File "/opt/ml/code/run_clm.py", line 241, in main obj = dtype(inputs) File "", line 15, in init

HuggingFaceDocBuilderDev commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

HuggingFaceDocBuilderDev commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

HuggingFaceDocBuilderDev commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

HuggingFaceDocBuilderDev commented 3 weeks ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

HuggingFaceDocBuilderDev commented 2 days ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!