Open Moustafa-Banbouk opened 1 year ago
For a sequence-to-sequence model like mT5, you also need to set evaluate_generated_text = True
in your model args. This could however slow down evaluation significantly.
Thanks Thilina for your fast response, I tried your suggestion; however, now I am getting a new error "TypeError: list indices must be integers or slices, not str" although that my training and evaluation datasets fields are all string.
The github issues log have a similar issue #1524 but still it is not answered and it refers to the following commit: https://github.com/ThilinaRajapakse/simpletransformers/commit/3f75da042a8ca1200126eb16bf8ca00d7c67854b
Appreciating your support,
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
[<ipython-input-96-e42a27b68e46>](https://localhost:8080/#) in <cell line: 105>()
--> 105 model.train_model(train_df, eval_data=eval_df, f1=f1_multiclass)
3 frames
[/usr/local/lib/python3.10/dist-packages/simpletransformers/t5/t5_model.py](https://localhost:8080/#) in train_model(self, train_data, output_dir, show_running_loss, args, eval_data, verbose, **kwargs)
227 os.makedirs(output_dir, exist_ok=True)
228
--> 229 global_step, training_details = self.train(
230 train_dataset,
231 output_dir,
[/usr/local/lib/python3.10/dist-packages/simpletransformers/t5/t5_model.py](https://localhost:8080/#) in train(self, train_dataset, output_dir, show_running_loss, eval_data, verbose, **kwargs)
603 ):
604 # Only evaluate when single GPU otherwise metrics may not average well
--> 605 results = self.eval_model(
606 eval_data,
607 verbose=verbose and args.evaluate_during_training_verbose,
[/usr/local/lib/python3.10/dist-packages/simpletransformers/t5/t5_model.py](https://localhost:8080/#) in eval_model(self, eval_data, output_dir, verbose, silent, **kwargs)
934 prefix + input_text
935 for prefix, input_text in zip(
--> 936 eval_dataset["prefix"], eval_dataset["input_text"]
937 )
938 ]
[/usr/local/lib/python3.10/dist-packages/simpletransformers/t5/t5_utils.py](https://localhost:8080/#) in __getitem__(self, index)
197
198 def __getitem__(self, index):
--> 199 return self.examples[index]
TypeError: list indices must be integers or slices, not str
Describe the bug A clear and concise description of what the bug is. Please specify the class causing the issue. I am trying to fine-tune an mT5 model while monitoring some custom evaluation metrics. When train_model starts, I am getting an error "All arrays must be of the same length". I tried to run train_model without the custom eval_metrics and training run successfully without the above error.
To Reproduce
Error stack received:
Expected behavior Fine tuning should run as expected without the error
Screenshots Datasets used during training (no null values inside dataset):
Desktop (please complete the following information): The code is running in a google Collab Pro+ environment
Additional context Not Applicable