Open feralvam opened 4 years ago
this bug usually caused by you use an old version preprocess.py to process the data then use the 0.10 version to train
This issue has been automatically marked as stale. If this issue is still affecting you, please leave any comment (for example, "bump"), and we'll keep it open. We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!
@NonvolatileMemory Do you have any suggestions? Other than re-processing the data using v0.10 preprocess.py.
What is your question?
Hello! Some months ago, I successfully used the previous version for multilingual translation with custom datasets. I recently noticed there's a new one so I wanted to test it using my same datasets. Unfortunately, I've come across an error that I hope you can help me with. I'm not sure if it's a bug or if I'm doing anything wrong.
Code
For my purposes, I'm trying to train a one-to-many model for language pairs: "orig-simp,orig-para,orig-split,orig-comp". These are not really "languages", but monolingual English data for different text-to-text generation tasks.
This is the preprocessing step:
This is the training step (exactly the same as in the example in the repo):
In case it's necessary, this is the content of
langs.txt
:When I ran the training command, I got the following error message before training began:
Any idea what could be the problem? Thanks!
What have you tried?
The error message is too general to find anything useful about it using google. I also tried to search the issues here but I was unsuccessful. So, I haven't been able to try anything in particular.
What's your environment?