Closed alaneckhardt closed 3 years ago
Sorry, my mistake. I passed source.spm
as vocabulary, it should have been opus.spm32k-spm32k.vocab.yml
. Also the train and dev texts should be encoded using preprocess.sh
. Closing and sorry for confusion.
Bug description
I want to fine-tune Marian models trained on Opus. Particularly, I'm working currently with https://huggingface.co/Helsinki-NLP/opus-mt-en-cs I have two text files with sources and targets and I want to fine tune the model on this data. When I run the training script below, I get following error:
There should be a 32k vocabulary, but it seems that the model has actually 58100. I tried also en-de model and it has embedding size 65000. I tried using
--dim-vocabs 58100 58100
but it didn't help, as the accompanied .spm models have really 32k vocab size.How to reproduce
Here is a minimal example. There has to be text files
sources.txt
andtargets.txt
.