Closed genbei closed 2 years ago
As in README, the inputs should be tokenized and each line is a source language sentence and its target language translation, separated by (|||). You can see some examples in the examples
folder.
I haven't fine-tuned xlm-roberta-large before, but I think you can use the code in the xlmr
branch and tune some parameters (e.g. align_layer, learning_rate, max_steps) and see if you can get reasonable performance.
Because I have never trained the pre-training model, I have a small question about what the paralleldata input format looks like
TRAIN_FILE=/path/to/train/file
. Do you need a separator between src and tgt? What is the format? In addition, can you fine-tune the xlm-roberta-large model?And the xlm-roberta-base file contains these contents:
config.json 、gitattributes 、pytorch_model.bin 、sentencepiece.bpe.model 、tokenizer.json
This error occurred while running:
Now I really want to use bilingual data to continue training xlm-roberta-base model and ask for advice through TLM task.