Can I use TransQuest to train a QE model with my own language pair and data?

[maohbao]

Can I use TransQuest to train a QE model with my own language pair and data, how? Is there any example to do this?

[TharinduDR]

Yes, you can. You can find an example here - https://github.com/TharinduDR/TransQuest/blob/master/examples/wmt_2020/ro_en/trans_quest.py

[maohbao]

Please help me understand this line of code:

model = QuestModel(MODEL_TYPE, MODEL_NAME, num_labels=1, use_cuda=torch.cuda.is_available(), args=transformer_config)

MODEL_TYPE, MODEL_NAME is defined in transformer_config.py as:

MODEL_TYPE = "xlmroberta" MODEL_NAME = "xlm-roberta-large"

xlmroberta or xlm-roberta-large means cross-language pre-trained model, is it? but where are the related model files stored, and how to train those pre-trained model?

Thank you very much!

H. Bao

[TharinduDR]

You are correct: xlm-roberta-large is a cross-lingual transformer model. You can read more about it in here - https://arxiv.org/pdf/1911.02116.pdf. However, we don't train the model, we just use the pre-trained model. You don't need to download it separately. The base library for this framework is huggingface transformers - https://github.com/huggingface/transformers. Based on that the xlm-roberta-large model will be downloaded automatically from huggingface model hub - https://huggingface.co/models

[maohbao]

But I can't find the code in this project that downloading xlm-roberta-large model file from huggingface......I am really not familiar with the structure of this project.

[TharinduDR]

It downloads the models in here. https://github.com/TharinduDR/TransQuest/blob/master/transquest/algo/transformers/run_model.py using transformers library. Check for lines like

self.model = model_class.from_pretrained(model_name, config=self.config, **kwargs) self.tokenizer = tokenizer_class.from_pretrained(model_name, do_lower_case=self.args.do_lower_case, **kwargs)

[maohbao]

Perhaps what I need is a full example of: 1) Train a QE model with a new language pair, the format of the train data that it needed; 2) Predict with the trained QE model; 3) Evaluate the predicted result.

@TharinduDR: Thank you very much for your detailed reply!

[TharinduDR]

You will find a full example here - https://github.com/TharinduDR/TransQuest/blob/master/examples/wmt_2020/ro_en/trans_quest.py

It reads the training data, do the predictions and perform the evaluation

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.