Instructions For Runnig Model

[yusufani]

Hi, I couldn't find any file or instruction to use the model .I want to make text generation with the XLM model in English and Turkish . Here's what I've tried to run the model:

• Download pretrained model , bpe codes , vocabs under section "III. Applications: Supervised / Unsupervised MT" for English-French languanges . After that i tried to run translate.py with usage example in the file. To create sentences.bpe file , i used generate-embeddings.ipynb file . But I got the error:

FAISS library  was not found.
FAISS not available. Switching to standard nearest neighbors search implementation.
INFO - 10/01/19 12:48:51 - 0:00:01 - ============ Initialized logger ============
INFO - 10/01/19 12:48:51 - 0:00:01 - batch_size: 32
                                     command: python translate.py --exp_name translate --src_lang en --tgt_lang fr --model_path 'models/model-1-MT-ENG-FR/mlm_enfr_1024.pth' --output_path output --exp_id "vhx58jxgti"
                                     dump_path: ./dumped/translate/vhx58jxgti
                                     exp_id: vhx58jxgti
                                     exp_name: translate
                                     model_path: models/model-1-MT-ENG-FR/mlm_enfr_1024.pth
                                     output_path: output
                                     src_lang: en
                                     tgt_lang: fr
INFO - 10/01/19 12:48:51 - 0:00:01 - The experiment will be stored in ./dumped/translate/vhx58jxgti

INFO - 10/01/19 12:48:51 - 0:00:01 - Running command: python translate.py --exp_name translate --src_lang en --tgt_lang fr --model_path 'models/model-1-MT-ENG-FR/mlm_enfr_1024.pth' --output_path output
Model Parameters ->> {'same_enc_dec': True, 'emb_dim': 1024, 'n_layers': 6, 'n_heads': 8, 'dropout': 0.1, 'attention_dropout': 0.1, 'gelu_activation': True, 'share_inout_emb': True, 'sinusoidal_embeddings': False, 'asm': False, 'max_vocab': -1, 'min_count': 0, 'id2lang': {0: 'en', 1: 'fr'}, 'lang2id': {'en': 0, 'fr': 1}, 'n_langs': 2, 'n_words': 64139}
INFO - 10/01/19 12:49:00 - 0:00:10 - Supported languages: en, fr
Traceback (most recent call last):
  File "translate.py", line 141, in <module>
    main(params)
  File "translate.py", line 71, in main
    setattr(params, name, getattr(model_params, name))
AttributeError: 'AttrDict' object has no attribute 'bos_index'

• Also i tried to preprocess.py file for preprocessing but it's give me xxxx.pth file and i don't know what i have to do for use this file.

Can you share step-by-step instructions on how to run the model?

[aykutakdenizz]

That is exactly what I need. I search in many websites and tried everything. But I can not find anything.I can’t handle any successful method. I am very thankful if somebody solves this problem.

[ahmetbagci8]

I am working on similar situation. If anyone solves this problem please help me

[glample]

Hi,

Can you try to remove 'bos_index' from this line: https://github.com/facebookresearch/XLM/blob/master/translate.py#L70 and see what happens?

[glample]

The box_index is not used anymore, as we only add eos_index (end of sentence index) both before and after sentences.

[yusufani]

Actually I tried to remove 'bos_index' from line 70 but it didn't work . I got this Error from same line:

Traceback (most recent call last):
  File "translate.py", line 142, in <module>
    main(params)
  File "translate.py", line 72, in main
    setattr(params, name, getattr(model_params, name))
AttributeError: 'AttrDict' object has no attribute 'eos_index'

As you can see from below , model have only these parameters : {'same_enc_dec': True, 'emb_dim': 1024, 'n_layers': 6, 'n_heads': 8, 'dropout': 0.1, 'attention_dropout': 0.1, 'gelu_activation': True, 'share_inout_emb': True, 'sinusoidal_embeddings': False, 'asm': False, 'max_vocab': -1, 'min_count': 0, 'id2lang': {0: 'en', 1: 'fr'}, 'lang2id': {'en': 0, 'fr': 1}, 'n_langs': 2, 'n_words': 64139}

So , I must have made a mistake somewhere, but I can't find it, even though I've tried it over and over again. I'm very eager to use XLM. Could you tell me what to do step by step?

[glample]

Really sorry about this. There is something we probably did not do correctly and I'm not sure why you have this issue. I'll have a look at it next week because unfortunately I don't have much time these days..

However, it looks like it should be easy to fix. Can you try something like in this notebook? https://github.com/facebookresearch/XLM/blob/master/generate-embeddings.ipynb Typically this cell:

dico = Dictionary(reloaded['dico_id2word'], reloaded['dico_word2id'], reloaded['dico_counts'])
params.n_words = len(dico)
params.bos_index = dico.index(BOS_WORD)
params.eos_index = dico.index(EOS_WORD)
params.pad_index = dico.index(PAD_WORD)
params.unk_index = dico.index(UNK_WORD)
params.mask_index = dico.index(MASK_WORD)

If you have the dictionary stored in the reloaded model (which should really be the case) then it should be called dico_word2id: https://github.com/facebookresearch/XLM/blob/master/src/trainer.py#L519-L521

Actually, this is what the next line in translate.py is loading: https://github.com/facebookresearch/XLM/blob/master/translate.py#L74

[yusufani]

I really appreciate your help. But as I mentioned in my first message, I've already tried the code snippet you specified. In addition, I'm adding the results here, indicating that it was successfully completed.

Then we can say that now we are sure that there is no problem with the model we added. Somehow I have trouble calling in the translate.py file. Unfortunately, I cannot find and solve the problem because I do not know the internal structure of the model or code. I'll be waiting for your help when you're available.

Is it also possible to publish a notebook, kernel or code using the XLM model for young people like me , who want to learn and apply NLP? In this way, we can achieve much faster and trouble-free results.

[mohammedayub44]

I tried to troubleshoot this further. I set the params like below as mentioned in above comment: model_params.bos_index = dico.index(BOS_WORD) ... and added import statement from src.data.dictionary import Dictionary, BOS_WORD, EOS_WORD, PAD_WORD, UNK_WORD, MASK_WORD

to get rid of the above error. However, i get an error saying the reloaded model dict does not have 'encoder' or 'decoder'.

I check the reloaded model dict , it does not have 'encoder' and 'decoder' keys -