Displaying metrics from results on CoNLl

[Dimiftb]

Hi there,

So I believe I successfully managed to run your best model on CoNLl, however I was wondering how can I go about getting actual prediction values, e.g. Precision (Accuracy), F1 and Recall?

The current output that I have when running python train.py --config config/conll_03_english.yaml --test can be seen below:

Click to expand!

``` ModuleNotFoundError: No module named 'numpy.core._multiarray_umath' /content/ACE/flair/utils/params.py:104: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. dict_merge.dict_merge(params_dict, yaml.load(f)) 2021-07-07 10:38:05,848 Reading data from /root/.flair/datasets/conll_03 2021-07-07 10:38:05,848 Train: /root/.flair/datasets/conll_03/train.txt 2021-07-07 10:38:05,848 Dev: /root/.flair/datasets/conll_03/testa.txt 2021-07-07 10:38:05,848 Test: /root/.flair/datasets/conll_03/testb.txt 2021-07-07 10:38:13,533 {b'': 0, b'O': 1, b'S-ORG': 2, b'S-MISC': 3, b'B-PER': 4, b'E-PER': 5, b'S-LOC': 6, b'B-ORG': 7, b'E-ORG': 8, b'I-PER': 9, b'S-PER': 10, b'B-MISC': 11, b'I-MISC': 12, b'E-MISC': 13, b'I-ORG': 14, b'B-LOC': 15, b'E-LOC': 16, b'I-LOC': 17, b'': 18, b'': 19} 2021-07-07 10:38:13,533 Corpus: 14987 train + 3466 dev + 3684 test sentences [2021-07-07 10:38:13,922 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json from cache at /root/.cache/torch/transformers/b945b69218e98b3e2c95acf911789741307dec43c698d35fad11c1ae28bda352.9da767be51e1327499df13488672789394e2ca38b877837e52618a67d7002391 [2021-07-07 10:38:13,922 INFO] Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 0, "type_vocab_size": 2, "vocab_size": 28996 } [2021-07-07 10:38:14,289 INFO] loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-vocab.txt from cache at /root/.cache/torch/transformers/5e8a2b4893d13790ed4150ca1906be5f7a03d6c4ddf62296c383f6db42814db2.e13dbb970cb325137104fb2e5f36fe865f27746c6b526f6352861b1980eb80b1 [2021-07-07 10:38:14,680 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json from cache at /root/.cache/torch/transformers/b945b69218e98b3e2c95acf911789741307dec43c698d35fad11c1ae28bda352.9da767be51e1327499df13488672789394e2ca38b877837e52618a67d7002391 [2021-07-07 10:38:14,680 INFO] Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "output_hidden_states": true, "pad_token_id": 0, "type_vocab_size": 2, "vocab_size": 28996 } [2021-07-07 10:38:14,753 INFO] loading weights file https://cdn.huggingface.co/bert-base-cased-pytorch_model.bin from cache at /root/.cache/torch/transformers/d8f11f061e407be64c4d5d7867ee61d1465263e24085cfa26abf183fdc830569.3fadbea36527ae472139fe84cddaa65454d7429f12d543d80bfc3ad70de55ac2 [2021-07-07 10:38:28,899 INFO] All model checkpoint weights were used when initializing BertModel. [2021-07-07 10:38:28,899 INFO] All the weights of BertModel were initialized from the model checkpoint at bert-base-cased. If your task is similar to the task the model of the ckeckpoint was trained on, you can already use BertModel for predictions without further training. [2021-07-07 10:38:37,167 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased-config.json from cache at /root/.cache/torch/transformers/45629519f3117b89d89fd9c740073d8e4c1f0a70f9842476185100a8afe715d1.65df3cef028a0c91a7b059e4c404a975ebe6843c71267b67019c0e9cfa8a88f0 [2021-07-07 10:38:37,168 INFO] Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "directionality": "bidi", "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 0, "pooler_fc_size": 768, "pooler_num_attention_heads": 12, "pooler_num_fc_layers": 3, "pooler_size_per_head": 128, "pooler_type": "first_token_transform", "type_vocab_size": 2, "vocab_size": 119547 } [2021-07-07 10:38:37,520 INFO] loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased-vocab.txt from cache at /root/.cache/torch/transformers/96435fa287fbf7e469185f1062386e05a075cadbf6838b74da22bf64b080bc32.99bcd55fc66f4f3360bc49ba472b940b8dcf223ea6a345deb969d607ca900729 [2021-07-07 10:38:38,091 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased-config.json from cache at /root/.cache/torch/transformers/45629519f3117b89d89fd9c740073d8e4c1f0a70f9842476185100a8afe715d1.65df3cef028a0c91a7b059e4c404a975ebe6843c71267b67019c0e9cfa8a88f0 [2021-07-07 10:38:38,091 INFO] Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "directionality": "bidi", "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "output_hidden_states": true, "pad_token_id": 0, "pooler_fc_size": 768, "pooler_num_attention_heads": 12, "pooler_num_fc_layers": 3, "pooler_size_per_head": 128, "pooler_type": "first_token_transform", "type_vocab_size": 2, "vocab_size": 119547 } [2021-07-07 10:38:38,161 INFO] loading weights file https://cdn.huggingface.co/bert-base-multilingual-cased-pytorch_model.bin from cache at /root/.cache/torch/transformers/3d1d2b2daef1e2b3ddc2180ddaae8b7a37d5f279babce0068361f71cd548f615.7131dcb754361639a7d5526985f880879c9bfd144b65a0bf50590bddb7de9059 [2021-07-07 10:39:00,902 INFO] All model checkpoint weights were used when initializing BertModel. [2021-07-07 10:39:00,902 INFO] All the weights of BertModel were initialized from the model checkpoint at bert-base-multilingual-cased. If your task is similar to the task the model of the ckeckpoint was trained on, you can already use BertModel for predictions without further training. [2021-07-07 10:39:04,978 INFO] Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex . [2021-07-07 10:39:05,768 INFO] instantiating registered subclass relu of [2021-07-07 10:39:05,769 INFO] instantiating registered subclass relu of [2021-07-07 10:39:05,769 INFO] instantiating registered subclass relu of [2021-07-07 10:39:05,770 INFO] instantiating registered subclass relu of [2021-07-07 10:39:05,868 INFO] Initializing ELMo. [2021-07-07 10:39:33,761 INFO] loading Word2VecKeyedVectors object from /root/.flair/embeddings/glove.gensim [2021-07-07 10:39:34,377 INFO] loading vectors from /root/.flair/embeddings/glove.gensim.vectors.npy with mmap=None [2021-07-07 10:39:38,741 INFO] setting ignored attribute vectors_norm to None [2021-07-07 10:39:38,741 INFO] loaded /root/.flair/embeddings/glove.gensim [2021-07-07 10:39:39,293 INFO] loading Word2VecKeyedVectors object from /root/.flair/embeddings/en-fasttext-news-300d-1M [2021-07-07 10:39:43,391 INFO] loading vectors from /root/.flair/embeddings/en-fasttext-news-300d-1M.vectors.npy with mmap=None tcmalloc: large alloc 1200005120 bytes == 0x55bcac878000 @ 0x7f107e69f1e7 0x7f107bec7601 0x7f107bf302b0 0x7f107bf30e4e 0x7f107bfc635a 0x55bc2ab90d54 0x55bc2ab90a50 0x55bc2ac05105 0x55bc2abff4ae 0x55bc2ab923ea 0x55bc2ac0132a 0x55bc2abff4ae 0x55bc2ab923ea 0x55bc2ac0132a 0x55bc2ab9230a 0x55bc2ac0060e 0x55bc2abff4ae 0x55bc2ab92a81 0x55bc2ab92ea1 0x55bc2ac01bb5 0x55bc2abff7ad 0x55bc2ab92a81 0x55bc2ab92ea1 0x55bc2ac01bb5 0x55bc2abff7ad 0x55bc2ab923ea 0x55bc2ac047f0 0x55bc2abff7ad 0x55bc2ab92c9f 0x55bc2abd3d79 0x55bc2abd0cc4 [2021-07-07 10:40:16,461 INFO] setting ignored attribute vectors_norm to None [2021-07-07 10:40:16,461 INFO] loaded /root/.flair/embeddings/en-fasttext-news-300d-1M [2021-07-07 10:40:32,540 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-english-config.json from cache at /root/.cache/torch/transformers/4df1826a1128bbf8e81e2d920aace90d7e8a32ca214090f7210822aca0fd67d2.af9bc4ec719428ebc5f7bd9b67c97ee305cad5ba274c764cd193a31529ee3ba6 [2021-07-07 10:40:32,541 INFO] Model config XLMRobertaConfig { "_num_labels": 8, "architectures": [ "XLMRobertaForTokenClassification" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "eos_token_id": 2, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 1024, "id2label": { "0": "B-LOC", "1": "B-MISC", "2": "B-ORG", "3": "I-LOC", "4": "I-MISC", "5": "I-ORG", "6": "I-PER", "7": "O" }, "initializer_range": 0.02, "intermediate_size": 4096, "label2id": { "B-LOC": 0, "B-MISC": 1, "B-ORG": 2, "I-LOC": 3, "I-MISC": 4, "I-ORG": 5, "I-PER": 6, "O": 7 }, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "xlm-roberta", "num_attention_heads": 16, "num_hidden_layers": 24, "output_past": true, "pad_token_id": 1, "type_vocab_size": 1, "vocab_size": 250002 } [2021-07-07 10:40:32,901 INFO] loading file https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-english-sentencepiece.bpe.model from cache at /root/.cache/torch/transformers/431cf95b26928e8ff52fd32e349c1de81e77e39e0827a725feaa4357692901cf.309f0c29486cffc28e1e40a2ab0ac8f500c203fe080b95f820aa9cb58e5b84ed [2021-07-07 10:40:33,866 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-english-config.json from cache at /root/.cache/torch/transformers/4df1826a1128bbf8e81e2d920aace90d7e8a32ca214090f7210822aca0fd67d2.af9bc4ec719428ebc5f7bd9b67c97ee305cad5ba274c764cd193a31529ee3ba6 [2021-07-07 10:40:33,866 INFO] Model config XLMRobertaConfig { "_num_labels": 8, "architectures": [ "XLMRobertaForTokenClassification" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "eos_token_id": 2, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 1024, "id2label": { "0": "B-LOC", "1": "B-MISC", "2": "B-ORG", "3": "I-LOC", "4": "I-MISC", "5": "I-ORG", "6": "I-PER", "7": "O" }, "initializer_range": 0.02, "intermediate_size": 4096, "label2id": { "B-LOC": 0, "B-MISC": 1, "B-ORG": 2, "I-LOC": 3, "I-MISC": 4, "I-ORG": 5, "I-PER": 6, "O": 7 }, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "xlm-roberta", "num_attention_heads": 16, "num_hidden_layers": 24, "output_hidden_states": true, "output_past": true, "pad_token_id": 1, "type_vocab_size": 1, "vocab_size": 250002 } [2021-07-07 10:40:33,925 INFO] loading weights file https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll03-english-pytorch_model.bin from cache at /root/.cache/torch/transformers/3a603320849fd5410edf034706443763632c09305bb0fd1f3ba26dcac5ed84b3.437090cbc8148a158bd2b30767652c9e66e4b09430bc0fa2b717028fb6047724 [2021-07-07 10:41:53,332 INFO] All model checkpoint weights were used when initializing XLMRobertaModel. [2021-07-07 10:41:53,333 INFO] All the weights of XLMRobertaModel were initialized from the model checkpoint at xlm-roberta-large-finetuned-conll03-english. If your task is similar to the task the model of the ckeckpoint was trained on, you can already use XLMRobertaModel for predictions without further training. 2021-07-07 10:41:54,876 Model Size: 1106399156 Corpus: 14987 train + 3466 dev + 3684 test sentences 2021-07-07 10:41:54,910 ---------------------------------------------------------------------------------------------------- 2021-07-07 10:41:56,883 loading file resources/taggers/en-xlmr-tuned-first_elmo_bert-old-four_multi-bert-four_word-glove_word_origflair_mflair_char_30episode_150epoch_32batch_0.1lr_800hidden_en_monolingual_crf_fast_reinforce_freeze_norelearn_sentbatch_0.5discount_0.9momentum_5patience_nodev_newner5/best-model.pt ^C ```

Thank you.

[wangxinyu0922]

It seems that you canceled the prediction of the model. This command will evaluate the model on the test set.

[Dimiftb]

I had an issue with my runtime and it was cancelling my run. I think I'm on the right track to successfully replicate the results. I'm running 1 12 gb Nvidia GPU for 1.3 hours now, how long should it take approximately?

[wangxinyu0922]

The code get stuck in the pdb module (the debugger in python), you may check these lines in the code. It seems that training_state.pt is missing.

[Dimiftb]

Hi again @wangxinyu0922,

So I managed to find what the issue was. I had to add the folder en-xlmr-tuned-first_elmo_bert-old-four_multi-bert-four_word-glove_word_origflair_mflair_char_30episode_150epoch_32batch_0.1lr_800hidden_en_monolingual_crf_fast_reinforce_freeze_norelearn_sentbatch_0.5discount_0.9momentum_5patience_nodev_newner5 to folder resource/taggers and xlm-roberta-large-finetuned-conll03-english to folder resources

Now I have the same problem as before where the program keeps terminating after 4 mins and 53 seconds of execution and you can see the output below. ^C is the only thing listed as a reason for end of termination, however it is not me that's terminating the program. I'm using colab, so I can't even terminate the code like that. What could be the cause of the issue? Thanks.

Click to expand

``` ModuleNotFoundError: No module named 'numpy.core._multiarray_umath' /content/flair/utils/params.py:104: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. dict_merge.dict_merge(params_dict, yaml.load(f)) 2021-07-08 10:46:25,146 Reading data from /root/.flair/datasets/conll_03 2021-07-08 10:46:25,146 Train: /root/.flair/datasets/conll_03/train.txt 2021-07-08 10:46:25,146 Dev: /root/.flair/datasets/conll_03/testa.txt 2021-07-08 10:46:25,146 Test: /root/.flair/datasets/conll_03/testb.txt 2021-07-08 10:46:31,428 {b'': 0, b'O': 1, b'B-PER': 2, b'E-PER': 3, b'S-LOC': 4, b'B-MISC': 5, b'I-MISC': 6, b'E-MISC': 7, b'S-MISC': 8, b'S-PER': 9, b'B-ORG': 10, b'E-ORG': 11, b'S-ORG': 12, b'I-ORG': 13, b'B-LOC': 14, b'E-LOC': 15, b'I-PER': 16, b'I-LOC': 17, b'': 18, b'': 19} 2021-07-08 10:46:31,428 Corpus: 14987 train + 3466 dev + 3684 test sentences [2021-07-08 10:46:31,610 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json from cache at /root/.cache/torch/transformers/b945b69218e98b3e2c95acf911789741307dec43c698d35fad11c1ae28bda352.9da767be51e1327499df13488672789394e2ca38b877837e52618a67d7002391 [2021-07-08 10:46:31,610 INFO] Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 0, "type_vocab_size": 2, "vocab_size": 28996 } [2021-07-08 10:46:31,770 INFO] loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-vocab.txt from cache at /root/.cache/torch/transformers/5e8a2b4893d13790ed4150ca1906be5f7a03d6c4ddf62296c383f6db42814db2.e13dbb970cb325137104fb2e5f36fe865f27746c6b526f6352861b1980eb80b1 [2021-07-08 10:46:31,958 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json from cache at /root/.cache/torch/transformers/b945b69218e98b3e2c95acf911789741307dec43c698d35fad11c1ae28bda352.9da767be51e1327499df13488672789394e2ca38b877837e52618a67d7002391 [2021-07-08 10:46:31,959 INFO] Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "output_hidden_states": true, "pad_token_id": 0, "type_vocab_size": 2, "vocab_size": 28996 } [2021-07-08 10:46:32,161 INFO] loading weights file https://cdn.huggingface.co/bert-base-cased-pytorch_model.bin from cache at /root/.cache/torch/transformers/d8f11f061e407be64c4d5d7867ee61d1465263e24085cfa26abf183fdc830569.3fadbea36527ae472139fe84cddaa65454d7429f12d543d80bfc3ad70de55ac2 [2021-07-08 10:46:45,943 INFO] All model checkpoint weights were used when initializing BertModel. [2021-07-08 10:46:45,943 INFO] All the weights of BertModel were initialized from the model checkpoint at bert-base-cased. If your task is similar to the task the model of the ckeckpoint was trained on, you can already use BertModel for predictions without further training. [2021-07-08 10:46:48,258 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased-config.json from cache at /root/.cache/torch/transformers/45629519f3117b89d89fd9c740073d8e4c1f0a70f9842476185100a8afe715d1.65df3cef028a0c91a7b059e4c404a975ebe6843c71267b67019c0e9cfa8a88f0 [2021-07-08 10:46:48,258 INFO] Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "directionality": "bidi", "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 0, "pooler_fc_size": 768, "pooler_num_attention_heads": 12, "pooler_num_fc_layers": 3, "pooler_size_per_head": 128, "pooler_type": "first_token_transform", "type_vocab_size": 2, "vocab_size": 119547 } [2021-07-08 10:46:48,411 INFO] loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased-vocab.txt from cache at /root/.cache/torch/transformers/96435fa287fbf7e469185f1062386e05a075cadbf6838b74da22bf64b080bc32.99bcd55fc66f4f3360bc49ba472b940b8dcf223ea6a345deb969d607ca900729 [2021-07-08 10:46:48,780 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased-config.json from cache at /root/.cache/torch/transformers/45629519f3117b89d89fd9c740073d8e4c1f0a70f9842476185100a8afe715d1.65df3cef028a0c91a7b059e4c404a975ebe6843c71267b67019c0e9cfa8a88f0 [2021-07-08 10:46:48,780 INFO] Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "directionality": "bidi", "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "output_hidden_states": true, "pad_token_id": 0, "pooler_fc_size": 768, "pooler_num_attention_heads": 12, "pooler_num_fc_layers": 3, "pooler_size_per_head": 128, "pooler_type": "first_token_transform", "type_vocab_size": 2, "vocab_size": 119547 } [2021-07-08 10:46:48,969 INFO] loading weights file https://cdn.huggingface.co/bert-base-multilingual-cased-pytorch_model.bin from cache at /root/.cache/torch/transformers/3d1d2b2daef1e2b3ddc2180ddaae8b7a37d5f279babce0068361f71cd548f615.7131dcb754361639a7d5526985f880879c9bfd144b65a0bf50590bddb7de9059 [2021-07-08 10:47:11,264 INFO] All model checkpoint weights were used when initializing BertModel. [2021-07-08 10:47:11,264 INFO] All the weights of BertModel were initialized from the model checkpoint at bert-base-multilingual-cased. If your task is similar to the task the model of the ckeckpoint was trained on, you can already use BertModel for predictions without further training. [2021-07-08 10:47:13,717 INFO] Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex . [2021-07-08 10:47:14,264 INFO] instantiating registered subclass relu of [2021-07-08 10:47:14,264 INFO] instantiating registered subclass relu of [2021-07-08 10:47:14,265 INFO] instantiating registered subclass relu of [2021-07-08 10:47:14,265 INFO] instantiating registered subclass relu of [2021-07-08 10:47:14,323 INFO] Initializing ELMo. [2021-07-08 10:47:33,415 INFO] loading Word2VecKeyedVectors object from /root/.flair/embeddings/glove.gensim [2021-07-08 10:47:34,004 INFO] loading vectors from /root/.flair/embeddings/glove.gensim.vectors.npy with mmap=None [2021-07-08 10:47:38,431 INFO] setting ignored attribute vectors_norm to None [2021-07-08 10:47:38,431 INFO] loaded /root/.flair/embeddings/glove.gensim [2021-07-08 10:47:38,868 INFO] loading Word2VecKeyedVectors object from /root/.flair/embeddings/en-fasttext-news-300d-1M [2021-07-08 10:48:15,857 INFO] setting ignored attribute vectors_norm to None [2021-07-08 10:48:15,857 INFO] loaded /root/.flair/embeddings/en-fasttext-news-300d-1M [2021-07-08 10:48:31,294 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-english-config.json from cache at /root/.cache/torch/transformers/4df1826a1128bbf8e81e2d920aace90d7e8a32ca214090f7210822aca0fd67d2.af9bc4ec719428ebc5f7bd9b67c97ee305cad5ba274c764cd193a31529ee3ba6 [2021-07-08 10:48:31,295 INFO] Model config XLMRobertaConfig { "_num_labels": 8, "architectures": [ "XLMRobertaForTokenClassification" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "eos_token_id": 2, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 1024, "id2label": { "0": "B-LOC", "1": "B-MISC", "2": "B-ORG", "3": "I-LOC", "4": "I-MISC", "5": "I-ORG", "6": "I-PER", "7": "O" }, "initializer_range": 0.02, "intermediate_size": 4096, "label2id": { "B-LOC": 0, "B-MISC": 1, "B-ORG": 2, "I-LOC": 3, "I-MISC": 4, "I-ORG": 5, "I-PER": 6, "O": 7 }, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "xlm-roberta", "num_attention_heads": 16, "num_hidden_layers": 24, "output_past": true, "pad_token_id": 1, "type_vocab_size": 1, "vocab_size": 250002 } [2021-07-08 10:48:31,455 INFO] loading file https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-english-sentencepiece.bpe.model from cache at /root/.cache/torch/transformers/431cf95b26928e8ff52fd32e349c1de81e77e39e0827a725feaa4357692901cf.309f0c29486cffc28e1e40a2ab0ac8f500c203fe080b95f820aa9cb58e5b84ed [2021-07-08 10:48:32,185 INFO] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/xlm-roberta-large-finetuned-conll03-english-config.json from cache at /root/.cache/torch/transformers/4df1826a1128bbf8e81e2d920aace90d7e8a32ca214090f7210822aca0fd67d2.af9bc4ec719428ebc5f7bd9b67c97ee305cad5ba274c764cd193a31529ee3ba6 [2021-07-08 10:48:32,186 INFO] Model config XLMRobertaConfig { "_num_labels": 8, "architectures": [ "XLMRobertaForTokenClassification" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "eos_token_id": 2, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 1024, "id2label": { "0": "B-LOC", "1": "B-MISC", "2": "B-ORG", "3": "I-LOC", "4": "I-MISC", "5": "I-ORG", "6": "I-PER", "7": "O" }, "initializer_range": 0.02, "intermediate_size": 4096, "label2id": { "B-LOC": 0, "B-MISC": 1, "B-ORG": 2, "I-LOC": 3, "I-MISC": 4, "I-ORG": 5, "I-PER": 6, "O": 7 }, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "xlm-roberta", "num_attention_heads": 16, "num_hidden_layers": 24, "output_hidden_states": true, "output_past": true, "pad_token_id": 1, "type_vocab_size": 1, "vocab_size": 250002 } [2021-07-08 10:48:32,407 INFO] loading weights file https://cdn.huggingface.co/xlm-roberta-large-finetuned-conll03-english-pytorch_model.bin from cache at /root/.cache/torch/transformers/3a603320849fd5410edf034706443763632c09305bb0fd1f3ba26dcac5ed84b3.437090cbc8148a158bd2b30767652c9e66e4b09430bc0fa2b717028fb6047724 [2021-07-08 10:49:49,905 INFO] All model checkpoint weights were used when initializing XLMRobertaModel. [2021-07-08 10:49:49,905 INFO] All the weights of XLMRobertaModel were initialized from the model checkpoint at xlm-roberta-large-finetuned-conll03-english. If your task is similar to the task the model of the ckeckpoint was trained on, you can already use XLMRobertaModel for predictions without further training. 2021-07-08 10:49:51,469 Model Size: 1106399156 Corpus: 14987 train + 3466 dev + 3684 test sentences 2021-07-08 10:49:51,499 ---------------------------------------------------------------------------------------------------- 2021-07-08 10:49:53,711 loading file resources/taggers/en-xlmr-tuned-first_elmo_bert-old-four_multi-bert-four_word-glove_word_origflair_mflair_char_30episode_150epoch_32batch_0.1lr_800hidden_en_monolingual_crf_fast_reinforce_freeze_norelearn_sentbatch_0.5discount_0.9momentum_5patience_nodev_newner5/best-model.pt ^C ```

[wangxinyu0922]

It's strange. I have never used colab before so I do not know the reason. From your log, the program terminated when reading the model, so I suspect if there is a CPU memory limit for colab and it automatically kill the program when reading the model.

[Dimiftb]

Hi @wangxinyu0922

Thanks for your reply. This seems highly unlikely as colab wouldn't just terminate execution due to insufficient resources without an error message informing that are not enough resources to complete the process. Anyway, I will attempt to run the code on my machine to see if I will get the same results and if I do I'll write back for further assistance.

Thank you very much for helping me this far.

[nguyenbh]

@Dimiftb How is your progress on running locally?

[nguyenbh]

Close because of no response from the OP.