XLNet: ImportError: /opt/conda/lib/python3.6/site-packages/fused_layer_norm_cuda.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZNK2at11ATenOpTable11reportErrorEN3c1012TensorTypeIdE

[mfaisal]

I am trying to use xlnet base but getting an error:

learner = BertLearner.from_pretrained_model(databunch, args.model_name, metrics=metrics, device=device, logger=logger, output_dir=args.output_dir, finetuned_wgts_path=None, warmup_steps=10, multi_gpu=args.multi_gpu, is_fp16=True, multi_label=True, logging_steps=0)

10/18/2019 00:36:04 - INFO - transformers.file_utils - https://s3.amazonaws.com/models.huggingface.co/bert/xlnet-base-cased-config.json not found in cache or force_download set to True, downloading to /tmp/tmprecagwg5 100%|██████████| 641/641 [00:00<00:00, 177099.59B/s] 10/18/2019 00:36:04 - INFO - transformers.file_utils - copying /tmp/tmprecagwg5 to cache at /root/.cache/torch/transformers/c9cc6e53904f7f3679a31ec4af244f4419e25ebc8e71ebf8c558a31cbcf07fc8.ef1824921bc0786e97dc88d55eb17aabf18aac90f24bd34c0650529e7ba27d6f 10/18/2019 00:36:04 - INFO - transformers.file_utils - creating metadata file for /root/.cache/torch/transformers/c9cc6e53904f7f3679a31ec4af244f4419e25ebc8e71ebf8c558a31cbcf07fc8.ef1824921bc0786e97dc88d55eb17aabf18aac90f24bd34c0650529e7ba27d6f 10/18/2019 00:36:04 - INFO - transformers.file_utils - removing temp file /tmp/tmprecagwg5 10/18/2019 00:36:04 - INFO - transformers.configuration_utils - loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/xlnet-base-cased-config.json from cache at /root/.cache/torch/transformers/c9cc6e53904f7f3679a31ec4af244f4419e25ebc8e71ebf8c558a31cbcf07fc8.ef1824921bc0786e97dc88d55eb17aabf18aac90f24bd34c0650529e7ba27d6f 10/18/2019 00:36:04 - INFO - transformers.configuration_utils - Model config { "attn_type": "bi", "bi_data": false, "clamp_len": -1, "d_head": 64, "d_inner": 3072, "d_model": 768, "dropout": 0.1, "end_n_top": 5, "ff_activation": "gelu", "finetuning_task": null, "initializer_range": 0.02, "layer_norm_eps": 1e-12, "mem_len": null, "n_head": 12, "n_layer": 12, "n_token": 32000, "num_labels": 13, "output_attentions": false, "output_hidden_states": false, "pruned_heads": {}, "reuse_len": null, "same_length": false, "start_n_top": 5, "summary_activation": "tanh", "summary_last_dropout": 0.1, "summary_type": "last", "summary_use_proj": true, "torchscript": false, "untie_r": true, "use_bfloat16": false }

10/18/2019 00:36:04 - INFO - transformers.file_utils - https://s3.amazonaws.com/models.huggingface.co/bert/xlnet-base-cased-pytorch_model.bin not found in cache or force_download set to True, downloading to /tmp/tmpgq8bybxi 100%|██████████| 467042463/467042463 [00:35<00:00, 13098175.50B/s] 10/18/2019 00:36:40 - INFO - transformers.file_utils - copying /tmp/tmpgq8bybxi to cache at /root/.cache/torch/transformers/24197ba0ce5dbfe23924431610704c88e2c0371afa49149360e4c823219ab474.7eac4fe898a021204e63c88c00ea68c60443c57f94b4bc3c02adbde6465745ac 10/18/2019 00:36:42 - INFO - transformers.file_utils - creating metadata file for /root/.cache/torch/transformers/24197ba0ce5dbfe23924431610704c88e2c0371afa49149360e4c823219ab474.7eac4fe898a021204e63c88c00ea68c60443c57f94b4bc3c02adbde6465745ac 10/18/2019 00:36:42 - INFO - transformers.file_utils - removing temp file /tmp/tmpgq8bybxi 10/18/2019 00:36:42 - INFO - transformers.modeling_utils - loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/xlnet-base-cased-pytorch_model.bin from cache at /root/.cache/torch/transformers/24197ba0ce5dbfe23924431610704c88e2c0371afa49149360e4c823219ab474.7eac4fe898a021204e63c88c00ea68c60443c57f94b4bc3c02adbde6465745ac

ImportError Traceback (most recent call last)

in 3 finetuned_wgts_path=None, warmup_steps=10, 4 multi_gpu=args.multi_gpu, is_fp16=True, ----> 5 multi_label=True, logging_steps=0) /opt/conda/lib/python3.6/site-packages/fast_bert/learner_cls.py in from_pretrained_model(dataBunch, pretrained_path, output_dir, metrics, device, logger, finetuned_wgts_path, multi_gpu, is_fp16, loss_scale, warmup_steps, fp16_opt_level, grad_accumulation_steps, multi_label, max_grad_norm, adam_epsilon, logging_steps) 67 68 if multi_label == True: ---> 69 model = model_class[1].from_pretrained(pretrained_path, config=config, state_dict=model_state_dict) 70 else: 71 model = model_class[0].from_pretrained(pretrained_path, config=config, state_dict=model_state_dict) /opt/conda/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs) 340 341 # Instantiate model. --> 342 model = cls(config, *model_args, **model_kwargs) 343 344 if state_dict is None and not from_tf: /opt/conda/lib/python3.6/site-packages/transformers/modeling_xlnet.py in __init__(self, config) 979 self.num_labels = config.num_labels 980 --> 981 self.transformer = XLNetModel(config) 982 self.sequence_summary = SequenceSummary(config) 983 self.logits_proj = nn.Linear(config.d_model, config.num_labels) /opt/conda/lib/python3.6/site-packages/transformers/modeling_xlnet.py in __init__(self, config) 594 self.word_embedding = nn.Embedding(config.n_token, config.d_model) 595 self.mask_emb = nn.Parameter(torch.FloatTensor(1, 1, config.d_model)) --> 596 self.layer = nn.ModuleList([XLNetLayer(config) for _ in range(config.n_layer)]) 597 self.dropout = nn.Dropout(config.dropout) 598 /opt/conda/lib/python3.6/site-packages/transformers/modeling_xlnet.py in (.0) 594 self.word_embedding = nn.Embedding(config.n_token, config.d_model) 595 self.mask_emb = nn.Parameter(torch.FloatTensor(1, 1, config.d_model)) --> 596 self.layer = nn.ModuleList([XLNetLayer(config) for _ in range(config.n_layer)]) 597 self.dropout = nn.Dropout(config.dropout) 598 /opt/conda/lib/python3.6/site-packages/transformers/modeling_xlnet.py in __init__(self, config) 413 def __init__(self, config): 414 super(XLNetLayer, self).__init__() --> 415 self.rel_attn = XLNetRelativeAttention(config) 416 self.ff = XLNetFeedForward(config) 417 self.dropout = nn.Dropout(config.dropout) /opt/conda/lib/python3.6/site-packages/transformers/modeling_xlnet.py in __init__(self, config) 221 self.seg_embed = nn.Parameter(torch.FloatTensor(2, self.n_head, self.d_head)) 222 --> 223 self.layer_norm = XLNetLayerNorm(config.d_model, eps=config.layer_norm_eps) 224 self.dropout = nn.Dropout(config.dropout) 225 /opt/conda/lib/python3.6/site-packages/apex/normalization/fused_layer_norm.py in __init__(self, normalized_shape, eps, elementwise_affine) 131 132 global fused_layer_norm_cuda --> 133 fused_layer_norm_cuda = importlib.import_module("fused_layer_norm_cuda") 134 135 if isinstance(normalized_shape, numbers.Integral): /opt/conda/lib/python3.6/importlib/__init__.py in import_module(name, package) 124 break 125 level += 1 --> 126 return _bootstrap._gcd_import(name[level:], package, level) 127 128 /opt/conda/lib/python3.6/importlib/_bootstrap.py in _gcd_import(name, package, level) /opt/conda/lib/python3.6/importlib/_bootstrap.py in _find_and_load(name, import_) /opt/conda/lib/python3.6/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_) /opt/conda/lib/python3.6/importlib/_bootstrap.py in _load_unlocked(spec) /opt/conda/lib/python3.6/importlib/_bootstrap.py in module_from_spec(spec) /opt/conda/lib/python3.6/importlib/_bootstrap_external.py in create_module(self, spec) /opt/conda/lib/python3.6/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds) ImportError: /opt/conda/lib/python3.6/site-packages/fused_layer_norm_cuda.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZNK2at11ATenOpTable11reportErrorEN3c1012TensorTypeIdE Here his parameter settings: args = Box({ "train_size": -1, "val_size": -1, "log_path": LOG_PATH, "full_data_dir": DATA_PATH, "data_dir": DATA_PATH, "no_cuda": False, "bert_model": BERT_PRETRAINED_PATH, "output_dir": OUTPUT_PATH, "max_seq_length": 256, "do_train": True, "do_eval": False, "multi_gpu":True, "batch_size_per_gpu":4, "model_type": 'xlnet', "task_name": 'intent', "model_name": 'xlnet-base-cased', "num_train_epochs": 16, "learning_rate": 5e-5 }) Have anyone got similar error?

[DanyalAndriano]

ERROR - transformers.configuration_utils - Model name 'roberta-base-uncased' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc). We assumed 'roberta-base-uncased' was a path or url but couldn't find any file associated to this path or url.