search

ThilinaRajapakse / pytorch-transformers-classification

Based on the Pytorch-Transformers library by HuggingFace. To be used as a starting point for employing Transformer models in text classification tasks. Contains code to easily train BERT, XLNet, RoBERTa, and XLM models for text classification.
Apache License 2.0
306 stars 97 forks source link

RuntimeError: Trying to create tensor with negative dimension -1: [-1, 768] #44

Closed iSteal closed 4 years ago

iSteal commented 4 years ago 
model = model_class.from_pretrained(args['model_name'])
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-11-2a8eccfdb8d2> in <module>
----> 1 model = model_class.from_pretrained(args['model_name'])

C:\Python37\Lib\site-packages\pytorch_transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    534 
    535         # Instantiate model.
--> 536         model = cls(config, *model_args, **model_kwargs)
    537 
    538         if state_dict is None and not from_tf:

C:\Python37\Lib\site-packages\pytorch_transformers\modeling_xlnet.py in __init__(self, config)
   1108         self.num_labels = config.num_labels
   1109 
-> 1110         self.transformer = XLNetModel(config)
   1111         self.sequence_summary = SequenceSummary(config)
   1112         self.logits_proj = nn.Linear(config.d_model, config.num_labels)

C:\Python37\Lib\site-packages\pytorch_transformers\modeling_xlnet.py in __init__(self, config)
    729         self.n_layer = config.n_layer
    730 
--> 731         self.word_embedding = nn.Embedding(config.n_token, config.d_model)
    732         self.mask_emb = nn.Parameter(torch.FloatTensor(1, 1, config.d_model))
    733         self.layer = nn.ModuleList([XLNetLayer(config) for _ in range(config.n_layer)])

C:\Python37\Lib\site-packages\torch\nn\modules\sparse.py in __init__(self, num_embeddings, embedding_dim, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse, _weight)
     95         self.scale_grad_by_freq = scale_grad_by_freq
     96         if _weight is None:
---> 97             self.weight = Parameter(torch.Tensor(num_embeddings, embedding_dim))
     98             self.reset_parameters()
     99         else:

RuntimeError: Trying to create tensor with negative dimension -1: [-1, 768]
iSteal commented 4 years ago

I think I need to run the colab_quickstart instead of the run_model.

ThilinaRajapakse commented 4 years ago

Please consider using Simple Transformers as this repo is not actively maintained.