XLNet config num_labels and _num_labels

arkhan19 commented 2 years ago

Environment info

transformers version:
Platform:
Python version:
PyTorch version (GPU?):
Tensorflow version (GPU?):
Using GPU in script?:
Using distributed or parallel set-up in script?:

Who can help

Information

Model I am using (Bert, XLNet ...):

The problem arises when using:

[ ] the official example scripts: (give details below)
[ ] my own modified scripts: (give details below)

The tasks I am working on is:

[ ] an official GLUE/SQUaD task: (give the name)
[ ] my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

Create XLNetModel
Perform Sequence classification with more than 2 labels
config output before training, you can see "_num_labels":2.

Expected behavior

XLNet config is declaring "_num_labels": 2, when config.num_labels is being used in the model. I don't know why is that.

LysandreJik commented 2 years ago

Hello! Could you provide a reproducible code example? I don't manage to reproduce:

>>> from transformers import XLNetConfig
>>> config = XLNetConfig(num_labels=4)
>>> config
XLNetConfig {
  "attn_type": "bi",
  "bi_data": false,
  "bos_token_id": 1,
  "clamp_len": -1,
  "d_head": 64,
  "d_inner": 4096,
  "d_model": 1024,
  "dropout": 0.1,
  "end_n_top": 5,
  "eos_token_id": 2,
  "ff_activation": "gelu",
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1",
    "2": "LABEL_2",
    "3": "LABEL_3"
  },
  "initializer_range": 0.02,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1,
    "LABEL_2": 2,
    "LABEL_3": 3
  },
  "layer_norm_eps": 1e-12,
  "mem_len": 512,
  "model_type": "xlnet",
  "n_head": 16,
  "n_layer": 24,
  "pad_token_id": 5,
  "reuse_len": null,
  "same_length": false,
  "start_n_top": 5,
  "summary_activation": "tanh",
  "summary_last_dropout": 0.1,
  "summary_type": "last",
  "summary_use_proj": true,
  "transformers_version": "4.16.2",
  "untie_r": true,
  "use_mems_eval": true,
  "use_mems_train": false,
  "vocab_size": 32000
}

>>> config.num_labels
4
>>> config._num_labels
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/lysandre/Workspaces/Python/transformers/src/transformers/configuration_utils.py", line 250, in __getattribute__
    return super().__getattribute__(key)
AttributeError: 'XLNetConfig' object has no attribute '_num_labels'

arkhan19 commented 2 years ago

I am using different pertained XLNet Model, I guess, it's specific to that. Sorry for raising the issue.

huggingface / transformers