search

prateekjoshi565 / Fine-Tuning-BERT

Apache License 2.0
139 stars 119 forks source link

Multi-label classification issue #2

Open rajae-Bens opened 3 years ago

rajae-Bens commented 3 years ago

Hi,

thank s for this great tutorial I want to apply this for a multi label text classification problem. My labels are of this format tensor([[0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], ..., [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0]]) I changed the softmax function in the bert model by the sigmoid function but when I tried to train the model I got this error multi-target not supported at /pytorch/aten/src/THCUNN/generic/ClassNLLCriterion.cu:18

Could u help plz thank u

prateekjoshi565 commented 3 years ago

Hi, you don't have to change the softmax function. Just replace the output units from 2 to the number of classes in your dataset in the layer below:

self.fc2 = nn.Linear(512,2)

rajae-Bens commented 3 years ago

First, thank u for responding. Well, I am confused now because in all tutorials Ive read about multi label classification, they all recommend using sigmoid for this kind of problems. Can u explain more plz Thank u