search

uditarora / ood-text-emnlp

Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them"
MIT License
25 stars 6 forks source link

how to handle -1 label class for `mnli` in distribution and `snli` out of distribution dataset. #1

Closed islamazhar closed 2 years ago

islamazhar commented 2 years ago

Hi,

I was trying to run the model for mnli in distribution and snli out of distribution dataset using the following command (The mnli model is already pretained)

id_data="mnli"
ood_data="snli"
python3 msp_eval.py --model_path roberta_ckpts/roberta-$id_data --dataset_name $ood_data --fname ${id_data}_$ood_data --batch_size 8

However as snli has -1 class label showing the following error in criterion function which is nn.CrossEntropyLoss()

https://github.com/uditarora/ood-text-emnlp/blob/c0e1dacbd3c89ebc30bd19e34d6f6db4ae7541eb/roberta_fine_tune.py#L140

The error message is:

cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [2,0,0] Assertion t >= 0 && t < n_classes failed.

How to solve the problem? Any suggestions? Thanks in advance.

uditarora commented 2 years ago

Hi! You can filter out these examples during pre-preprocessing. This is what we did for our work separately. Let me know if you need help with it.