search

Tencent / NeuralNLP-NeuralClassifier

An Open-source Neural Hierarchical Multi-label Text Classification Toolkit
Other
1.85k stars 406 forks source link

evalution threshold out of bound #66

Closed ghost closed 4 years ago

ghost commented 4 years ago

Please help me to debug this, i'm not clear why this is happening, Happy to share more detials if needed.

Use dataset to generate dict.
Size of doc_label dict is 97
Size of doc_token dict is 31732
Size of doc_char dict is 26
Size of doc_token_ngram dict is 0
Size of doc_keyword dict is 0
Size of doc_topic dict is 0
Shrink dict over.
Size of doc_label dict is 97
Size of doc_token dict is 30079
Size of doc_char dict is 26
Size of doc_token_ngram dict is 0
Size of doc_keyword dict is 0
Size of doc_topic dict is 0
Traceback (most recent call last):
  File "train.py", line 245, in <module>
    train(config)
  File "train.py", line 213, in train
    trainer.eval(train_data_loader, model, optimizer, "Train", epoch)
  File "train.py", line 105, in eval
    return self.run(data_loader, model, optimizer, stage, epoch)
  File "train.py", line 155, in run
    is_flat=self.conf.eval.is_flat, is_multi=is_multi)
  File "/content/NeuralNLP-NeuralClassifier/evaluate/classification_evaluate.py", line 267, in evaluate
    if prob_np[predict_label_idx[j]] > threshold:
IndexError: index 97 is out of bounds for axis 0 with size 97
coderbyr commented 4 years ago

Please help me to debug this, i'm not clear why this is happening, Happy to share more detials if needed.

Use dataset to generate dict.
Size of doc_label dict is 97
Size of doc_token dict is 31732
Size of doc_char dict is 26
Size of doc_token_ngram dict is 0
Size of doc_keyword dict is 0
Size of doc_topic dict is 0
Shrink dict over.
Size of doc_label dict is 97
Size of doc_token dict is 30079
Size of doc_char dict is 26
Size of doc_token_ngram dict is 0
Size of doc_keyword dict is 0
Size of doc_topic dict is 0
Traceback (most recent call last):
  File "train.py", line 245, in <module>
    train(config)
  File "train.py", line 213, in train
    trainer.eval(train_data_loader, model, optimizer, "Train", epoch)
  File "train.py", line 105, in eval
    return self.run(data_loader, model, optimizer, stage, epoch)
  File "train.py", line 155, in run
    is_flat=self.conf.eval.is_flat, is_multi=is_multi)
  File "/content/NeuralNLP-NeuralClassifier/evaluate/classification_evaluate.py", line 267, in evaluate
    if prob_np[predict_label_idx[j]] > threshold:
IndexError: index 97 is out of bounds for axis 0 with size 97

The label dict size is 97, which means label id ranges from 0 ~ 96, so 97 is out of bound, you can check whether the predict id is right or not.