search

OctoberChang / X-Transformer

X-Transformer: Taming Pretrained Transformers for eXtreme Multi-label Text Classification
BSD 3-Clause "New" or "Revised" License
135 stars 28 forks source link

Assertion error in evaluation assert tY.shape == pY.shape fails #9

Closed vinaysetty closed 3 years ago

vinaysetty commented 3 years ago

Hi,

I have been trying to run the classifier on a couple of custom datasets I have with 300 and 700 classes. But I keep running into this assertion any idea what goes wrong?

08/03/2020 01:39:52 - INFO - transformers.modeling_utils - loading weights file save_models/dbpedia_summaries/pifa-tfidf-s0/matcher/roberta-large/pytorch_model.bin 08/03/2020 01:40:23 - INFO - main - Running evaluation 08/03/2020 01:40:23 - INFO - main - Num examples = 2379160 08/03/2020 01:40:23 - INFO - main - Batch size = 224 Traceback (most recent call last): File "xbert/transformer.py", line 678, in main() File "xbert/transformer.py", line 653, in main trn_loss, trn_metrics, C_trn_pred, trn_embeddings = matcher.predict(args, X_trn, C_trn, topk=args.only_topk, get_hidden=True) File "xbert/transformer.py", line 450, in predict eval_metrics = rf_linear.Metrics.generate(C_eval_true, C_eval_pred, topk=args.only_topk) File "pt1.2_xmlc_transformer/lib/python3.7/site-packages/xbert-0.1-py3.7-linux-x86_64.egg/xbert/rf_linear.py", line 205, in generate assert tY.shape == pY.shape, "tY.shape = {}, pY.shape = {}".format(tY.shape, pY.shape) AssertionError: tY.shape = (2379312, 8), pY.shape = (2379160, 8)

Thanks.

DarshanPatel11 commented 3 years ago

Facing the same issue. AssertionError: tY.shape = (165816, 128), pY.shape = (165463, 128) @vinaysetty Did you found any solution as you closed this issue?

follperson commented 3 years ago

Same issue here, essentially the same traceback. Did either of you figure it out? For reference this came in the evaluation phase after an apparently successful training session. 

11/30/2020 16:20:53 - INFO - transformers.modeling_utils -   loading weights file save_models/twit201612/pifa-tfidf-s0/matcher/bert-large-cased-whole-word-masking/pytorch_model.bin
11/30/2020 16:21:06 - INFO - __main__ -   ***** Running evaluation *****
11/30/2020 16:21:06 - INFO - __main__ -     Num examples = 2031503
11/30/2020 16:21:06 - INFO - __main__ -     Batch size = 32
Traceback (most recent call last):
  File "xbert/transformer.py", line 678, in <module>
    main()
  File "xbert/transformer.py", line 653, in main
    trn_loss, trn_metrics, C_trn_pred, trn_embeddings = matcher.predict(args, X_trn, C_trn, topk=args.only_topk, get_hidden=True)
  File "xbert/transformer.py", line 450, in predict
    eval_metrics = rf_linear.Metrics.generate(C_eval_true, C_eval_pred, topk=args.only_topk)
  File "/user/miniconda3/envs/pt1.2_xmlc_transformer/lib/python3.7/site-packages/xbert-0.1-py3.7-linux-x86_64.egg/xbert/rf_linear.py", line 205, in generate
    assert tY.shape == pY.shape, "tY.shape = {}, pY.shape = {}".format(tY.shape, pY.shape)
AssertionError: tY.shape = (2057841, 1024), pY.shape = (2031503, 1024)