Not Able to get Accuracy Score

ebanalyse / NERDA

Framework for fine-tuning pretrained transformers for Named-Entity Recognition (NER) tasks

MIT License

155 stars 35 forks source link

Not Able to get Accuracy Score #39

Open ParthP8399 opened 2 years ago

ParthP8399 commented 2 years ago

ValueError: Found input variables with inconsistent numbers of samples: [18734, 18733]

I am able to get F1 scores when i pass test_dict to evaluate function as model.evaluate_performance(test_dict) but if i pass "True"as a parameter to model.evaluate_performance(test_dict,True) i am getting a ValueError. I crossed checked the test_dict manually there are total 18734 samples in y_test so why is the function missing out one value in y_pred that is the predicted number of samples ?.

gklabs commented 1 year ago

It happened to me too, I think it is because NERDA model does not output predictions for sequences whose length is greater than max_seq_len