urchade / GLiNER

Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts) @ NAACL 2024
https://arxiv.org/abs/2311.08526
Apache License 2.0
1.47k stars 126 forks source link

Failed to test Gliner finetuned model #213

Open gptob opened 4 days ago

gptob commented 4 days ago

Hi, I have finetuned the gliner model, now I wished to test it on my testset to have a classification report, but I keep experiencing this problem on the predict method, even adding a function for calculating metrics to the trainer generates error. (The data is structured in the same way as the fine tuning was done)

config.json not found in /home/models/checkpoint-11500 Traceback (most recent call last): File "/home/Gliner_FineTuning_eval.py", line 76, in print(trainer.predict(test_data)) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lib/python3.12/site-packages/transformers/trainer.py", line 3946, in predict output = eval_loop( ^^^^^^^^^^ File "/home/lib/python3.12/site-packages/transformers/trainer.py", line 4051, in evaluation_loop for step, inputs in enumerate(dataloader): File "/home/lib/python3.12/site-packages/accelerate/data_loader.py", line 552, in iter current_batch = next(dataloader_iter) ^^^^^^^^^^^^^^^^^^^^^ File "/home/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 701, in next data = self._next_data() ^^^^^^^^^^^^^^^^^ File "/home/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 757, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch return self.collate_fn(data) ^^^^^^^^^^^^^^^^^^^^^ File "/home/lib/python3.12/site-packages/transformers/trainer_utils.py", line 843, in call return self.data_collator(features) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lib/python3.12/site-packages/gliner/data_processing/collator.py", line 29, in call raw_batch = self.data_processor.collate_raw_batch(input_x, entity_types = self.entity_types) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lib/python3.12/site-packages/gliner/data_processing/processor.py", line 191, in collate_raw_batch class_to_ids, id_to_classes = self.batch_generate_class_mappings(batch_list, negatives) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lib/python3.12/site-packages/gliner/data_processing/processor.py", line 162, in batch_generate_class_mappings negatives = self.get_negatives(batch_list, 100) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lib/python3.12/site-packages/gliner/data_processing/processor.py", line 77, in get_negatives types = set([el[-1] for el in b['ner']]) ~^^^^^^^ KeyError: 'ner'