urchade / GLiNER

Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts) @ NAACL 2024
https://arxiv.org/abs/2311.08526
Apache License 2.0
1.4k stars 122 forks source link

Order of labels impacts detection confidence scores #192

Open saibhamidipati opened 1 month ago

saibhamidipati commented 1 month ago

I'm using the PII model using the following script:

model = GLiNER.from_pretrained("urchade/gliner_multi_pii-v1")
model.eval()
entities = model.predict_entities(text, labels)

for entity in entities:
    print(entity["text"], "=>", entity["label"], "=>", entity["score"] )

If my order of labels changes, I get a different score for the detected entities. Why is this happening?

urchade commented 1 month ago

Hi, it is because the model uses positional encoding Bi-encode version of gliner of @Ingvarstep should not have this effect

saibhamidipati commented 1 month ago

Thanks for your response, but I don't find bi-encoder version for gliner_multi_pii-v1. Is it available yet?