how many candidates can test model(unifiedQA-uncased/best-model.pt) involve?

allenai / unifiedqa

UnifiedQA: Crossing Format Boundaries With a Single QA System

https://arxiv.org/abs/2005.00700

Apache License 2.0

426 stars 43 forks source link

how many candidates can test model(unifiedQA-uncased/best-model.pt) involve? #22

Closed wgkwgk closed 3 years ago

wgkwgk commented 3 years ago

x = model.generate_from_string("what do you take when do you go hunting? \n (A) gun (B) wow (C) beat (D) fuck (E) carry (F) cast (G) challenge (H) stop (I) drop (J) leave (K) turn (L) \ climb (M) no (N) enter (O) move ", tokenizer=tokenizer) -> ['gun']

but if there are so many candidates, it shows like that -> ['catalogue (O) working']

danyaljj commented 3 years ago

It's been trained on up to 5 candidates, but I think it generally works fine on more candidates (obviously the performance would deteriorate with more candidates).