search

facebookresearch / GENRE

Autoregressive Entity Retrieval
Other
763 stars 102 forks source link

is prefix_allowed_tokens_fn only working for seq2seq model.generate? #101

Closed wuzhiye7 closed 1 year ago

wuzhiye7 commented 1 year ago

i tried another encoder only model with prefix_allowed_tokens_fn . found that this function not work well. it always generate one token which is not in allowed tokens list . so,is prefix_allowed_tokens_fn just working for hf seq2seq model.generate?

wuzhiye7 commented 1 year ago

now i found this is a bug for transformers PrefixConstrainedLogitsProcessor ,haved fixed it

Chanwhistle commented 8 months ago

Hi, I want to use prefix_allowed_tokens_fn in decoder only model, but I can't find how to do it. Can you tell me how did you done prefix_allowed_tokens_fn to encoder only model? That could be a hint for me, too