Extremely slow when running without prompts under zero-shot setting

[pluiez]

I tested neurologic decoding with given prompts and without prompts on a V100 GPU device. The former took ~62 seconds to decode the top-10 examples from dev set, while the latter took 24 minutes. Thus on average one input example requires 2.4 minutes to generate a text. It this normal as expected?

[GloriaXimingLu]

Hi, thanks for the question. Yes it's as expected, since neurologic decoding without prompt generate sentences for all possible prompts in a prompt set and return the one with the highest likelihood.