Does automodel_vllm need to produce logprobs

[arjunguha]

There is code in automodel_vllm to produce logprobs. Can we either:

• Put it behind a flag, or
• Remove it if it isn not used

@cassanof any opinions?

[cassanof]

I use it because if maximum likelihood with large samples significantly improves performance over greedy it likely means that the model is underfit.

Sure i can put it behind a flag