is_return_log_probs is required for decoupled model?

triton-inference-server / fastertransformer_backend

BSD 3-Clause "New" or "Revised" License

411 stars 134 forks source link

Open flexwang opened 1 year ago

flexwang commented 1 year ago

Looks like if is_return_log_probs is set to False, then the decoupled model does not return anything.