Is there any way to retrieve logprob in DeepSpeed-mii, just as we retrieve logprob by specifying options in vLLM?
My intention is to find out how the output from deepspeed-mii, vLLM and huggingface pipeline differs by retrieving the probability distribution.
Is there any way to retrieve logprob in DeepSpeed-mii, just as we retrieve logprob by specifying options in vLLM? My intention is to find out how the output from deepspeed-mii, vLLM and huggingface pipeline differs by retrieving the probability distribution.
Related Issues