I found that the input token logprob is supported by Offline Inference Pipeline, as mentioned in doc.
However, when I deployed a VLM server, I found that this feature was not supported. How to enable this on the server? Thanks.
Related resources
The code I deploy a server is lmdeploy serve api_server openbmb/MiniCPM-Llama3-V-2_5 --server-port 23333 --session-len 4096
cjfcsjt
commented
1 month ago
Motivation
I found that the input token logprob is supported by Offline Inference Pipeline, as mentioned in doc. However, when I deployed a VLM server, I found that this feature was not supported. How to enable this on the server? Thanks.
Related resources
The code I deploy a server is
lmdeploy serve api_server openbmb/MiniCPM-Llama3-V-2_5 --server-port 23333 --session-len 4096Additional context
No response