[Usage]: how to acquire logits in vllm

[zeyang12-jpg]

Your current environment

The output of `python collect_env.py`

How would you like to use vllm

I want to acquire logits when I run benchmark_throughput.py to do the softmax optimization, but the output in vllm doesn't have logits, how can I acquire it.

Before submitting a new issue...

• [X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[zeyang12-jpg]

my env is MI250 and ROCM 6.1

[hmellor]

Would a logits_processor work for your use case? https://docs.vllm.ai/en/latest/dev/sampling_params.html