[Doc]: batch example file has outdated parameters

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

https://docs.vllm.ai

Apache License 2.0

30.09k stars 4.55k forks source link

[Doc]: batch example file has outdated parameters #9969

Open staeiou opened 1 week ago

staeiou commented 1 week ago

📚 The doc issue

in examples/offline_inference_openai.md, the linked examples/openai_example_batch.jsonl uses max_completion_tokens instead of max_tokens, causing an error when the example is run.

Suggest a potential alternative/fix

PR incoming

Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

DarkLight1337 commented 1 week ago

cc @gcalmettes since you made this change.

DarkLight1337 commented 1 week ago

@staeiou Please make sure you are using the same version of vLLM/docs. The latest code should support max_completion_tokens, but if you're installing vLLM via PyPI, you should refer to the stable version of the docs.