Closed antonioglass closed 9 months ago
{ "input": { "prompt": "<s>[INST] Why is RunPod the best platform? [/INST]", "sampling_params": { "max_tokens": 100, "stop": [ "USER:", "User:" ] } } }
It worked with the previous version of a worker-vllm: https://github.com/runpod-workers/worker-vllm/tree/4f792062aaea02c526ee906979925b447811ef48
Fixed in latest version
It worked with the previous version of a worker-vllm: https://github.com/runpod-workers/worker-vllm/tree/4f792062aaea02c526ee906979925b447811ef48