Closed KaraKaraWitch closed 4 months ago
cc @xwjiang2010
@KaraKaraWitch Can you try out #5888 and see if it fixes this issue?
Tested commit https://github.com/ywang96/vllm/commit/7097a204835360890606c362034e0cd847bd97ba and it appears to have resolved the issue.
Your current environment
🐛 Describe the bug
When running multiple requests to phi-3 vision, I sometimes get this error. Causing the app to refuse further requests.
Stacktrace on VLLM side:
The workaround I used was to send 1 request at a time with a multiprocessing lock from my client process.