Open eeeeeunjung opened 7 months ago
Hi @eeeeeunjung, can you confirm this behavior also occurs in the 23.12 release as well?
CC; @tanmayv25 for vis
Hi @eeeeeunjung, can you confirm this behavior also occurs in the 23.12 release as well?
CC; @tanmayv25 for vis
oh sorry, i have no machine to test it now.
Is this bug resolved in the 23.12 release? thanks.
any updates?
Any update ? Not being able to rely on this is a big issue.
CC: @rmccorm4
Description i want to use the model's queue policy(max queue length and timeout),but i found triton does not handle requests in the accurate too,and i found this issue https://github.com/triton-inference-server/server/issues/5783, i guess this is the same question. When multiple requests are sent continuously at short intervals, the maximum queue length and queue waiting timeout sometimes do not take effect as expected
Triton Information triton 23.10 Ubuntu 22.04.3
To Reproduce easy way to reproduce:
i add sleep time in the testcase,and it failed.
Expected behavior add sleep time to test_max_queue_size and it success.