This PR addresses a potential issue in the api_generate_stream function where the semaphore might not be properly released in all scenarios. Specifically, if an exception occurs during the execution of worker.generate_stream(params), the create_background_tasks(request_id) function might not be called. This could lead to the semaphore not being properly released.
Related issue number (if applicable)
Closes #3389
Checks
[x] I've run format.sh to lint the changes in this PR.
[x] I've included any doc changes needed.
[x] I've made sure the relevant tests are passing (if applicable).
Why are these changes needed?
This PR addresses a potential issue in the
api_generate_stream
function where the semaphore might not be properly released in all scenarios. Specifically, if an exception occurs during the execution ofworker.generate_stream(params)
, thecreate_background_tasks(request_id)
function might not be called. This could lead to the semaphore not being properly released.Related issue number (if applicable)
Closes #3389
Checks
format.sh
to lint the changes in this PR.