Closed 0xfacade closed 1 year ago
:x: pre-commit failed.
Please run pre-commit run --all-files
locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md
@0xfacade Do you still want to complete work on the todo points (unticked checkboxes) or can it be merged as it is?
@olliestanley @andreaskoepf This can be merged! Will work on the extension for the stop generating feature now
In this PR, I clean up the
handle_worker()
method a bit so that I can later extend it (in a future PR). There are no functional changes in this PR.Changes:
HandleWorkerContext
that also features methods for initialization and destructionSessionManager
FuturesManager
The last change is the most important one for my future changes. In the main loop of
handle_worker()
, we were already waiting for two different types of futures: newly dequeued work requests from the Redis work queue, and responses from the worker received over the websocket. I'll need to add a third type of future next that allows us to listen to requests to stop generating text (#2815). The results of the different futures used to be differentiated based on their return type, which was very hard to read. I've created a decorator inFuturesManager
that wraps the awaitable in another awaitable that returns a tuple, where the first entry is aFutureType
enum value, and the second value is the result of awaiting the passed in awaitable. This makes it easy to distinguish what type of result was received.I tested my changes by spinning up the inference server + worker with
docker compose
. Then I used thetext-client
to interface with the server.Open issues:
pre-commit
hook - working on ittesting?There don't seem to be any tests