LAION-AI / Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
https://open-assistant.io
Apache License 2.0
36.94k stars 3.22k forks source link

Cleanup `handle_worker()` in preparation for #2815 (Stop generation early) #3573

Closed 0xfacade closed 1 year ago

0xfacade commented 1 year ago

In this PR, I clean up the handle_worker() method a bit so that I can later extend it (in a future PR). There are no functional changes in this PR.

Changes:

The last change is the most important one for my future changes. In the main loop of handle_worker(), we were already waiting for two different types of futures: newly dequeued work requests from the Redis work queue, and responses from the worker received over the websocket. I'll need to add a third type of future next that allows us to listen to requests to stop generating text (#2815). The results of the different futures used to be differentiated based on their return type, which was very hard to read. I've created a decorator in FuturesManager that wraps the awaitable in another awaitable that returns a tuple, where the first entry is a FutureType enum value, and the second value is the result of awaiting the passed in awaitable. This makes it easy to distinguish what type of result was received.

I tested my changes by spinning up the inference server + worker with docker compose. Then I used the text-client to interface with the server.

Open issues:

github-actions[bot] commented 1 year ago

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

andreaskoepf commented 1 year ago

@0xfacade Do you still want to complete work on the todo points (unticked checkboxes) or can it be merged as it is?

0xfacade commented 1 year ago

@olliestanley @andreaskoepf This can be merged! Will work on the extension for the stop generating feature now