artmoskvin / autocoder

Coding agent prototype
0 stars 0 forks source link

Enable response streaming #19

Open artmoskvin opened 4 months ago

artmoskvin commented 4 months ago

Currently, AI class handles only batch completions. To improve UX it should also support streaming.

devbuddytest[bot] commented 4 months ago

Here's the plan for implementing response streaming in the AI class:

  1. Refactor the AI class: Modify the AI class to include a method for streaming responses. This method should take the same parameters as the call method but return a generator instead of a string.

  2. Update the call method: The call method should be updated to use the new streaming method. This will ensure that all existing code that uses the call method will still work as expected.

  3. Update the BaseChatModel interface: The BaseChatModel interface should be updated to include a method for streaming responses. This method should take the same parameters as the existing method but return a generator instead of a string.

  4. Update all BaseChatModel implementations: All classes that implement the BaseChatModel interface should be updated to include the new streaming method. This will likely involve refactoring the existing method to use the new streaming method.

  5. Test the new implementation: Finally, the new streaming implementation should be thoroughly tested to ensure it works as expected. This should include both unit tests and integration tests.