vizhub-core / vzcode

Mob Programming Code Editor
MIT License
71 stars 14 forks source link

Ability to stop AI while it is generating #288

Open curran opened 1 year ago

curran commented 1 year ago

As a user leveraging the AI assist feature, when I can see that the AI is going in the wrong direction or has provided enough output, I want to be able to stop the generation, so that I don't need to wait for it to complete and can seamlessly continue working.

curran commented 1 year ago

Idea: hitting the Escape key during generation can send another request to the server to cancel the ongoing generation. We may need to generate an ID to specify which ongoing generation to terminate.

This may not seem like a big deal with GPT3.5 Turbo, as it's so insanely fast, but as soon as we switch to the slower (but better quality) GPT4, this could be a serious usability issue.

curran commented 1 year ago

From https://github.com/openai/openai-node#streaming-responses

If you need to cancel a stream, you can break from the loop or call stream.controller.abort().

curran commented 1 year ago

Some thoughts:

curran commented 1 year ago

Done in #316

curran commented 1 month ago

Reopening this as it's not currently functioning. I think we took it out at some point.