search

yaroslavyaroslav / OpenAI-sublime-text

First class Sublime Text AI assistant with GPT-4o and llama.cpp support!
MIT License
130 stars 11 forks source link

Add streaming support #16

Closed yaroslavyaroslav closed 1 year ago

yaroslavyaroslav commented 1 year ago

Nowadays OpenAI chat API supports streaming through SSE. Thus it's improves usability's lot, since it's not leaving user strengt staring in the display for a tenth of seconds waiting for a model response. So it should be implemented in this plug-in.

yaroslavyaroslav commented 1 year ago

And it's actually already done in a rough way with the latest develop commit.

But it's not a production ready yet, since this did broke all other features of the plug-in (original competition, insert and so on).

Anyway planning to release this feature within this months as 2.1.0.

UPD: updated version to a correct one.

yaroslavyaroslav commented 1 year ago

Not enough time for that so far, and got some nasty bugs with new streaming implementation. So no new release in the nearest weeks, but still hoping to make it within July.

yaroslavyaroslav commented 1 year ago

Speaking about testing @rorymullen

yaroslavyaroslav commented 1 year ago

Leaving this for posterity. Yesterday, I performed a significant code refactoring for a plugin. As a result, the codebase has become much more maintainable. From the users' perspective, this means that the streaming 2.1.0 release is now much closer than before. It only requires the implementation of a single error handle to make it happen. Therefore, the new release date is expected to be around the middle of July.

UPD: Updated version to a correct one.

yaroslavyaroslav commented 1 year ago

So we're almost there. current develop 598c5d2ecab34bf313798a0c80fe9d370a78eadc handles deletion of two farthest conversation parts. Now user even has a choice to cancel that, like in case if they prefer to copy it somewhere beforehand.

This was the last feature to implement within 2.1.0 release. So now it's just the release notes the one that left to do to release this one.

UPD: Updated version to a correct one.

yaroslavyaroslav commented 1 year ago

Implemented in 2.1.0 release 7f6202ac53b94371e1aff9fd312cdbf84ec94679