Open reneleonhardt opened 6 months ago
@carlrobertoh I converted conversations
on top of this branch, any chance that you can merge this first?
Another 29 files changed, still 120 to go 😅
I haven't had a chance to review this PR yet, but I'll try to do it sometime early next week. Also, I'll probably do the next release without these refactorings because our test coverage is bad, and I don't want to risk breaking anything.
Sounds good 👍
Speaking of which, please try Llama 3 locally first before releasing 😅
The response seems a bit off. Also, the success callback isn't being triggered once the server is up and running, so it looks like it's booting up forever.
Was able to fix the response by modifying the prompt and adding a stop token.
Thank you for testing and fixing! Stream was not the problem, begin_of_text
isn't needed, but assistant
, stopTokens and server hack (?).
Are other models running with msg
instead of message
, is the hack "official"?
Thank you for testing and fixing! Stream was not the problem,
begin_of_text
isn't needed, butassistant
,stopTokens
and server hack (?).
I followed the template format from their official guide. Even if it's not required, I'd still like to have it included unless it breaks something.
Are other models running with msg instead of message, is the hack "official"?
I called it a hack because the way we detect if the server is up and running is through the server logs. I assume they changed their logging structure (from message
to msg
), and that broke our trigger condition.
Hey, I'm putting this refactoring on pause for a bit until I have run a compatibility comparison between different IDE builds. Most likely, this won't cause any issues, but I just want to be sure, since I'm slowly considering supporting older versions again.
I suggest streamlining all
actionPerformed
:requireNonNull()
or handle null accordingly?