Closed Stargate256 closed 4 months ago
If you are using the OpenAI api, this is already done, it transmits a [DONE]
token as seen here: https://github.com/LostRuins/koboldcpp/blob/concedo/koboldcpp.py#L695
For the KoboldAPI, it simply ends the stream when completed. You can modify the above line to add whatever you like.
Can I somehow have it transmit an end of stream character after the stream has ended?
SO basically I would like to have a special character sequence or just a single ASCII character that would be transmitted when token generation ends or it reaches a stop character.
If the feature doesn't exist could you please point me to the right direction of knowing which file must I modify and then recompile the project to get such a feature?