search

lmg-anon / mikupad

LLM Frontend in a single html file
https://lmg-anon.github.io/mikupad/mikupad.html
Creative Commons Zero v1.0 Universal
233 stars 26 forks source link

[Request] Add token IDs to the object yielded by parseEventStream() #53

Closed neCo2 closed 5 months ago

neCo2 commented 5 months ago

Started working on phrase biasing, and being able to monitor the stream of token ids instead of the token strings would make life much easier for me. I'd do it myself, but I couldn't figure it out.

lmg-anon commented 5 months ago

parseEventStream's only job is to return the event stream as json, and it doesn't look like any backend sends back token ids in the completion stream. So I feel like the only choice would be to detokenize each token.

neCo2 commented 5 months ago

I see, that's a shame. Guess I'll have to figure something out for kobold then.