search

jonverrier / BraidEng

Engine code for an OpenAI assistant that likes to talk about AI, written using Typescript, Node.js, & the Azure stack. GPT-3 backed with a store of AI Canon documents.
GNU Affero General Public License v3.0
0 stars 1 forks source link

Force token estimation before messages are stored in shared structure #31

Closed jonverrier closed 1 month ago

jonverrier commented 1 month ago

Then it is calculated once and read many times. Example - force the function before stream out? Impacts performance with lots of messages, Can increase read-back limit then if we want.

jonverrier commented 1 month ago

Fixed in latest build - set on stream in/out