Closed magedhelmy1 closed 1 year ago
stream: false
response is returned as one big chunk when model stops generating
stream: true
response is returned in numerous smaller chunks as it is generated
like it is in GPT4-32k
That's just a model with larger context size. It has nothing to do with how data is returned to user.
Hi Mariodian,
having
stream: false,
gives the error
Cannot read properties of undefined (reading 'content')
Yeah, it makes sense. You'd have to heavily modify useSubmit.ts which currently only reads from stream.
Hi, I was wondering if there is somewhere in the code where can I tweak the typing speed or make it paste the output in chunks like it is in GPT4-32k. Thank you! :)