Closed mshepard closed 4 months ago
Instructing the Assistant to limit responses to ~200 tokens fixes the problem, but that’s a weak workaround.
Seems to be a timeout issue on the Run when executed through the production environment (Vercel). Still investigating.
I am also getting this when using Vercel. Also no issue when running locally. Very annoying..... Use case we are playing with won't work with limiting responses to 200 tokens.
Pro account timeout configurable upto 5 mins https://vercel.com/guides/what-can-i-do-about-vercel-serverless-functions-timing-out
Pro account timeout configurable upto 5 mins https://vercel.com/guides/what-can-i-do-about-vercel-serverless-functions-timing-out
This is it! I am currently only on Free plan while we play with ideas so I added:
export const maxDuration = 60; // 60 seconds maximum duration on free plan
to app/api/assistants/threads/[threadId]/messages/route.ts
Can't imagine a reponse needing more than 60 secs (esp. with the assistant using the 4o model!)
Thanks @ameer-clara for the direction. Appreciate it.
Ah ok. Thanks @ameer-clara ! Very much appreciated!
Thanks all for pitching in the advice!
Getting this error when assistant returns longer responses (~250 tokens): "Uncaught (in promise) Error: Final run has not been received"
The response is truncated at this point.
I deployed the app to Vercel. I do not encounter this when running the code in my local dev environment.
Thinking that a possible workaround would be to set max_tokens, but it seems that is not available (yet?) with the Assistants API?