add opt for passing abort signal to chatCompletion function

dexaai / dexter

LLM tools used in production at Dexa

MIT License

75 stars 5 forks source link

Closed cfortuner closed 5 months ago

cfortuner commented 5 months ago

Attempts to add the option to pass an abort signal through to the chatCompletion function.

(This isn't fully working yet.)

Notes:

Aborting the request seems to only work semi-consistently.
For Streaming, the work around is to just check if the signal is aborted when the chunk is received and handle it there.

TLDR still investigating how to abort properly

vercel[bot] commented 5 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
dexter	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 31, 2024 11:26pm