search

Chainlit / literalai-python

https://docs.getliteral.ai/
Apache License 2.0
8 stars 0 forks source link

[ENG-1159] Handle SDK failures more gracefully #53

Closed desaxce closed 5 months ago

desaxce commented 5 months ago

In this pull request, we launch API calls in separate threads with try/except blocks which log the raised exception.

I tried a couple of use cases with the examples/Graceful SDK.ipynb notebook (Github wouldn't let me upload it on the PR directly), I had to commit it. To remove before merging

It's a RAG application which has attempts to perform with literal_client.step ... as step and a with literal_client.thread ... as thread. I added a time(1) in each so that I could stop the server in the middle of the clause. I timed the whole application so that I could ensure the application still runs in the same overall time when Literal cannot be reached.

The one thing we can notice in Threads is that the non-closed ones appear with their UUID:

image

And without the full sequence:

image

This change is Reviewable

linear[bot] commented 5 months ago
ENG-1159 Handle SDK failures more gracefully

[image.png](https://uploads.linear.app/6a41a374-76e1-40c2-a405-51050761590c/6ffa64f7-f38d-4403-a1bd-cdc30804194e/ed4a3c0a-3914-4a4c-8c41-1314dd7473fa) client.step()

desaxce commented 5 months ago

@willydouhard As much as I tried it with my sample RAG application, I'm not confident the "separate thread" bit doesn't impact some other part of the app that's unknown to me.