Chainlit / chainlit

Build Conversational AI in minutes ⚡️
https://docs.chainlit.io
Apache License 2.0
6.1k stars 777 forks source link

Chainlit deployment for multiple worker for production #719

Open suman12345678 opened 5 months ago

suman12345678 commented 5 months ago

Hi Team,

Thanks a lot for the wonderful work. I know more documentation will be available in future, but here is some questions I have if you can help me.

Are there any ways to use multiple worker while deploying in production? Are there any default setting of number of worker when we run 'chainlit run myapp.py' Where does the session stored? (In case I want to run with multiple workers, can I assume that for a particular session it can always find its session variable ? in cloud the assigned worker for a request can be changed for a single session unless session affinity is in place)

willydouhard commented 5 months ago

This is not possible atm, we would have to use gunicorn to spawn uvicorn workers I guess.

suman12345678 commented 5 months ago

Thanks Chainlit/chainlit

Could you please help me to understand how to run using gunicorn. Let's say I have chainlit program available as app.py. I can run that using 'chainlit run app.py' . Now with gunicorn how to run that? Are there any coding changes required?

On Sat, Feb 3, 2024, 3:51 AM Willy Douhard @.***> wrote:

This is not possible atm, we would have to use gunicorn to spawn uvicorn workers I guess.

— Reply to this email directly, view it on GitHub https://github.com/Chainlit/chainlit/issues/719#issuecomment-1925222588, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGVIXV7GUFF63M2GNACEIA3YRX3CHAVCNFSM6AAAAABCVQITX6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRVGIZDENJYHA . You are receiving this because you authored the thread.Message ID: @.***>

deepzliu commented 4 months ago

uvicorn.Config has 'workers' parameter, how about exposure in chainlit?

suman12345678 commented 4 months ago

Hi Team

We can expose in chainlit. But can we do something so that after certain time let's say 10 minutes session variables are automatically deleted? Can we do that please suggest.

On Tue, Feb 27, 2024, 10:36 PM Deepliu @.***> wrote:

uvicorn.Config has 'workers' parameter, how about exposure in chainlit?

— Reply to this email directly, view it on GitHub https://github.com/Chainlit/chainlit/issues/719#issuecomment-1968155062, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGVIXV3NJBBEJFDBALLV633YV2Q4BAVCNFSM6AAAAABCVQITX6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRYGE2TKMBWGI . You are receiving this because you authored the thread.Message ID: @.***>

nestorcolt-clv commented 3 months ago

Ya, I cannot deploy to multiple fargate tasks because gunicorn in the background has some issues. Related to:

https://github.com/benoitc/gunicorn/issues/1194

I'm looking forward to an answer to this to implement a workaround.