search

kr8s-org / kr8s

A batteries-included Python client library for Kubernetes that feels familiar for folks who already know how to use kubectl
https://kr8s.org
BSD 3-Clause "New" or "Revised" License
796 stars 41 forks source link

kr8s and gevent (wsgi) #481

Open teocns opened 2 weeks ago

teocns commented 2 weeks ago

Which project are you reporting a bug for?

kr8s

What happened?

When I was just about to celebrate testing a flask app in production.... kr8s took it away from me 🤣

So it turns out it doesn't play well with wsgi. From my analysis it's merely due to the async keyword-statements which gevent hallucinates on top of, as it runs its own internal monkey patching on python's threading system library

As a temporary fix, I'm now running a sync-native fork.

Looking for ideas and brainstorming onto ways we can merge this, if it's the case

Anything else?

https://github.com/fastapi/fastapi/discussions/6395

jacobtomlinson commented 2 weeks ago

I think I would agree with the comment by tiangolo https://github.com/fastapi/fastapi/discussions/6395#discussioncomment-5129869. If gevent is doing some monkey patching of Python internals then we can't guarantee that it will work. It sounds like nobody should be using gevent 😅.

That being said I'd be interested to understand more about what is going wrong. If you can share some small examples of how to reproduce the issues you are seeing I can dig in and see if there are any small things we can change to get things to play nicely.

Out of interest if creating a sync-native fork is your current workaround why not use a sync-native library like kubernetes or pykube-ng? I'm mainly asking because I'm excited that you seem to like kr8s enough to perservere here.

teocns commented 2 weeks ago

I'm mainly asking because I'm excited that you seem to like kr8s enough to perservere here.

That answers it partially 💯 on the other hand my backend's implementation is already founded around kr8s' APIObject, serialization and lookup features - would've taken me longer to adopt a different API than quickly refactoring async-to-sync.

If you can share some small examples of how to reproduce the issues you are seeing I can dig in and see if there are any small things we can change to get things to play nicely.

I think this is easy to reproduce: setup flask w/ gevent, and the issue happens exactly when invoking asgiref.async_to_sync, which flask[async] uses in current_app.ensure_sync. You will see a bunch of nasty runtime errors coming out of the threading module...

It sounds like nobody should be using gevent 😅.

I'm in grief

jacobtomlinson commented 2 weeks ago

Could you share a step by step that I can copy and paste, I'm not familiar with gevent and flask[async].

https://matthewrocklin.com/minimal-bug-reports

teocns commented 2 weeks ago

I'll be happy to, give me a few days and I'll setup a branch for us 👍