search

spyoungtech / grequests

Requests + Gevent = <3
https://pypi.python.org/pypi/grequests
BSD 2-Clause "Simplified" License
4.46k stars 331 forks source link

Docker crash #164

Open BIGdeadLock opened 1 year ago

BIGdeadLock commented 1 year ago

I get the following error when I run my grequest based code in a docker container:

image

spyoungtech commented 1 year ago

It looks like you're trying to use grequests in combination with some kind of webserver. Fundamentally, your problem is compatibility with gevent

grequests relies on gevent and uses monkey-patching to enable concurrency. This often causes conflicts with other packages that either (1) also try to monkey-patch the same modules, (2) use the patched modules in a manner that is incompatible with the monkey-patched version of the module or (3) just plain don't play nice with gevent. Because webservers fundamentally will use the same underlying modules (threading, sockets, ssl, and so on) it is not uncommon for such problems to occur.

Based on the error you have, it's most likely that you have code somewhere that is running unpatched modules blocking gevent from switching threads. This can happen if a module imports a module that needs to be patched before the patching actually occurs, for example, if you import another module before grequests.

You can also browse other gevent-related issues in this repo to find others with similar problems and workarounds.

Some common solutions that may fix your problem include:

  1. Ensuring that grequests is imported first before any other packages. This may require special consideration, depending how you are starting your server (this is the most likely the cause of your problem)
  2. If you are using a debugger, use a gevent-compatible debugger (or disable the debugger)
  3. Upgrading/downgrading the version of gevent used
  4. Configure the concurrency model of your WSGI server to be compatible with gevent (e.g., gevent vs eventlet backends)
  5. Use something else, like requests-threads or requests-futures instead, which don't rely on gevent/monkeypatching

Without more information about your code, dockerfile, etc., it's difficult to provide any further guidance.

BIGdeadLock commented 1 year ago

I tried looking in all other relevant issues and still did not find my answer. Generally, I import monkey from gevent and patch it. This is the first thing to be imported in my project. I later import grequest in another module (it's the first import in that module). Generally I use flask server. Can it cause problems with grequests? I see other people use both of them without problems. I also use Joblib and python's native threading lib for threading, can they cause problems with greenlet threads?