search

spyoungtech / grequests

Requests + Gevent = <3
https://pypi.python.org/pypi/grequests
BSD 2-Clause "Simplified" License
4.46k stars 331 forks source link

Collision with boto3 causes LoopExit exception #123

Closed orenherman closed 3 years ago

orenherman commented 6 years ago

Hi,

Encountered an error when using boto3 python client for S3. The new API of S3 supports the method upload_file/upload_fileobj which I guess internally uses gevent. Importing both grequests and boto3 causes uploads to fail sporadically with the following exception:

LoopExit: This operation would block forever

This happens also when there's no use of grequests functionality only importing the library (maybe monkey_patching collision)

Your help is appreciated.

nguyenbathanh commented 6 years ago

Got the same problem.

micossow commented 6 years ago

Same here, including grequests to my package has broken at least celery and pymongo (both use gevent). LoopExit error happening in random places.

ecamaj commented 6 years ago

any news?

spyoungtech commented 6 years ago

Similar issues have been reported in the past. For example, simply importing grequests can break functionality for the requests module, depending on the order they are imported. Changing the order of imports has been reported as a solution. See #103

Perhaps changing the order of your imports may help. Otherwise, it may be a more complicated issue to tackle. Note the update to the readme on recommendations for alternatives to grequests if all else fails.

FaridLU commented 4 years ago

I had faced the same issue. I couldn't find any solution with grequests. Then I followed this blog post: https://realpython.com/python-concurrency/ and solved the issue.

Then I made a package by following the approach of that blog, so that this helps to a lot of people, who are experiencing same issue. This package is lot more faster than grequests too.

Here is the package: fgrequests

I would love to hear feedback about this package.

Thank you.

spyoungtech commented 4 years ago

Makes sense to me. I think this also parallels efforts of similar projects like requests-threads and requests-futures.

Regarding fgrequests, the principle seems simple enough. I didn't get time to try it out yet, but from a casual glance at the source, it would be nice if it supported all the keyword arguments and functionality of requests so the API could be more substitutable. For example, user-provided sessions, cookies, stream, ssl verification, etc.

Hope that feedback is helpful @FaridLU

Cheers.

FaridLU commented 4 years ago

@spyoungtech thanks for your valuable feedback. Yes I will add all of these. This was an initial release with the necessary functionalities.