search

JWCook / requests-ratelimiter

Easy rate-limiting for python requests
MIT License
84 stars 5 forks source link

Update for compatibility with pyrate-limiter v3 #78

Open JWCook opened 1 year ago

jamesmhbarry commented 1 year ago

Has there been any progress on this issue? I am getting this error message:


    from .requests_ratelimiter import *
  File "~/anaconda3/lib/python3.11/site-packages/requests_ratelimiter/requests_ratelimiter.py", line 8, in <module>
    from pyrate_limiter import Duration, Limiter, RequestRate
ImportError: cannot import name 'RequestRate' from 'pyrate_limiter' (~/anaconda3/lib/python3.11/site-packages/pyrate_limiter/__init__.py)```
JWCook commented 1 year ago

It's supposed to be pinned to ^2.8, but it looks like I neglected to do that for the conda-forge recipe. I'll get that updated.

As for moving to pyrate-limiter v3, it seems like there are a few features missing (which the maintainer is still working on), so I'm holding off on this issue until that's sorted out.

JWCook commented 1 year ago

This has been fixed on conda-forge in v0.4.2.

jamesmhbarry commented 1 year ago

This has been fixed on conda-forge in v0.4.2.

Great, thanks!

desaijimmy commented 4 months ago

It's supposed to be pinned to ^2.8, but it looks like I neglected to do that for the conda-forge recipe. I'll get that updated.

As for moving to pyrate-limiter v3, it seems like there are a few features missing (which the maintainer is still working on), so I'm holding off on this issue until that's sorted out.

Hi JWCook - Thanks for maintaining this lib. Just curious if this can be upgraded to work with pyrate-limiter v3 now? :)

Thanks!

JWCook commented 4 months ago

Probably! I will take another look soon to see if pyrate-limiter v3 has re-implemented all the features I need. If not, I'll submit some issues and/or PRs for whatever is missing.

desaijimmy commented 4 months ago

Probably! I will take another look soon to see if pyrate-limiter v3 has re-implemented all the features I need. If not, I'll submit some issues and/or PRs for whatever is missing.

Look forward to it. Thanks very much!

JWCook commented 3 months ago

The main thing still missing from v3 is the multiprocess-compatible SQLite backend I originally submitted here: https://github.com/vutran1710/PyrateLimiter/pull/68. That's a pretty important feature that I and others are using via requests-ratelimiter. I made an issue for that upstream.

It's also missing the contextmanager features I added here: https://github.com/vutran1710/PyrateLimiter/pull/25, although that's probably something I could work around.

As far as I can tell, pyrate-limiter v3 isn't finished: https://github.com/vutran1710/PyrateLimiter/issues/118. I'm willing to help get it there, but IMO the current releases on PyPI should have been published with a pre-release tag like -alpha. Is there a specific feature it provides that you'd like to use? Just wondering if there's a short-term workaround I could provide.

desaijimmy commented 3 months ago

Thanks for following Jordan - much appreciated!

I've managed to get a workaround for now, so it's not a problem. I just run in to issues when upgrading packages via pip as it results in some conflicts with other packages that I use. However that's a minor issue for now as I can manually sort it out by deleting the latest pyrate-limiter package and just reinstalling requests-ratelimiter, which then installs the appropriate version of pyrate-limiter.

Hopefully the upstream issues get resolved soon!

Thanks again for following up on this!

tgrandje commented 4 weeks ago

Is there a specific feature it provides that you'd like to use? Just wondering if there's a short-term workaround I could provide.

@JWCook to add to this discussion, I'm personnally interested by the new PostgresBucket backend.

My use case is a package running API calls but used in different pods/containers: I need to track this whatever the actual number of pods currently running. I was thinking of either trying to create a new custom bucket on a S3 storage system or using the PostgresBucket (which is not supporting async, and I'm not sure about the implications yet)...

Note : if not that, the package I'm speaking of has it's own way of tracking calls which is stored in a parquet file, which could easily be stored on a S3... But I'm way more in favor of using requests-ratelimiter, though I'm not sure that's (yet) possible...

Do you have any suggestions?