bbangert / retools

Redis Tools
MIT License
138 stars 41 forks source link

Limiter functionality. #22

Closed heynemann closed 10 years ago

heynemann commented 10 years ago

I had to limit running parallel requests and thought of building my own lib. Thankfully I found retools and thought of contributing to it, instead.

The limiter class allows you to lock a specific number of items for a given operation. For instance:

limiter = Limiter(limit=10, prefix='parallel-requests')

for i in range(100):
    url = 'http://my.domain.com/%d.html' % i
    if limiter.acquire_limit(url):
        async_http_request(url, callback=handle_result)  # only 10 of these will get executed in parallel.

The app can then do something else or wait until it can acquire a limit. I hope you do consider this a generic enough construct that it should be in retools.

I tried hard to comply to the code notation and docs already in the project. If I missed something, please let me know and I'll gladly update the pull request.

The test covers 100% of the code and with both unit (mock redis) and functional (real redis) tests.

heynemann commented 10 years ago

Sorry about the failed build. Hopefully everything is ok now!

heynemann commented 10 years ago

@bbangert any chance this gets merged and a release done? I'm really in need of this code for my app. If you need anything else to be done, please let me know.

bbangert commented 10 years ago

Looks great, merging and releasing! :)

heynemann commented 10 years ago

thank you very much!