Asynchronous requests with grequests?

[sprin]

grequests can be dropped in for asynchronous API requests with gevent: sprin/box.py@1cb1c30a39a9e8a08c52808716157e27c5be9dd6

You probably don't want to drag gevent/grequests in as a hard deps, but perhaps grequests can be imported conditionally?

This change enabled me do recursive operations almost two orders of magnitude faster, eventually running into rate-limiting. However, the pool can be used to limit the number of concurrent requests to avoid rate-limiting.

Here is recursive folder walking and folder creation done asynchronously.

Aside: If the those recursive functions are of interest to integrate into box.py, I'd be happy to make it more general and provide tests.

[tals]

I'm not sure if I understand - grequests is there to provide convenience if you want to build batches of stuff or do things asynchronously. But the code in the patch just spawns a job and waits on it right away. Wouldn't the control of the greenlet be taken away anyway when they hit the httplib part? You can achieve the rate-limiting by just using a pool in your code's get_subfolders_inner() (right now you're doing gevent.spawn), instead of pulling it into the box client.

I've actually been thinking of making the library more friendly by providing helpers and wrapping the api output in a nicer structure (so things like iterating over a folder would be much more pleasant to do). Maybe snag the model pieces from uber.py? btw - I saw the json() hack in ItemAlreadyExists. Would be nice if the exception would just have the id of the pre-existing object when its available (added to my todo list) :)

[sprin]

You can achieve the rate-limiting by just using a pool in your code's get_subfolders_inner() (right now you're doing gevent.spawn), instead of pulling it into the box client."

Yes, I came to the same conclusion later on. I don't think there's much to be gained by making box.py gevent-aware since each call to the client can be wrapped in a greentlet spawned from a pool.

I've actually been thinking of making the library more friendly by providing helpers and wrapping the api output in a nicer structure

I am not a fan of data wrapper classes. When I get complex JSON from an API, the first thing I do is to simplify it into a structure using lists/dicts that makes sense to me before passing it around in my application.

Would be nice if the exception would just have the id of the pre-existing object when its available (added to my todo list) :)

Thanks!