couchbaselabs / cbfs

Distributed Blobstore using Couchbase Server
http://dustin.github.com/2012/09/27/cbfs.html
Apache License 2.0
188 stars 40 forks source link

Don't request objects already being retrieved. #120

Closed dustin closed 11 years ago

dustin commented 11 years ago

We burn a lot of resources having multiple concurrent requests for the same objects that we already have open request to retrieve under certain situations.

We just need a set of object IDs currently being retrieved, add it to the set when a request somes in and remove it when we complete the request. Only queue processing if the entry didn't exist.