search

getlantern / laeproxy

Lantern App Engine Proxy - free proxy anyone can deploy to App Engine for use with Lantern desktop clients
GNU General Public License v3.0
6 stars 6 forks source link

tune RANGE_REQ_SIZE #3

Open lanterndev opened 12 years ago

lanterndev commented 12 years ago

When a Range header is not sent by the upstream requester, laeproxy requests a range of ~32MB, the current maximum App Engine response size. Because App Engine apps cannot send urlfetch responses until they've been completely downloaded into memory, the agent downstream can experience lag (in particular when the destination server is slow).

Relevant App Engine feature requests: http://code.google.com/p/googleappengine/issues/detail?id=1903 http://code.google.com/p/googleappengine/issues/detail?id=4888

lanterndev commented 12 years ago

The current urlfetch quotas impose a maximum rate of 22MB of bandwidth consumed per minute. So with our current RANGE_REQ_SIZE of ~32MB (the current max App Engine response size), we exceed the per-minute quota after a single (non-range) request for a sufficiently large resource.

Assuming the destination server supports range requests, the more we lower RANGE_REQ_SIZE, the more requests we'll be unable to fulfill in a single response for clients making regular, non-range requests under the assumption that we can serve larger responses, i.e. pre- getlantern/lantern#243 -fixed versions of Lantern.

lanterndev commented 12 years ago

@myleshorton, @forkner: Given the above, would lowering RANGE_REQ_SIZE to Lantern's new chunk size (2,000,000 bytes) make sense, or since Lantern will now only be making range requests of this size anyway, should laeproxy stick with a larger RANGE_REQ_SIZE to support older versions?

myleshorton commented 12 years ago

Yeah, those quotas are annoying -- definitely hit up against that in my testing.

I'm not too worried about old Lantern versions -- I say set it to the current Lantern chunk size. Any way to return to Lantern more detailed quota info so Lantern can request future chunks from somewhere else when getting close to quota?

Really need app engine premiere.