search

internetarchive / liveweb

Liveweb proxy of the Wayback Machine project
https://web.archive.org/
44 stars 13 forks source link

Implement size and time limits for fetching web pages #7

Closed nibrahim closed 12 years ago

nibrahim commented 12 years ago

The arc_proxy, while fetching a resource should have a time and size limit so that it is more reliable.

nibrahim commented 12 years ago

Threshold on size to keep resources in memory : 1MB Maximum payload size allowed : 1GB Absolute time limit : 60 seconds.

anandology commented 12 years ago

Can be configured by setting max_request_time and max_response_size config parameters.