search

thequbit / BarkingOwl

scalable web scraper framework for finding documents on websites.
GNU General Public License v3.0
19 stars 7 forks source link

If file type comes back with null, try again with bigger header size #40

Closed thequbit closed 9 years ago

thequbit commented 9 years ago

If the file type comes back as null, we should really try again ... since it should be SOMETHING we can type.

thequbit commented 9 years ago

Implemented. Configurable re-try count, that is defaulted to 6. If 6 errors happen, the entire file is downloaded.

fixed in version v0.6.1.1