newsdev / stevedore

search document dumps: ingest and explore in one extensible framework
Apache License 2.0
124 stars 10 forks source link

uploader sometimes chokes on big chunks #7

Closed jeremybmerrill closed 8 years ago

jeremybmerrill commented 8 years ago

if there's a

#<Manticore::SocketTimeout: Read timed out> Upload error: Read timed out Read timed out.

then we should cut the slice_size in half or something

jeremybmerrill commented 8 years ago

fixed with slice-size stuff ehre: https://github.com/newsdev/stevedore-uploader/commit/6948fe58dcfa4d839cd8e0b96c65b3abfb1d8c0f