search

simonaoliver / metageta

Automatically exported from code.google.com/p/metageta
Other
0 stars 0 forks source link

metageta suspends work when extracting info from partially written ECW #30

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Steps that will reproduce the problem?
1. crawl folder containing partially written ECW - typically tmp files with an 
ecw prefix are located in the write directory.

Once incomplete file in encountered crawler suspends work

Ideal is for crawler to identify partially written file and skip to the next 
file.

Original issue reported on code.google.com by simonaol...@gmail.com on 11 Jun 2010 at 3:49

GoogleCodeExporter commented 9 years ago

Original comment by pinner.luke@gmail.com on 11 Jun 2010 at 4:13

GoogleCodeExporter commented 9 years ago 
The crawler works on that particular ECW. It just takes about 3 hours to 
process. I assume this is because the file would be ~45GB uncompressed and is 
not properly optimised (as it is very incomplete).

Original comment by pinner.luke@gmail.com on 16 Jun 2010 at 12:01