search

mapbox / preprocessorcerer

Perform preprocessorcery and pick parts on particularly persnickity uploads
ISC License
12 stars 8 forks source link

exit 3 on unrecognized projection #48

Open isiyu opened 9 years ago

isiyu commented 9 years ago

preprocessorcerer was exiting with an exit code of 8 because of an unrecognized projection. Per chat with @rclark preprocessorcerer should identify these cases and exit 3, along with an appropriate error message

cc @GretaCB

rclark commented 9 years ago

Could you provide a link to a file with an unrecognized projection? We'll need a fixture to test this case.

isiyu commented 9 years ago

this was the test case upload that was causing the issue. s3://mapbox/tmp/isiyu/dimentians_unrecognizedProjection.tif

mapsam commented 8 years ago

Digging into this one a bit more, I resized the tiff provided by @isiyu so it's a bit more manageable:

s3://mapbox/playground/mapsam/dimentians_unrecognizedProjection-small.tif

We get segfault when running perform-preprocessorcery dimentians_unrecognizedProjection-small.tif. This originates in wmtiff, so I think the issue should be handled there. https://github.com/mapbox/node-wmtiff/issues/9

mapsam commented 8 years ago

Just noting, that even our try/catch here https://github.com/mapbox/preprocessorcerer/blob/master/preprocessors/tif-reproject.preprocessor.js#L6-L7 isn't failing, and preprocessorcerer just hangs without failing. Is it worth making wmtiff asynchronous so we can catch and return errors from gdal?

@isiyu @rclark @GretaCB

springmeyer commented 7 years ago

Is it worth making wmtiff asynchronous so we can catch and return errors from gdal?

Per https://github.com/mapbox/node-wmtiff/issues/9#issuecomment-305852692 I don't think making wmtiff async will make any difference. The segfault needs to be fixed at the source.

just hangs without failing - It is possible that is a symptom of the underlying bug. Sometimes a segfault can manifest randomly as a hang in the threadpool (I see this on OS X occasionally). So, you might run the test and 9 times out of 10 it would segfault, but the other time it will hang. Profiling in Activity Monitor on OS X will show one the the threads hung in memory allocation. An explanation for this is that a bug that causes a segfault is potentially corrupting memory. When memory is corrupted anything goes.

GretaCB commented 7 years ago

When memory is corrupted anything goes.

This makes so much sense. I was locally testing scanline vs pyramid tiling for a troublesome file last week, and sometimes it would just hang/timeout, other times it would segfault. Thanks for this response @springmeyer