NeoGeographyToolkit / StereoPipeline

The NASA Ames Stereo Pipeline is a suite of automated geodesy & stereogrammetry tools designed for processing planetary imagery captured from orbiting and landed robotic explorers on other planets.
Apache License 2.0
479 stars 168 forks source link

Handle better ISIS cubes that mishandle no-data #290

Closed oleg-alexandrov closed 10 months ago

oleg-alexandrov commented 4 years ago

Sometimes ISIS cubes don't handle no-data values properly, with the result that some pixels are assigned a valid value of -340282346638529993179660072199368212480.000, or something like that.

ASP's stereo and mapproject choke on this. Worse, mapproject is able to create equally large pixels but with opposite sign somehow when it does interpolation, and after this a holy mess results.

The easiest fix for mapproject is to be able to mask out not just a given value, but any values less than or equal to a value, when it is invoked with --nodata-value. So create_mask should be replaced with create_mask_less_or_equal. This should hopefully not result in large positive values either. The stereo command already works that way.

Of course, the user still needs to pass an appropriate --nodata-value, which should be not the one declared in the .cub file, which is the smallest float, which allows monstrosities as in the first paragraph to squeak though, but something somewhat larger than that.

An ideal fix would go further. Such scenarios should be detected automatically and quietly fixed rather than the user having to specify a correct nodata cutoff.

I ran into this with the Mimas cub file N1644778567 but likely can be easy to test with any data. I recall I ran into this a while ago as well, so the issue could be more general than that.

rbeyer commented 4 years ago

An example file would be useful.

The trick is that while GeoTIFFs allow for only a single --nodata-value, ISIS cubes can have many, typically 5 (Null, LRS, LIS, HRS, HIS). Basically, for each 'pixel type' of ISIS cube, there are different valid numerical ranges, and values above or below that valid range are 'special' pixels, essentially there are whole ranges of "--nodata-value".

These values are defined here: https://github.com/USGS-Astrogeology/ISIS3/blob/dev/isis/src/base/objs/SpecialPixel/SpecialPixel.h

Perhaps the first test would be to run the ISIS program stats and see if the particular file has multiple 'special pixels.' If so, you can try and "collapse" these all to a single value. You might be able to do it with fewer steps, but you can use stretch to first convert the non-Null special pixels to some actual, but unlikely 'real' pixel value, then use specpix to convert those values to NULL. After this two-step, you should have an ISIS cube that stats should only show Valid and Null pixel values, but no LIS, LRS, HIS, or HRS pixels. Now a process (like our mapproject) which can only deal with a single no-data value should work as expected.

oleg-alexandrov commented 10 months ago

This issue hasn't occurred since.