NeoGeographyToolkit / StereoPipeline

The NASA Ames Stereo Pipeline is a suite of automated geodesy & stereogrammetry tools designed for processing planetary imagery captured from orbiting and landed robotic explorers on other planets.
Apache License 2.0
493 stars 173 forks source link

Subpixel alignment in pc_align #230

Closed oleg-alexandrov closed 1 year ago

oleg-alexandrov commented 5 years ago

ICP, by definition, does not do subpixel alignment of DEMs, as it imagines the DEM as a cloud of points (at grid locations). That is good enough for generic alignment of clouds, but for alignment to a DEM we can do subpixel alignment, once a first alignment transform is computed to make the clouds roughly aligned.

To test if ICP subpixel alignment is good, one could create a DEM from a PC file, then shift the PC by half a grid point (using an initial transform in pc_align) and create a second DEM. The question is if pc_align can compute that half-pixel transform.

There are two ways to proceed here. The quick way is to see if the existing --alignment-method least-squares is any good (again with an initial transform already applied, otherwise this will fail). This one takes a point on the source DEM, projects it down to the datum, finds the lon and lat,, and does bilinear interpolation into the reference DEM to find the closest point. This is a rough approximation of course. A finer computation would then look at several neighbors, say 5 pixels each way from the point found previously, find the closest triangular face of the DEM (interpreted as surface) and the closest point on that triangulated surface. The nice thing about this alignment method is that it uses Google CERES so is very robust to outliers.

A second way is to add the Nuth and Kaab algorithm that David and Amaury use. This one is written in Python. One can test it first if it is any good before implementing it in C++ (since we don't wan to depend on numpy, etc).

oleg-alexandrov commented 5 years ago

Another idea (James's) is to use our --subpixel-mode 4 (Guizar-Sicairos sub-pixel phase correlator) to match hillshaded reference and source DEMs to get subpixel alignment. I am a bit skeptical of this. It is not often that reference and source DEMs are at similar enough resolutions that subpixel alignment of hillshaded images will result in subpixel alignment of the DEMs in pc_align, and that subpixel mode is also quite slow. Our hillshading mode is meant to correct gross alignment errors. I still think that once two DEMs are close to within a pixel alignment (well, just the reference must be a DEM), some kind of least squares solver that interprets the reference DEM as a surface should be easy to implement and give subpixel alignment. It would also work for a sparse source cloud.