search

NeoGeographyToolkit / StereoPipeline

The NASA Ames Stereo Pipeline is a suite of automated geodesy & stereogrammetry tools designed for processing planetary imagery captured from orbiting and landed robotic explorers on other planets.
Apache License 2.0
479 stars 168 forks source link

Incorrect lMask #302

Closed foobarbecue closed 2 years ago

foobarbecue commented 4 years ago

Either I'm misunderstanding the masking process that happens in stereo_pprc, or the lMask that's being produced is incorrect. Can you confirm that this is not the intended behavior, and suggest what I might be doing wrong, or whether this is a bug?

lMask.tif and -L.tif screenshot from stereo_gui (seems correct): image

lMask.tif and -L.tif screenshot from stereo_gui (seems incorrect): image

As a result, I get spurious correlation in the output where there wasn't any input data (this is the DRG output from running point2dem with --orthoimage: image

To reproduce:

oleg-alexandrov commented 4 years ago

This does not look good but it is not unexpected. After the correlation happens, each pixel P in the left image is associated with a pixel Q in the right image. They both will be within their masks. However, when Q is the wrong point to associate with P, at triangulation stage you will get a 3D location that, when visualized, as you do, appears to be outside of the region covered by the images (and will have wrong height as well, but that you can't see in this picture).

We have some filters for dealing with this. point2dem should filter out the biggest triangulation errors (you can look at its params for doing that). There is also --erode-max-size in stereo filtering, and a bunch more options for that command. And in stereo_tri there is --near-universe-radius and --far-universe-radius.

You can also try the MGM algorithm --stereo-algorithm 2 (with parallel_stereo), which uses semi-global matching which should improve things, or at least the default --stereo-algorithm 0 with --subpixel-mode 3 with should do a better job at refinement.

You can also mapproject your images onto a smoother version of this DEM and redo stereo per the mapprojection chapter in the book. (Use parallel_stereo for that, and you can try with smaller clips.) That operation will unwarp some of the distortions due to the terrain being steep and will result in less wrong points.

I do agree that we could filter more even based on the picture you see, but it won't be just mask-based filtering, since, again, each of those 3D wrong points actually should project in both images, those points look sideways because point2dem "flattens" the cloud and a point that is deep down and a little sideways but still projecting into the cameas now looks just being to the side.

foobarbecue commented 4 years ago

Thanks so much for the detailed reply!

I took a close look at my settings and realized that I was using alignmentmethod affineepipolar. Since I had already map-projected the images beforehand using cam2map4stereo.py, I figured I would try setting alignmentmethod to none. My output now has no stray points to speak of!

With alignmentmethod none, my left mask is blank... I guess that's normal?

I'm going to close this since the the resolution doesn't require any changes to the software -- feel free to reopen if you disagree.

oleg-alexandrov commented 4 years ago

I am glad it works. I think we could indeed improve our filtering. With alignment method none and mapprojection at some point I put some logic that does extra filtering (https://github.com/NeoGeographyToolkit/StereoPipeline/blob/master/src/asp/Tools/stereo_pprc.cc#L317) but I don't think that should have kicked in for you as you used cam2map rather than our own mapprojection.

Anyhow, noise is always an issue and we could do more to deal with it. Not sure what to say about the left mask. Maybe blank here means that all the pixels are valid. I don't recall now how that logic works.

foobarbecue commented 4 years ago

Well, I've got masks on my mind again -- maybe just because everybody's wearing them!

After switching to MGM and looking into this further, I'm increasingly convinced that there's a problem in the implementation of masking.

My right mask makes sense, but my left mask is entirely blank (all 0 values).

My understanding is (please correct if wrong):

Here's a video to show this, in QGIS. Sorry about the jerky cursor.

The layers are (in order shown in the Layers panel on left):

maskingProblem2

(Man, I hate the gif format, but is there another way? It seems to be the only video upload format github will accept...)

For now, I have implemented my own masking using image footprint shapefiles and gdalwarp -crop_to_cutline , as a workaround.

I think this might actually be a good first issue for me to try and fix this myself. I haven't been able to successfully compile ASP yet but I've gotten pretty close... maybe this weekend.

PS- It's freaky what a good job MGM does of filling in elevations in areas where there is only one image! How does that happen!?

foobarbecue commented 4 years ago

Seems the problem is in the normalization step. nodata in left my input image becomes 0.0278 (greater than zero) in my stereo-L.tif file, whereas my nodata in my stereo-R.tif file the nodata becomes -0.091265 . The masking algorithm then presumably fails to understand that 0.0278 is nodata.

foobarbecue commented 4 years ago

Adding the --individually-normalize option makes the problem go away.

oleg-alexandrov commented 4 years ago

The mask not necessarily masks pixels in one image that are missing in the second image. In fact, our masking is very light-weight, sometimes it does nothing at all, so accepts all pixels as valid. I don't recall the details now.

It is odd that your mask has only zeros only. I'd think it should have only ones sometimes, or at least mostly ones and some zeros. Maybe if the --nodata-value in stereo is not set well it blanks the whole image. I am not sure. You can try running stereo with raw images, without cam2map, and see how it looks then. Anyhow, I don't recall running into empty masks before.

foobarbecue commented 4 years ago

I think the problem is that pair normalization is failing to preserve nodata in some edge cases. I'll submit a PR.

foobarbecue commented 4 years ago

By the way I'm trying to get ASP to compile... running into problems with boost when it's trying to link las2las from libLAS I think... I guess my boost library was compiled with the wrong gcc or something? ASP needs gcc 5 right?

oleg-alexandrov commented 4 years ago

Compiling ASP is a pain. You can just turn off liblas and the tools in ASP which depend on it, which consist, I hope, only of point2dem, point2las, and pc_align. Hope you have no other problems. Likely you will.

On Mon, Apr 6, 2020 at 10:09 PM Aaron Curtis notifications@github.com wrote:

Maybe it's the same as #290 https://github.com/NeoGeographyToolkit/StereoPipeline/issues/290 ? By the way I'm trying to get ASP to compile... running into problems with boost when it's trying to link las2las from libLAS I think.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/NeoGeographyToolkit/StereoPipeline/issues/302#issuecomment-610176612, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKDU3E2MVRZM7KWIN2TP6TRLKYOXANCNFSM4LSL7M4Q .

foobarbecue commented 4 years ago

Haha ok. I'll probably just try the virtual machine approach.