feature request... alignment of images using match points

[jchollingsworth]

Hi,

Is there currently a way to collect matches between two mapprojected images, to estimate the transform which will bring them into alignment, and then save a mosaic of the two images?

This essentially what image_mosaic is doing. However, I have some images which are overlapping vertically. And it would be good to preserve the projection info if possible.

Cheers, J

[oleg-alexandrov]

No, there is no honest solution, maybe we will create one at some point. For now there is an inane solution. ipmatch can create match points between mapprojected images. pc_align can take those images and "align" them to each other using match points with the --match-file command and 0 iterations as if they were DEMs and write to disk the "aligned" version as a cloud (the image intensity is the height). That one can be converted back to a mapprojected image with point2dem (one can play with --search-radius-factor to reduce aliasing, and the --gaussian-sigma-factor for the Gaussian used is also customizable in point2dem).

Yes, inane this is, but note that mosaicking will have to use some kind of gridding and interpolation anyway, and Gaussian bluring and sampling could be as good as anything perhaps.

On Mon, Jun 3, 2019 at 8:34 AM jchollingsworth notifications@github.com wrote:

Hi,

Is there currently a way to collect matches between two mapprojected images, to estimate the transform which will bring them into alignment, and then save a mosaic of the two images?

This essentially what image_mosaic is doing. However, I have some images which are overlapping vertically. And it would be good to preserve the projection info if possible.

Cheers, J

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NeoGeographyToolkit/StereoPipeline/issues/265?email_source=notifications&email_token=AAKDU3DOVBSGRMZSV5N255TPYU2YXA5CNFSM4HSJIHCKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GXKD7VA, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKDU3FPYAZRJZHSBDF7AH3PYU2YXANCNFSM4HSJIHCA .

[jchollingsworth]

Actually, that's a nice solution. I was forgetting you can give pc_align match points.

Great :) thank you!

[ShashankBice]

Hi Oleg, I was wanting to test this from a long time, I tested it today! I took two clipped Landsat scenes from different dates, computed ips and matches, and used pc_align as pc_align --num-iterations 0 --match-file xyz.match ref.tif source.tif --save-transformed-source-point -o run/run . Next, I did point2dem with options of point2dem run/run-trans_source.tif --tr 15.0. To check the solution, I computed disparity between the initial ref.tif with the run/run-trans_source-DEM.tif, and the returned disparity shows a columnar pattern where the left image is disparity in x an right in disparity in y. Is this the effect of aliasing as you mentioned in your answer? Not a priority, just wanted to check this out ! Thanks :)

This is the match distribution and images for context:

[oleg-alexandrov]

I am not sure what to make of the columar pattern. In stereo_gui one can overlay georeferenced images. One can overlay them before this hackish alignment, and after. The hope is that after the alignment they are on top of each other, and that they would not get more blury. In short, this calls for a visual examination.

Note that of course this silly approach does not correct for local distortion that shows up when one mapprojects on a non-flat terrain.

[jchollingsworth]

This columnar disparity pattern looks normal to me... it tells you that the difference between the original and the transformed is a 2D ramp. Which sounds correct. It is simply showing you the misregistration error between two Landsat-8 scenes (which are co-registered to <<1 pixels). What you have done is to minimize the misregistration error using pc_align between two different scenes.

As Oleg says, this approach will only remove the very long wavelength misregistration error... you may have some high frequency noise, which will correlate with topography. That can be due to the orthorectification from using a lower res DEM (especially if the mis-registration error is somewhat high), and illumination differences between the images, which bias the correlator.

I'm also not sure what the precision of this alignment method is?... particularly at the sub-pixel scale. Maybe it's ok... I didn't test much. Would be interesting to know.

Cheers,

James

On Wed, Jul 17, 2019, 6:10 AM Oleg Alexandrov notifications@github.com wrote:

I am not sure what to make of the columar pattern. In stereo_gui one can overlay georeferenced images. One can overlay them before this hackish alignment, and after. The hope is that after the alignment they are on top of each other, and that they would not get more blury. In short, this calls for a visual examination.

Note that of course this silly approach does not correct for local distortion that shows up when one mapprojects on a non-flat terrain.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/NeoGeographyToolkit/StereoPipeline/issues/265?email_source=notifications&email_token=AJKN7DYW4GOUGBZSSR6W2SDP72SUDA5CNFSM4HSJIHCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2DBOMY#issuecomment-512104243, or mute the thread https://github.com/notifications/unsubscribe-auth/AJKN7D6C2S72ZXV6L52AYN3P72SUDANCNFSM4HSJIHCA .

[jchollingsworth]

Hi Oleg,

I also did some experimentation with this today... I noticed something odd. :(

I have two satellite images, cropped to the same region, and at the same resolution. I have lots of matches between the two (17474!): ipmatch gives the following terminal output: Found 17494 putative matches. --> Similarity: Matrix3x3((0.999932,2.61127e-05,0.031979)(-2.61127e-05,0.999932,-0.784817)(0,0,1)) Found 17474 final matches. Writing match file: src_ref.match

... so the transformation matrix given above looks sensible... basically there is only a very small shift between the two images, with almost no scaling difference.

However, when I run pc_align with this match file, and the two input images, I get this: Alignment transform (origin is planet center): 1.59836 -0.0519244 0.0745972 912492 0.0485487 1.59856 0.072472 2.58848e+06 -0.0768369 -0.070093 1.59756 -2.72169e+06 0 0 0 1 Maximum displacement of points between the source cloud with any initial transform applied to it and the source cloud after alignment to the reference: 0 m Transform scale - 1 = 0.742914

So the scaling factor is really really big. However, the scaling factor from ipmatch estimate is very small (which is correct). I can't think why pc_align is computing such a weird scaling?

My pc_align call is: pc_align --match-file src_ref.match --num-iterations 0 src.tif ref.tif -o trans_src_ref --save-inv-transformed-reference-points --max-displacement -1

(I've switched my src and ref, which might appear confusing... but I'm using --save-inv-transformed-reference-points).

If I run point2dem on the output trans_src_ref-trans_reference.tif, and then compare with the original (untransformed) image, I get this:

The "aligned" image is in the correct place, but with terrible scaling and nasty downsampling artifacts.

Maybe one way to help such situations is to have some control on the type of transform computed with pc_align when using match points (similar to when using the hillshade option). Switching off scaling would be a solution. Though I'd rather understand what's causing the initial problem :)

[oleg-alexandrov]

Uh oh. Maybe one can try to get just a handful of matches instead of the current large number and examine them carefully. Maybe it gets confused by outliers. I recall I put in outlier removal logic but go figure. One can also as a sanity check create just 4 match pairs by hand. Thank you for looking at this.

On Sat, Jul 20, 2019 at 3:45 AM jchollingsworth notifications@github.com wrote:

Hi Oleg,

I also did some experimentation with this today... I noticed something odd. :(

I have two satellite images, cropped to the same region, and at the same resolution. I have lots of matches between the two (17474!): [image: image] https://user-images.githubusercontent.com/39116687/61577332-519db200-aae6-11e9-809e-6240e0b218a0.png ipmatch gives the following terminal output: Found 17494 putative matches. --> Similarity: Matrix3x3((0.999932,2.61127e-05,0.031979)(-2.61127e-05,0.999932,-0.784817)(0,0,1)) Found 17474 final matches. Writing match file: src_ref.match

... so the transformation matrix given above looks sensible... basically there is only a very small shift between the two images, with almost no scaling difference.

However, when I run pc_align with this match file, and the two input images, I get this: Alignment transform (origin is planet center): 1.59836 -0.0519244 0.0745972 912492 0.0485487 1.59856 0.072472 2.58848e+06 -0.0768369 -0.070093 1.59756 -2.72169e+06 0 0 0 1 Maximum displacement of points between the source cloud with any initial transform applied to it and the source cloud after alignment to the reference: 0 m Transform scale - 1 = 0.742914

So the scaling factor is really really big. However, the scaling factor from ipmatch estimate is very small (which is correct). I can't think why pc_align is computing such a weird scaling?

My pc_align call is: pc_align --match-file src_ref.match --num-iterations 0 src.tif ref.tif -o trans_src_ref --save-inv-transformed-reference-points --max-displacement -1

(I've switched my src and ref, which might appear confusing... but I'm using --save-inv-transformed-reference-points).

If I run point2dem on the output trans_src_ref-trans_reference.tif, and then compare with the original (untransformed) image, I get this: [image: image] https://user-images.githubusercontent.com/39116687/61577741-dfc86700-aaeb-11e9-800b-f41cd94ef59a.png

The "aligned" image is in the correct place, but with terrible scaling and nasty downsampling artifacts.

Maybe one way to help such situations is to have some control on the type of transform computed with pc_align when using match points (similar to when using the hillshade option). Switching off scaling would be a solution. Though I'd rather understand what's causing the initial problem :)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/NeoGeographyToolkit/StereoPipeline/issues/265?email_source=notifications&email_token=AAKDU3A6AAXTYHBYWLCQLELQALUFTA5CNFSM4HSJIHCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NLYDY#issuecomment-513457167, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKDU3AJKYRLT4TDF3MAFLLQALUFTANCNFSM4HSJIHCA .

[oleg-alexandrov]

Is there currently a way to collect matches between two mapprojected images, to estimate the transform which will bring them into alignment, and then save a mosaic of the two images?

The new image_align tool appears to resolve this. It does not create a mosaic, but the second image is saved with alignment applied to it, so it can be mosaicked to the first one with dem_mosaic. https://stereopipeline.readthedocs.io/en/latest/tools/image_align.html