pc_align from ASP 2.7 yields different results than 2.6.2

[adehecq]

Running pc_align with hillshading option to coregister two DEMs fails using ASP 2.7.0 (latest build) while it works well with 2.6.2. Below is the precise command I use:

pc_align 
data/srtm-HMA-30m-lcc.dem_pair_1_2.TIF stereo_sub2/pair_1_2/DZB1205-500014L009_010-DEM.tif 
-o stereo_sub2/pair_1_2/temp_pc_v2.7.0-alpha-2020-06-18/tmp 
--save-transformed-source-points 
--initial-transform-from-hillshading similarity --alignment-method similarity-point-to-point 
--max-displacement 500 
--ipfind-options "--ip-per-image 5000000" 
--ipmatch-options "--inlier-threshold 10 --ransac-constraint similarity --ransac-iterations 100000" 

I believe that the alignment-method 'similarity-point-to-point' is not properly taken into account in the new pc_align, because the output transformation does not include any rotation (non diagonal terms are 0). See the logs below. The same issue arise with two previous versions that I had available: 2.6.2_post-2019-12-16-x86_64-Linux and 2.7.0-alpha-2020-06-18-x86_64-Linux. So the issue appeared between 2019-06-17 and 2019-12-16.

With ASP 2.7.0-2020-12-07-x86_64-Linux (high output error, 0 non diagonal terms):

2020-12-07 01:44:18 {0} [ console ] : Input: error percentile of smallest errors (meters): 16%: 15.396, 50%: 51.842, 84%: 150.607
2020-12-07 01:44:18 {0} [ console ] : Input: mean of smallest errors (meters): 25%: 12.0871, 50%: 24.9485, 75%: 41.4726, 100%: 78.4606
2020-12-07 01:44:18 {0} [ console ] : Initial error computation took 0.176425 [s]
2020-12-07 01:44:26 {0} [ console ] : Match ratio: 0.75001
2020-12-07 01:44:26 {0} [ console ] : Alignment took 7.93867 [s]
2020-12-07 01:44:26 {0} [ console ] : Number of errors: 100000
2020-12-07 01:44:26 {0} [ console ] : Output: error percentile of smallest errors (meters): 16%: 14.7076, 50%: 53.3027, 84%: 149.474
2020-12-07 01:44:26 {0} [ console ] : Output: mean of smallest errors (meters): 25%: 11.577, 50%: 24.3102, 75%: 42.8192, 100%: 76.9949
2020-12-07 01:44:26 {0} [ console ] : Final error computation took 0.174822 [s]
2020-12-07 01:44:26 {0} [ console ] : Alignment transform (origin is planet center):
0.992479        0        0  8514.96
       0 0.992479        0  37651.7
       0        0 0.992479  31072.5
       0        0        0        1

With ASP 2.6.2-2019-06-17-x86_64-Linux (low output error, non 0 non diagonal terms):

2020-12-07 01:40:47 {0} [ console ] : Input: error percentile of smallest errors (meters): 16%: 15.5053, 50%: 51.4214, 84%: 149.964
2020-12-07 01:40:47 {0} [ console ] : Input: mean of smallest errors (meters): 25%: 12.1523, 50%: 24.8503, 75%: 41.3294, 100%: 78.1723
2020-12-07 01:40:47 {0} [ console ] : Initial error computation took 0.20816 [s]
2020-12-07 01:41:05 {0} [ console ] : Match ratio: 0.75001
2020-12-07 01:41:05 {0} [ console ] : Alignment took 18.4194 [s]
2020-12-07 01:41:05 {0} [ console ] : Number of errors: 100000
2020-12-07 01:41:05 {0} [ console ] : Output: error percentile of smallest errors (meters): 16%: 1.68016, 50%: 6.35118, 84%: 19.8867
2020-12-07 01:41:05 {0} [ console ] : Output: mean of smallest errors (meters): 25%: 1.32339, 50%: 2.85572, 75%: 5.0618, 100%: 11.1268
2020-12-07 01:41:05 {0} [ console ] : Final error computation took 0.204415 [s]
2020-12-07 01:41:05 {0} [ console ] : Alignment transform (origin is planet center):
   0.993255  0.00165594  -0.0036792     15452.4
-0.00166092    0.993261 -0.00134152     41303.5
 0.00367695  0.00134766    0.993256     17847.9
          0           0           0           1

Note: I don't believe this comes from the IP matches. There is a very slight difference between the IPs found with both versions, but not significant to cause the issue. As you can see, the input errors in both cases are very similar. I have tried running pc_align v 2.7.0 with the initial transform obtained from pc_align 2.6.2, and then it gives good results, simply because the alignment-method is this time correctly used and the non diagonal terms are not 0:

StereoPipeline-2.7.0-2020-12-07-x86_64-Linux/libexec/pc_align 
data/srtm-HMA-30m-lcc.dem_pair_1_2.TIF stereo_sub2/pair_1_2/DZB1205-500014L009_010-DEM.tif 
-o stereo_sub2/pair_1_2/temp_pc_v2.7_with_2.6.2_transf/tmp 
--save-transformed-source-points 
--max-displacement 500
--initial-transform stereo_sub2/pair_1_2/temp_pc_v2.6.2-2019-06-17/tmp-transform.txt 
--alignment-method similarity-point-to-point 

yields to satisfying results:

2020-12-07 01:51:34 {0} [ console ] : Input: error percentile of smallest errors (meters): 16%: 1.70951, 50%: 6.41674, 84%: 20.2244
2020-12-07 01:51:34 {0} [ console ] : Input: mean of smallest errors (meters): 25%: 1.33957, 50%: 2.88833, 75%: 5.12177, 100%: 13.9284
2020-12-07 01:51:34 {0} [ console ] : Initial error computation took 0.173587 [s]
2020-12-07 01:51:42 {0} [ console ] : Match ratio: 0.75001
2020-12-07 01:51:42 {0} [ console ] : Alignment took 7.80338 [s]
2020-12-07 01:51:42 {0} [ console ] : Number of errors: 100000
2020-12-07 01:51:42 {0} [ console ] : Output: error percentile of smallest errors (meters): 16%: 9.25377, 50%: 19.164, 84%: 32.3702
2020-12-07 01:51:42 {0} [ console ] : Output: mean of smallest errors (meters): 25%: 7.18006, 50%: 11.6504, 75%: 15.3208, 100%: 24.097
2020-12-07 01:51:42 {0} [ console ] : Final error computation took 0.172908 [s]
2020-12-07 01:51:42 {0} [ console ] : Alignment transform (origin is planet center):
   0.993255  0.00165594  -0.0036792     15468.7
-0.00166092    0.993261 -0.00134152     41299.4
 0.00367695  0.00134766    0.993256     17821.6
          0           0           0           1

[oleg-alexandrov]

This looks like a big deal. I don't know when I will have time to look at it, regretfully.

[adehecq]

Thanks. In the meantime, is there a way to replace the new pc_align with the older pc_align in my ASP folder? (I would like to avoid hard-coding paths to the scripts in my code) I tried setting a symlink to bin/pc_align and libexec/pc_align but it didn't make a difference.

[oleg-alexandrov]

The pc_align in the bin directory is just a shell script. You can edit it, and change all the environmental variables there to point to your old directory and old executable. It should work, I see no reason why not, but you need to ensure all variables are modified that way.

On Sun, Dec 13, 2020 at 11:29 PM Amaury Dehecq notifications@github.com wrote:

Thanks. In the meantime, is there a way to replace the new pc_align with the older pc_align in my ASP folder? (I would like to avoid hard-coding paths to the scripts in my code) I tried setting a symlink to bin/pc_align and libexec/pc_align but it didn't make a difference.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/NeoGeographyToolkit/StereoPipeline/issues/324#issuecomment-744234589, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKDU3GSEALHB6N4U42LLQDSUW5FDANCNFSM4UQJN3EA .

[adehecq]

Ok it makes sense! I modified the 6th line to point to the other version in bin/pc_align: TOPLEVEL=~/bin/StereoPipeline-2.6.2-2019-06-17-x86_64-Linux/ And that solves the issue (temporarily).

[oleg-alexandrov]

My apologies for not looking at this so far. Now got back to ASP work for a while.

This was a bug which was happening when --initial-transform-from-hillshading similarity was on and outlier removal was taking place. I should have paid more attention when implementing this.

I now also exposed the outlier removal parameters for this mode, via

--initial-transform-outlier-removal-params pct factor

This should make it to the daily build in a day or so.

[adehecq]

Thanks!

[adehecq]

What is the status of this issue? I'm not sure the fix you mention @oleg-alexandrov solved the issue.

Even with ASP 3.0, I keep having consistently bad results with pc_align, with the alignment method 'similarity-point-to-point'. By bad, I mean that in most cases, the output error is larger than the input error.

To make it simpler, I worked on a case without the --initial-transform-from-hillshading and for an already aligned, nice and easy DEM. Here is the command I run:

pc_align stereo_sub2_mgm_xc0/coreg/ref_dem_resmasked.tif \
stereo_sub2_mgm_xc0/coreg/DZB1210-500032L003_004-DEM_coreg.tif \
-o stereo_sub2_mgm_xc0/coreg/test_pc2/tmp1 \
--save-transformed-source-points --alignment-method similarity-point-to-point --max-displacement 50

This is what I get. With ASP 3.0.0-2021-07-27-x86_64-Linux

Input: error percentile of smallest errors (meters): 16%: 0.75, 50%: 2.55273, 84%: 5.58984
Input: mean of smallest errors (meters): 25%: 0.588015, 50%: 1.21804, 75%: 1.95526, 100%: 3.26808
Initial error computation took 0.198846 [s]
writing to stereo_sub2_mgm_xc0/coreg/test_pc/tmp1-iterationInfo.csv
Match ratio: 0.75001
Alignment took 11.3365 [s]
Number of errors: 100000
Output: error percentile of smallest errors (meters): 16%: 14.367, 50%: 18.318, 84%: 22.342
Output: mean of smallest errors (meters): 25%: 12.944, 50%: 15.0043, 75%: 16.5345, 100%: 18.3476
Final error computation took 0.184348 [s]
Alignment transform (origin is planet center):
       1        0        0   9.7216
       0        1        0 -7.08055
       0        0        1 -24.1511
       0        0        0        1

Only the identity matrix, error increases.

With ASP 2.6.2-2019-06-17-x86_64-Linux

Input: error percentile of smallest errors (meters): 16%: 0.75, 50%: 2.55273, 84%: 5.58984
Input: mean of smallest errors (meters): 25%: 0.588015, 50%: 1.21804, 75%: 1.95526, 100%: 3.26808
Initial error computation took 0.198491 [s]
writing to stereo_sub2_mgm_xc0/coreg/test_pc2/tmp1-iterationInfo.csv
Match ratio: 0.75001
Alignment took 13.6677 [s]
Number of errors: 100000
Output: error percentile of smallest errors (meters): 16%: 0.756321, 50%: 2.56466, 84%: 5.61376
Output: mean of smallest errors (meters): 25%: 0.590612, 50%: 1.22375, 75%: 1.96422, 100%: 3.28155
Final error computation took 0.226061 [s]
Alignment transform (origin is planet center):
     0.99997  2.77446e-05  1.02807e-05      33.9678
-2.77447e-05      0.99997  5.68029e-06     -73.3428
-1.02806e-05 -5.68057e-06      0.99997      173.883
           0            0            0            1

Non-zero non-diagonal elements, error nearly constant (it means my initial alignment was already perfect 😉).

[adehecq]

To illustrate, this is what the content of the end-errors.csv looks like, in the 2.6.2 and 3.0 cases respectively.

[adehecq]

On the other hand, version 3.0 works much better with the option '--match-file', so you definitely fixed some issues!

[oleg-alexandrov]

The bug I fixed was for --initial-transform-from-hillshading similarity. I suggest you look at this, without the --alignment-method similarity-point-to-point option, and compare ASP 3.0.0 to the one which you think was working better.

I guess --alignment-method similarity-point-to-point is another issue. I hope to find time to look at before the fiscal year is over at the end of the month, as then a new task list comes in, with other specific goals and many of them not related to ASP.

[adehecq]

Yes, that was the point of my previous message. I didn't test the --initial-transform-from-hillshading similarity option specifically, because I now generate the hillshades and matches outside of pc_align, but I feed in the match file with the option --match-file and it works much better now. So yes, your fix solved that issue, thanks!

In the very first example I provided, I was using both --initial-transform-from-hillshading similarity and --alignment-method similarity-point-to-point so the errors I was seeing were probably a combination of that issue you fixed, and the other one I just reported. Thanks in advance for looking into this!

[oleg-alexandrov]

I fixed the alignment methods point-to-point and similarity-point-to-point. Those actually broke after updating libpointmather a while ago, and I found and fixed the offending code in our fork of that library.

Now, that library has moved on since, and I'd need to check at some point if they still have that issue.

I added a new alignment method, similarity-point-to-plane. This works better than similarity-point-to-point when the translation is big and there is some scale difference, but does the same for small translation. (Just as point-to-plane works better than point-to-point in this situation.)