Program stopped after one iteration using dem_align.py command

[wandi12345]

Hi, Thank you for sharing. I am trying to register SRTM and Tandem x, but there is an error that I cannot search for the reason. Typed this command in the terminal: python3 dem_align.py -mode nuth ../data/SRTM.tif ../data/20131017.dem.tif Stopped after only one iteration.

Below is the running log:

Reference: ../data/SRTM.tif Source: ../data/20131017.dem.tif Mode: nuth Output: ../data/20131017.dem_dem_align/20131017.dem_SRTM

wandi@mefe54 172 <.../demcoreg/demcoreg>:python3 dem_align.py -mode nuth ../data/SRTM.tif ../data/20131017.dem.tif

Reference: ../data/SRTM.tif Source: ../data/20131017.dem.tif Mode: nuth Output: ../data/20131017.dem_dem_align/20131017.dem_SRTM

Warping all inputs to the following: Resolution: 0.0001701388891388889 Extent: [109.15996525581751, 30.199972163502004, 109.2100347006625, 30.240027719378002] Projection: '+proj=longlat +datum=WGS84 +no_defs' Resampling alg: cubic

1 of 2: ../data/SRTM.tif nl: 235 ns: 294 res: 0.000 0...10...20...30...40...50...60...70...80...90...100 - done. 2 of 2: [memory] nl: 235 ns: 294 res: 0.000 0...10...20...30...40...50...60...70...80...90...100 - done.

Reference DEM res: 0.00 Source DEM res: 0.00 Resolution for coreg: mean (0.00 m)

Iteration 1

Warping all inputs to the following: Resolution: 0.0001701388891388889 Extent: [109.15996525581751, 30.200045080430364, 109.20998608922434, 30.240027719378002] Projection: '+proj=longlat +datum=WGS84 +no_defs' Resampling alg: cubic

1 of 2: [memory] 2 of 2: [memory] Elevation difference stats for uncorrected input DEMs (src - ref) Removing outliers Initial pixel count: 69090 Absolute dz filter: 100.00 69090 Excluding values outside of range: -26.283 +/- 3*4.283 Excluding values outside of range: -39.131017 to -13.435633 63174 Computing slope Slope filter: 0.10 - 40.00 Initial count: 68036 Excluding values outside of range: 0.100000 to 40.000000 65121 Computing aspect Filtered difference map count: 60601 min: -39.12 max: -13.44 mean: -25.90 std: 4.35 med: -26.30 mad: 3.69 q1: -28.43 q2: -23.39 iqr: 5.04 mode: -26.51 p16: -29.81 p84: -21.66 spread: 4.07 Computing sub-pixel offset between DEMs using mode: nuth Computing common mask Initial sample count: 60601 Removing outliers 3-sigma filter: -780.30 - 543.09 60074 Computing 1-degree bin statistics: count Computing 1-degree bin statistics: median Computing fit [-11.40113551 -53.66257759 -79.27599968] Generating Nuth and Kaab plot /home/wandi/lz/demcoreg/demcoreg/demcoreg/demcoreg/coreglib.py:337: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray bp = ax.boxplot(np.array(output)[idx][::s], positions=bin_centers[::s], widths=widths[::s], showfliers=False, \ Median dz: -26.30 Nuth dz: -27.61 Incremental offset: dx=-9.18m, dy=+6.76m, dz=+26.30m Cumulative offset: dx=-9.18m, dy=+6.76m, dz=+26.30m Writing offset plot: ../data/20131017.dem_dem_align/20131017.dem_SRTM_nuth_iter01_plot.png Segmentation fault (core dumped)

[dshean]

Looks like your input DEMs are in geographic coordinates (EPSG:4326). Input DEMs should be in projected coordinate system with units of meters. See other recent issues for discussion. We need to add a check for this: https://github.com/dshean/demcoreg/issues/28

[wandi12345]

@dshean Thank you very much for your reply. But, the problem still exists after I input dem in projeced coordinate system. Still stopped after the first iteration.

(pygeotools-mefe54) wandi@mefe54 116 <.../demcoreg/demcoreg>:python3 dem_align.py -mode nuth ../data/SRTM.p.tif ../data/20131017.p.tif

Reference: ../data/SRTM.p.tif Source: ../data/20131017.p.tif Mode: nuth Output: ../data/20131017.p_dem_align/20131017.p_SRTM.p

Warping all inputs to the following: Resolution: 17.585 Extent: [322393.847, 3354357.957, 325105.927, 3361053.877] Projection: '+proj=tmerc +lat_0=0 +lon_0=111 +k=0.999599993228912 +x_0=500000 +y_0=0 +datum=NAD27 +units=m +no_defs' Resampling alg: cubic

1 of 2: ../data/SRTM.p.tif nl: 381 ns: 154 res: 17.585 0...10...20...30...40...50...60...70...80...90...100 - done. 2 of 2: [memory] nl: 381 ns: 154 res: 17.585 0...10...20...30...40...50...60...70...80...90...100 - done.

Reference DEM res: 28.75 Source DEM res: 6.42 Resolution for coreg: mean (17.59 m)

Iteration 1

Warping all inputs to the following: Resolution: 17.585 Extent: [322393.847, 3354353.992, 325101.93700000003, 3361053.877] Projection: '+proj=tmerc +lat_0=0 +lon_0=111 +k=0.999599993228912 +x_0=500000 +y_0=0 +datum=NAD27 +units=m +no_defs' Resampling alg: cubic

1 of 2: [memory] 2 of 2: [memory] Elevation difference stats for uncorrected input DEMs (src - ref) Removing outliers Initial pixel count: 55847 Absolute dz filter: 100.00 55221 Excluding values outside of range: -27.078 +/- 3*5.193 Excluding values outside of range: -42.656915 to -11.498969 52159 Computing slope Slope filter: 0.10 - 40.00 Initial count: 54806 Excluding values outside of range: 0.100000 to 40.000000 53694 Computing aspect Filtered difference map count: 51301 min: -42.65 max: -11.50 mean: -27.26 std: 5.58 med: -27.06 mad: 4.70 q1: -30.56 q2: -24.20 iqr: 6.36 mode: -27.18 p16: -32.69 p84: -22.15 spread: 5.27 Computing sub-pixel offset between DEMs using mode: nuth Computing common mask Initial sample count: 51301 Removing outliers 3-sigma filter: -1247.62 - 894.56 50666 Computing 1-degree bin statistics: count Computing 1-degree bin statistics: median Computing fit [ 22.25574828 -48.16465365 -85.73928272] Generating Nuth and Kaab plot /home/wandi/lz/demcoreg/demcoreg/demcoreg/demcoreg/coreglib.py:337: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray bp = ax.boxplot(np.array(output)[idx][::s], positions=bin_centers[::s], widths=widths[::s], showfliers=False, \ Median dz: -27.06 Nuth dz: -28.27 Incremental offset: dx=+16.58m, dy=-14.84m, dz=+27.06m Cumulative offset: dx=+16.58m, dy=-14.84m, dz=+27.06m Writing offset plot: ../data/20131017.p_dem_align/20131017.p_SRTM.p_nuth_iter01_plot.png Segmentation fault (core dumped)

[dshean]

Hmm. Is that png written out successfully?
Seg fault is not good, but I'm thinking this may be related to your environment/install or the input data files, and not dem_align.py.
Are you using the latest version of demcoreg package from github? Can you provide more information about your Python install? Can you provide the two tif files so we can attempt to reproduce?

[wandi12345]

@dshean Thank you for your quick reply. these are two data. data.zip This is install information wandi@mefe54 102 <.../demcoreg/demcoreg>:pip3 list Package Version Location

---

certifi 2020.6.20 chardet 4.0.0 cycler 0.10.0 demcoreg 0.5.0 GDAL 3.0.4 idna 2.10 imview 0.3.0 /home/wandi/demcoreg/demcoreg/imview kiwisolver 1.3.1 lib 3.0.0 matplotlib 3.4.0 matplotlib-colorbar 0.4.0 matplotlib-scalebar 0.7.2 numpy 1.18.5 Pillow 8.1.2 pip 21.0.1 pygeotools 0.6.0 pyparsing 2.4.7 pyproj 3.0.1 python-dateutil 2.8.1 requests 2.25.1 scipy 1.6.2 setuptools 51.1.0 six 1.15.0 urllib3 1.26.4 wget 3.2 wheel 0.34.2

[wandi12345]

and, png is not written out.

[dshean]

OK, I was unable to reproduce using latest from demcoreg and pygeotools.
Hard for me to diagnose. I think there may be issues with your data files, specifically your reprojection step - you specified the outdated NAD27 datum for N America and some transverse mercator with center longitude of +111° which is in China. Depending on your site, I recommend you specify the appropriate UTM zone relative to the WGS84 ellipsoid (e.g., https://www.spatialreference.org/ref/epsg/wgs-84-utm-zone-12n/) It's odd that you would get segfault while writing out a relatively small png. Maybe an issue with your local environment. If correct projection doesnt' fix the issue, could try with a fresh conda install of dependencies and latest source.

[wandi12345]

@dshean Thank you very much. After changing to AST data, the results can be obtained. There is a problem with my data, not dem_align.py.