Get SRTM working with Docker

[dabreegster]

1) Installing libspatialindex-dev started failing because of 404, so I added an apt-get update first 2) The SRTM download failed, because the base image doesn't have curl by default

With these changes, I've successfully run the tool on montlake for all 3 data sources. Analyzing them now.

[dabreegster]

I spoke too soon -- srtm also requires GDAL:

curl -s -o spool/N47/N47W123.hgt.gz.temp https://s3.amazonaws.com/elevation-tiles-prod/skadi/N47/N47W123.hgt.gz && mv s
pool/N47/N47W123.hgt.gz.temp spool/N47/N47W123.hgt.gz
gunzip spool/N47/N47W123.hgt.gz 2>/dev/null || touch spool/N47/N47W123.hgt
gdal_translate -q -co TILED=YES -co COMPRESS=DEFLATE -co ZLEVEL=9 -co PREDICTOR=2 spool/N47/N47W123.hgt cache/N47/N47W123.tif 2>/dev/null || touch cache/N47/N47W123.tif
rm spool/N47/N47W123.hgt
make: Leaving directory '/root/.cache/elevation/SRTM1'
make: Entering directory '/root/.cache/elevation/SRTM1'
gdalbuildvrt -q -overwrite SRTM1.vrt
make: gdalbuildvrt: Command not found
Makefile:14: recipe for target 'SRTM1.vrt' failed

I tossed gdal-bin in the Docker's apt-get, but that messes up the PROJ version, just like on my local system. I'll find a workaround to get the GDAL binaries onto the image not using apt, likely using the compile instructions you sent earlier.

[dabreegster]

Just a progress update: I added the gdal install instructions to the end of the Dockerfile:

RUN wget http://download.osgeo.org/gdal/3.2.1/gdal-3.2.1.tar.gz && \
tar -xvzf gdal-3.2.1.tar.gz && \
cd gdal-3.2.1 && \
./configure --with-spatialite && \
make -j8 && \
make install

And it almost worked, failing during linking:

ar: `u' modifier ignored since `D' is the default (see `U')
/bin/sed: can't read /usr/lib/x86_64-linux-gnu/libsqlite3.la: No such file or directory
libtool:   error: '/usr/lib/x86_64-linux-gnu/libsqlite3.la' is not a valid libtool archive
make[1]: *** [libgdal.la] Error 1

I'll keep iterating

[eldang]

In the script that I yanked that GDAL install code from, the following are installed earlier with a comment marking them as PROJ prerequisites:

sudo apt-get install -y sqlite3 libsqlite3-dev libtiff-dev libcurl4-openssl-dev pkg-config

My suspicion is that one or both of the sqlite packages there are needed by GDAL and not actually needed by PROJ, so not already installed in the base Docker image you started from.

[dabreegster]

Thanks for the hints -- I got the full image to build now! I'll try running it and forcing SRTM tonight.

[eldang]

Is this now running reliably enough that it's worth merging?

[dabreegster]

Is this now running reliably enough that it's worth merging?

I'll do a fresh build and import data with it now to be certain

[dabreegster]

Nope. :( It builds fine, but at runtime: gdalbuildvrt: error while loading shared libraries: libgdal.so.28: cannot open shared object file: No such file or directory

[eldang]

That error message looked familiar, and sure enough https://stackoverflow.com/a/32807686/2121470 is in my browser history. I haven't actually run into this with Docker/linux, but I think I had to do something similar on my Mac to get an older version of GDAL working. So if libgdal.so.28 exists somewhere in that Docker image, it may be fixable with either a symlink or a PATH edit.

I wish I understood what the .number at the end signified, given that it never seems to line up with the GDAL version....

[eldang]

Correction: if a file with the exact name exists this should be fixable with a PATH edit. If libgdal.so or one with a different .number exists, try the symlink.

[dabreegster]

Well that was silly. I was originally using a base image with proj, but there's also one with gdal installed: https://hub.docker.com/r/osgeo/gdal

After switching to it, I got the SRTM import working end-to-end! The results appear to be breaking the contour visualization, but that's another issue I'm looking into next.