search

ajnisbet / opentopodata

Open alternative to the Google Elevation API!
https://www.opentopodata.org
MIT License
314 stars 68 forks source link

Add BKG data (Germany) #19

Closed janusw closed 3 years ago

janusw commented 3 years ago

One of the strengths of OpenTopoData is its ability to handle various data sources (and the documentation on how to do it). I'm interested in adding yet another data source, namely the DGM data from the German 'Bundesamt für Kartographie und Geodäsie' (BKG).

They have data in different resolutions. The free version has a grid size of 200m, see: https://gdz.bkg.bund.de/index.php/default/digitale-geodaten/digitale-gelandemodelle/digitales-gelandemodell-gitterweite-200-m-dgm200.html (unfortunately it seems like this page is only available in German).

Of course this 200m data as such is not very interesting (since EU-DEM covers Germany with 25m resolution), but BKG also offers higher-resolution data commercially (down to 5m at least). Therefore it would be nice if one could make the 200m data work with OpenTopoData as a proof of concept at least.

The data is available in XYZ-ASCII or GRID-ASCII format (ARC/INFO), both of which seem to be supported by gdal (via the XYZ and AAIGrid drivers), so I hope it should be no problem to use them with OpenTopoData.

Feedback welcome ...

janusw commented 3 years ago

Ok, I think I already got it working. Here is what I did ...

That's about it. First tests seem to indicate that these data roughly agree with EU-DEM for the few sample points I tried, which I take as an indication that this works well in principle.

janusw commented 3 years ago

Actually the above procedure only works in this order:

If one places both files into the data directory before starting the server, one gets the error:

Config Error: Unknown dataset type for 'bkg200m'. Dataset should either be a single file, or split up into tiles with the lower-left corner coordinate in the filename like 'N20W120'.

Without the prj file one gets:

Dataset has no coordinate reference system. Check the file '{path}' is a geo raster. Otherwise you'll have to add the crs manually with a tool like gdaltranslate.

@ajnisbet What would be the best way to deal with this data?

janusw commented 3 years ago

One other thing that I noticed is that querying from the asc file seems to be quite slow (compared to EU-DEM, for example). This also means that I cannot query more than 20 points or so at a time, otherwise I get a gateway timeout (504).

I'll try converting it to compressed geotiff, as suggested here: https://www.opentopodata.org/datasets/emod2018/

janusw commented 3 years ago

Converting the asc to a tif solved both the problems with the prj file and the performance issues.

$ gdal_translate -co COMPRESS=DEFLATE -co PREDICTOR=2 dgm200_tm32.asc dgm200_tm32.tif
Input file size is 3918, 4854
0...10...20...30...40...50...60...70...80...90...100 - done.

The tif file alone seems to be sufficient now, and the prj is not needed any more. Also the file size is down from 134M to 30M, and it is much faster indeed.

ajnisbet commented 3 years ago

Thanks for bringing this up! Yeah Open Topo Data doesn't handle auxiliary files like .prj very elegantly, when it sees multiple files it thinks you're using a tiled dataset so complains that the .prj file isn't a proper raster.

The immediate fix is to add ".prj" to https://github.com/ajnisbet/opentopodata/blob/362045478d48f3f45e3d94afe681f2bb450cc9b1/opentopodata/config.py#L16

I can't remember how I made that list of extensions, I thought I pulled it from the gdal source code and that it would be complete but clearly it isn't, so I'd like to do a bit more research to find an authoritative list or method to classify aux files.

ajnisbet commented 3 years ago

And yeah .asc is a non-binary format that is really slow, converting to .geotiff is a good move.

ajnisbet commented 3 years ago

Should be fixed by https://github.com/ajnisbet/opentopodata/pull/20, thanks for the PR!

ajnisbet commented 3 years ago

I added the 200m dataset here: https://github.com/ajnisbet/opentopodata/commit/7bf303bc4ee981baede269e4232df82301073e77

janusw commented 3 years ago

I added the 200m dataset here: 7bf303b

Nice! Thank you so much.

Btw, I found one really small problem in that commit: The alt-text for the image is wrong ...

  <img src="/img/bkg200m.png" alt="EU-DEM elevation">