Open brocke opened 4 years ago
max-dn
commented
4 years ago See #191 ... We don't have better elevation data for the whole earth with a open license. So we have to accept these errors (an other example is this crater a little bit south)
brocke
commented
4 years ago OK, I see. I was nearly expecting something like this. I hope someday better data will be available to make OpenTopoMap even better.
lukey78
commented
4 years ago @max-dn I've now successfully created hillshade and contour lines from a set of worldwide DEMs with the following order: SRTMv3 1" -- Viewinder 1" (filling gaps) -- Viewfinder 3" (filling gaps) -- Sonnys DTM Europe Tiles 1" (overwrite).
This gives considerably better elevation data, but all .tif files are 340GB, and the contours database takes 1.5 TB...
lukey78
commented
4 years ago @max-dn: I noticed some strange behaviour with the 30x30m DEM (raw.tif) and the isolations calculation. In some areas way too many peaks are displayed in low zoom levels (otm_isolation = 100000). Do you have an idea if I need to change something in isolations.c?
This is the error I get, but I'm not sure if it's relevant:
ERROR 5: /mnt/data/srtm/raw.tif, band 1: Access window out of range in RasterIO(). Requested (408528,-3753) of size 2x2 on raster of 1285717x621432.
mboeringa
commented
4 years ago @max-dn: I noticed some strange behaviour with the 30x30m DEM (raw.tif) and the isolations calculation. In some areas way too many peaks are displayed in low zoom levels (otm_isolation = 100000). Do you have an idea if I need to change something in isolations.c?
To be honest, as to this specific issue of mountain top isolation, I would make the list of calculated isolation for each peak as done for OTM available online and create some MapRoulette challenge for it to have users add it to the existing peaks as a new tag. That way, anybody can use the data without need of pre-processing and complex calculations on huge raster datasets.
It is not like those mountains will move anytime soon... ;-)
So no need for dynamic calculation, enrich OSM once for the benefit of all, and be done with it. The same could be said for directions of mountain passes, if that is part OTM preprocessing, which I think it isn't yet, is it? I remember seeing at least one other website doing such calculations though...
max-dn
commented
4 years ago I noticed some strange behaviour with the 30x30m DEM (raw.tif) and the isolations calculation. In some areas way too many peaks are displayed in low zoom levels (otm_isolation = 100000). Do you have an idea if I need to change something in isolations.c?
I don't think... I tested it with a 50m grid (Bavaria) and a 10m grid (Austria) and both worked without adaptation.
ERROR 5: /mnt/data/srtm/raw.tif, band 1: Access window out of range in RasterIO(). Requested (408528,-3753) of size 2x2 on raster of 1285717x621432.
The error message just says that you found a mountain less than 100km from the border of the DEM-file.
I would make the list of calculated isolation for each peak as done for OTM available online and create some MapRoulette challenge for it to have users add it to the existing peaks as a new tag. That way, anybody can use the data without need of pre-processing and complex calculations on huge raster datasets.
I have weekly updated data of all isolations of peaks and orientations of the saddles: https://geo.dianacht.de/topo/saddledirection_viefinderpanoramas.100.txt https://geo.dianacht.de/topo/topographic_isolation_viefinderpanoramas.txt
I don't think these values should be mapped into OSM:
Every single one of these points can be seen completely different and for every other assumption you can find examples where you get better results. I'd rather have no data in OSM than these guesses... For saddles we take an already mapped (better, I hope) value into account, but not for the isolation.
lukey78
commented
4 years ago I don't think... I tested it with a 50m grid (Bavaria) and a 10m grid (Austria) and both worked without adaptation.
You're right, that was a misinterpretation by me. The script works as expected also with the 30m DEM. Sorry.
atorger
commented
3 years ago It would be of great if someone could gather all the various public elevation data sources there is and merge them into "current best" global data set and make it available for all map makers to use. Everyone would gain on that. Now each contour map maker needs to solve this problem on their own, and it's quite some work, and as new sources get published now and then and many cover only one country it's hard to keep track of the current best and merge all together.
I have noticed that Opencyclemap has much better elevation data for Sweden than Opentopomap has, I don't know what sources they are using but I'm assuming it's public. I've heard that there is a Japanese public global dataset that should be quite good.
Unfortunately the current elevation data used in Opentopomap data for Sweden is so bad that it doesn't line up with mapped peaks and there's many issues with false peaks and other artifacts, so the contours is currently mostly useful for an illustrative purpose rather than a real map for navigation in the mountains.
The government in Sweden has released public elevation data for the country. They have extremely precise elevation data, but the one released is only 50m resolution, I would still expect it to better than anything else currently in use in OSM-based maps as it has no issues with noise or alignment. I'd love to submit it and see it being used, but there's no place to submit it to.
There was a open-elevation.com project that could have solved this issue but that projects seems to have stalled. It's hard to keep these things running on a hobbyist level unless you have a well-organized community backing so when one maintainer steps down another can take over, and there is some financial backing for server hosting. Elevation data could be served as a static database only to map makers though so it wouldn't need to be a 24/7 availability.
Anyway, I think an open elevation database would be a great sub-project to OSM and that context would make it easier to get continuity in the project, but the leadership in OSM has decided to not deal with this stuff unfortunately.
If Opentopomap team could host such a database and use it in their own map, that would be great, but it seems to me that this is a quite small hobbyist project as well so it's out of the scope I guess.
lukey78
commented
3 years ago @atorger That is a really bautiful idea, I think that is neccssary, too. In a world of open data, why isn't there a globally available elevation model with the "best" data?
Maybe we could start with a repo where links to the best free elevation data in the world could be documented. Where local sources are available, we should use them, if not, we should use the more "global" one.
That'd a first step. Next step would be to generate a reliable workflow to generate a DEM from all that data: Hillshade TIFs in various resolutions, and a countours database. I've noticed that many processing steps fail due to the large amount of data. What works perfectly for small countries, fails with large data.
That brings me to the "problem": The processing is very resource intensive. I used a server with 256 GB RAM, and I needed about 3 TB of disk space to store all the data (original data, temporary data, processed data). Also, it took a week or so on this 8 core server to finish. Finally, the hillshades are ~300 GB, and the contours database is about ~1 TB. Ah, and without SSD storage the database stuff is too slow.
Updating something like that is a trip to hell.
Buuut... if there would be a docker image that is able to do all of that automatically and produce the final files, then one could rent a large machine, run the image, and transfer the final files to a "cheap" data space somewhere on the net. Cream of the pie would be a server hosting the contours database.
Well... what about starting with step 1? ;-)
atorger
commented
3 years ago Interesting to hear about the tough requirements on platform, not surprising, but I had never done the numbers myself.
I'm unfortunately not in a position to help with coding right now, I'm currently working with road imports in Sweden, and have some other projects scheduled after that...
(To make it "worse" I think there's an additional thing one would want to do with elevation data, now when consumer GPS with barometer sensor provides quite good elevation it would be nice to crowd source GPS tracks and get separate/complementary elevation data on roads. Generic elevation data can't do bridges or tunnels, and are generally problematic due to lack of resolution when roads goes near steep terrain. As a cyclist I see this quite often. I think Garmin and Strava and perhaps some other commercial companies already do this, the more people travel a road the more accurate the roads elevation becomes as more elevation data is gathered and averaged. Elevation on roads is very popular in applications for cyclists.
Here in Sweden we also have elevation data for the roads in the public national database, so it could actually be imported.)
lukey78
commented
3 years ago The first step:
Check this very famous and popular place at Obersee (right next to Königssee) in Berchtesgadener Land: https://opentopomap.org/#map=16/47.51641/12.98895
Contour lines seem to be completely messed up resp. incorrectly calculated. I really wonder what could have caused this. It’s strange because the lines itself are not broken, faulty or “glitchy”, they just show something which definitely doesn’t exist.