add support for a GDAL compatible (vector?) layer format

[khimaros]

following up on https://github.com/osmandapp/OsmAnd/issues/3388 as a separate issue to ensure broader visibility. N.B. i am open to writing the code to make this work, but want to validate the idea with the OSMAnd team before investing a large chunk of time and effort.

tl;dr, it would be great to support a format which GDAL or QGIS can work with natively. doing so would open up a wide variety of public data to be automatically converted for use in OSMAnd, using only free software tools.

as mentioned in the linked issue, GPKG comes in both raster and vector varieties. while i suspect it would be a larger development effort to support these in OSMAnd, vector formats can be quickly created from the GeoDB files offered by various government agencies on ArcGIS.

today a convoluted and inefficient method, such as the one i employed here, is required. this method produces very low resolution data at great computational (~20hrs running continuously on my laptop) and human expense. specifically: producing an OSMAnd offline layer from a 32Kpx x 32Kpx raster image from the original ~500MB GeoDB file results in an ~8.5GB database. this map starts to show aliasing around zoom level 3. to produce a reasonable looking image at zoom level 14 would require a ~900kpx x 900kpx raster. whereas the vector GPKG clocks in at ~2.5GB and looks perfect at all zoom levels.

[EugeneZmeuk]

Hi @khimaros We have a big task for the next year (1-2 Quarters) for adding US map data for OsmAnd. I mean PADUS, USGS topo, BLM etc. While we are collecting info about map resources and legalisation for the OsmAnd app. I think one part of map data will need supporting additional formats (not only OBF), or quick convertation. In any case, when we read feedback and question our users (https://www.reddit.com/r/OsmAnd/comments/zrd4wm/finally_loaded_blm_map/) we understand needs additional map data for this region.

[vshcherb]

@khimaros if you can explain please which vector format do you mean - GPKG -- GeoPackage vector (https://gdal.org/drivers/vector/gpkg.html#gpkg-geopackage-vector)?

We already support GDAL rasterizer for Geotiff hillshade, so I can ask in which direction at least we need to go with

[khimaros]

@EugeneZmeuk @vshcherb -- it's great to hear that your team is already thinking about this. thank you for your interest in the subject and replies here, and in general for creating such an incredible power tool for geo data in OSMAnd. i use it many times each day.

@EugeneZmeuk -- would it be helpful for me to link you to some resources for this public data? here are some interesting starting points: [0], [1]. the vast majority of these are published as GeoDatabase vector layers.

i'd be happy to help provide more indices, or to provide specific map examples if that would help. to me, the single most interesting piece of data while on the road in the USA is the Surface Management Agency data for finding public land.

one thing to consider is the use of these formats as a base layer vs. as an overlay/underlay. do you think it is practical for one format to support both of those cases in OSMAnd?

@vscherb -- yes, that is the one. it should be the same as the link in my issue description with "vector" anchor title. i would say that GPKG seems like a reasonable choice. it can also support multiple layers in a single file, which is helpful for some of the government data listed above.

[khimaros]

@vshcherb -- i will mention that any formats which require a rasterization step (including GTiff) have high resource demands to achieve adequate resolutions for use on mobile. this is the problem i've been running into with my GeoDatabase to OSMAnd pipeline.

i've been working on an improvement to my pipeline to rasterize smaller blocks iteratively to reduce RAM requirements. it has been a headache to align map extents to raster tiles such that they don't overlap at any zoom level. this may require different alignment at each zoom level which would be very cumbersome and require more computational time.

[EugeneZmeuk]

@khimaros our team made a test version of PAD-US data for viewing on South Dakota. You can unzip this file and add files to OsmAnd

Screenshot

Archive 2.zip

In any case, I think it will be easier if you write to my email eugene.kizevich@osmand.net Because I have more specific questions about maps US data

[vshcherb]

Do we have examples of GPKG raster and vector that we want to support. Especially vector to understand whether it comes with map style inside

[khimaros]

@vshcherb this is the vector map i'm most interested in: https://www.arcgis.com/home/item.html?id=6bf2e737c59d4111be92420ee5ab0b46

note, i've also been following @EugeneZmeuk posts in https://www.reddit.com/r/OsmAnd/comments/19erp5z/padus_usgs_topo_usgs_national_map_arcgis_world/

[EugeneZmeuk]

@khimaros we added for OsmAnd-android-full-arm64.apk nightly version of our Custom plugins (US Maps Plugin too). I publish more actual versions of custom plugins in this way.

BLM layer you can find on the PAD-US layer. I think it does not make sense to add BLM as a separate source (it's my opinion, but maybe for American users it makes sense)

[khimaros]

sorry for the delayed reply.

@EugeneZmeuk to clarify, while that vector map i linked above is maintained by the BLM, it actually inventories all "Surface Management Agencies", not just BLM managed land.

for example, it includes forestry service (USFS, NFS), national parks (NPS), state parks, military (DOD), indigenous land, private land, etc.

[EugeneZmeuk]

sorry for the delayed reply.

@EugeneZmeuk to clarify, while that vector map i linked above is maintained by the BLM, it actually inventories all "Surface Management Agencies", not just BLM managed land.

for example, it includes forestry service (USFS, NFS), national parks (NPS), state parks, military (DOD), indigenous land, private land, etc.

@khimaros, thanks for your clarifying