Multipoint support in Shapefile

[huming2207]

Hi @systemed

I've got an Australian G-NAF data converted in to Shapefile and I'm planning to use tilemaker to merge the G-NAF into OpenStreetMap.

But it looks like the Shapefile type 8 Multipoint isn't supported, as it keeps complaining Shapefile entity #xyz type 8 not supported, printed from here: https://github.com/systemed/tilemaker/blob/3ff08a3bc34f44a5e4c160c1a0b25ea647bd5bb1/src/shp_processor.cpp#L260

I will have a try and see if I can add the support for Multipoint tonight. If I make any progress I will post a PR.

Regards, Jackson

[huming2207]

By the way, I converted the G-NAF data to GeoJSON (and then later to Shapefile) by this: https://github.com/huming2207/admin-boundaries-mbtiles

You may download it and have a try if you would like to.

[huming2207]

Hmm I'm a bit confused, looks like the Multipoint here is probably the same as Point?

I don't know now, maybe I misused ogr2ogr when I converted my G-NAF GeoJSON file to Shapefile...

[systemed]

Interesting, I've never seen one of those in the wild :)

Does #640 fix it for you?

[huming2207]

Does https://github.com/systemed/tilemaker/pull/640 fix it for you?

Yea I guess should work and that actually was my plan, I've also implemented the same thing on my local machine.😅

But meanwhile the my shapefile only has multipoints with nVerticies is always 1, so I don't know if that really works for others. Right now I've done something like that on my local codebase and the map is still building. I will test it out and let you know if that works with my file first anyway.

[systemed]

Yes, the only shapefile I could find with multipoints also had nVertices==1!

[huming2207]

Yea I guess this is probably either for some very uncommon cases, or somehow ogr2ogr by default converted Shapefiles put (single) points to multipoints with nVerticies==1.

[huming2207]

Hi @systemed looks like somehow it didn't work for me, but maybe my config was wrong again?

Here's the behaviour on QGIS (as you can see a lot of dark green dots, which is the G-NAF house number points:

Here's the mbtiles generated by tilemaker, as you see there's no housenumber layer somehow:

Here's my Lua script: scripts-and-config.zip

It looks like if I remove the "combile_polygons_below": 13 in the housenumber config, tilemaker doesn't even use the housenumber shapefile at all. I will have another look tonight.

Regards, Jackson

[huming2207]

Here's what it will be like if I zoom in:

...as you see there's still no housenumber field shown.

[systemed]

For debugging vector tile output I usually use vt2geojson. You tell it to fetch a vector tile on the command line:

vt2geojson http://localhost:8080/13/4066/2712.pbf > vt.geojson

You can then open the geojson in a text editor and see if the features have been written. This way you know if the issue is with the vector tile itself or with the rendering.

If you do that, are the points in there? If not, could you put your shapefile somewhere so I can take a look at it?

[huming2207]

Thanks for the info. Indeed there's no housenumber:

Here's a pbf and the converted GeoJSON at zoom level 14: pbf-and-geojson.zip

[huming2207]

could you put your shapefile somewhere so I can take a look at it?

The shapefile is quite huge, I couldn't upload it easily but I will try. Meanwhile you may be able to generate your own. Basically I downloaded the GeoJSONs from here: https://github.com/huming2207/admin-boundaries-mbtiles/releases/tag/GNAF-2023-11 and then merged & converted to shapefile by using command ogr2ogr -f 'ESRI Shapefile' merge.shp -update -append ./housenumber/housenumber-***.geojson -nln merge

[huming2207]

I managed to put the shapefile on OneDrive: https://1drv.ms/u/s!ArAGkYutIt8Dg-NsJZc3BViavjjtjg?e=aPDZnr

Could you please have a look when you are free? Thanks! @systemed

[systemed]

I've had a look at the shapefile using #640 and the data successfully gets into the vector tile.

What I did:

• Added a new layer to the JSON config:

"hn_shapefile": { "minzoom": 14, "maxzoom": 14, "source": "/home/richard/tm_debug/housenumber_wgs84/housenumber_wgs84.shp", "source_columns": true },

• Added support in the Lua process (attribute_function):

function attribute_function(attr,layer)
    if layer=="hn_shapefile" then
        return { num=attr["housenumbe"] }
    elseif attr["featurecla"]=="Glaciated areas" then
        -- continued...

• Ran tilemaker (compiled with #640):

tilemaker /media/data1/planet/australia-latest.osm.pbf ~/tm_debug/australia_hn.mbtiles --process resources/process-omt-multipoint.lua --config resources/config-omt-multipoint.json

• Started the server:

tilemaker-server ~/tm_debug/australia_hn.mbtiles --static server/static/

• Fetched a sample vector tile:

vt2geojson http://peyresourde.local:8080/14/15058/9833.pbf

• And the result contained the house numbers:

    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          150.88413834571838,
          -33.90376116404979
        ]
      },
      "properties": {
        "num": "105A",
        "vt_layer": "hn_shapefile"
      }
    },
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          150.8750832080841,
          -33.90323578672987
        ]
      },
      "properties": {
        "num": "105A",
        "vt_layer": "hn_shapefile"
      }
    },
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          150.8828240633011,
          -33.89127589860132
        ]
      },
      "properties": {
        "num": "7C",
        "vt_layer": "hn_shapefile"
      }
    },

Looking at your config, it might be that you haven't told tilemaker which attributes to write in attribute_function. tilemaker will call attribute_function for each shapefile object, and expect you to return the vector tile attributes you want to be set for that object. It doesn't look like you're doing that.

[huming2207]

it might be that you haven't told tilemaker which attributes to write in attribute_function. tilemaker will call attribute_function for each shapefile object

Oh yes... I forgot to change the Lua script! Sorry!

Now I'm trying again, it will take a few hours to build. I will let you know if that works.

[huming2207]

Hi @systemed sorry for the late reply. I think it's working now, all the housenumber are available.