Catch more invalid files with verify command

[dabreegster]

I'm experimenting with generating pmtiles (with MVT) from Rust. https://github.com/dabreegster/od2net/blob/tippecanwho/tippe/src/main.rs is my initial attempt. It doesn't produce usable output yet, but I'm having trouble figuring out where the problem is.

https://www.dropbox.com/scl/fi/25bikp6xvb3l6we182zt9/bad_output.pmtiles?rlkey=wn19lcqeramyxfv93jtmwdjt5&dl=0 is a bad output I'm producing right now. pmtiles verify shows no errors and seems happy. Loading the file with https://protomaps.github.io/PMTiles doesn't show anything on the map, and the console has an error:

e24e0201-5cdb-4bcb-8107-86ee3d56b440:2 Uncaught Error: unknown command 4
    at wc.loadGeometry (e24e0201-5cdb-4bcb-8107-86ee3d56b440:2:90342)
    at sa (e24e0201-5cdb-4bcb-8107-86ee3d56b440:2:64684)
    at f0.populate (e24e0201-5cdb-4bcb-8107-86ee3d56b440:2:100722)
    at C.parse (e24e0201-5cdb-4bcb-8107-86ee3d56b440:2:217921)
    at e24e0201-5cdb-4bcb-8107-86ee3d56b440:2:220887
    at e24e0201-5cdb-4bcb-8107-86ee3d56b440:2:220047
    at u.Actor.processTask (e24e0201-5cdb-4bcb-8107-86ee3d56b440:2:194145)
    at D5.process [as _callback] (e24e0201-5cdb-4bcb-8107-86ee3d56b440:2:193365)
    at D5.MessageChannel._channel.port2.onmessage (e24e0201-5cdb-4bcb-8107-86ee3d56b440:2:159810)

The results of pmtiles show:

pmtiles spec version: 3
tile type: Vector Protobuf (MVT)
bounds: 0.000000,0.000000 0.000000,0.000000
min zoom: 0
max zoom: 0
center: 0.000000,0.000000
center zoom: 0
addressed tiles count: 1
tile entries count: 1
tile contents count: 1
clustered: true
internal compression: 2
tile compression: 1
vector_layers <object...>

I'm pretty certain I'm failing to populate some of the metadata correctly, or doing something very wrong with encoding geometry. My ask here is if it makes sense for go-pmtiles verify to check for possible problems more.

[bdon]

Can you try generating the file with a minimal PNG image, hardcoded bytes, and inspecting in the viewer?

The verify command is meant for verifying the container organization, and not the tile contents, so it is difficult to distinguish between the read failure being the container or the individual contents...

[dabreegster]

The container bit was just fine; the bug was in not getting coordinate transforms right at all in the MVT part. Stuart got a minimal working example in https://github.com/Urban-Analytics-Technology-Platform/od2net/pull/40.

Maybe what could be useful is something to verify MVT contents or better error messages, but I think the Uncaught Error: unknown command 4 is auto-generated protobuffer code anyway.

Thanks!

[bdon]

Here's a related issue of improving error messages in maplibre for malformed PBFs:

https://github.com/maplibre/maplibre-gl-js/issues/2809

My thinking right now is that verifying individual MVTs takes this project too far - a verify on a planet basemap would need to spend a lot more time parsing 100+ million tile contents, unless we make it optional (adding more complexity...)

[dabreegster]

For the record, figured this out: https://github.com/Urban-Analytics-Technology-Platform/od2net/commit/f7bfb02a9df18489537f71873633c77f74e25f91 There's a bug in https://crates.io/crates/mvt where asking to encode a LineString with 0 points doesn't fail as it should, but instead writes something invalid. Will be fixing upstream.

And I hope to have a working demo soon of a tippecanoe alternative specialized for a particular LineString case!