felt / tippecanoe

Build vector tilesets from large collections of GeoJSON features.
BSD 2-Clause "Simplified" License
875 stars 76 forks source link

Invalid metadata json Row #222

Open daniele-olivieri-rs opened 4 months ago

daniele-olivieri-rs commented 4 months ago

First, thank you for all y'all's work on tippecanoe -- it is a great utility and is very well documented.

I've been having an issue with tippecanoe producing invalid json row on the metadata table, which causes my tile server to be unable to load and serve the .mbtiles.

The issue is that when the max for a given field in the properties bag is being written, tippecanoe writes the following: "max":inf

Which is not valid json -- the inf should be surrounded by quotes "inf" to be valid. So something like "max":"inf". In the metadata file, the field is marked as "mixed" -- so I imagine a arbitrary string should be fine. At the very least the server should be able to load the file so long as the json is valid.

Input data info:

  1. The input file is newline delimited geojson.
  2. The input files is very large (100 million+ records).
  3. The field is a floating point number which is truncated to two decimal places before being written to the geojson file i.e. 12.34.
  4. The field has the accumulate-attribute:mean set for it.
  5. The issue only occurs when running in a container on a Ubuntu virtual machine (when I run the same container on my local box, which is Windows running Ubuntu on the Linux subsystem it does not occur).

For now I can attempt to triage by opening the .mbtiles file, detecting the error, and correcting the json but if you could fix the issue, it would be much appreciated.

daniele-olivieri-rs commented 1 month ago

Following up on this with new discovery.

When using a post-filter, with the same process/data, the outputted json can have inf as the value of a field.

For example: { "properties": { "myValue": inf }

Which is not valid json. When tippecanoe tries to read this from the post-filter, it errors out. Presumably because tippecanoe cannot parse the value back to json.

This inf issue seems to bubble up to the metadata mentioned in the initial issue if there is no post-filter.