mapbox / tippecanoe

Build vector tilesets from large collections of GeoJSON features.
BSD 2-Clause "Simplified" License
2.73k stars 432 forks source link

TileJSON file too big #941

Open arunasank opened 2 years ago

arunasank commented 2 years ago

I have a large number of unique properties, each associated with a different feature, which is causing the tileJSON file to bloat, and causing my mapbox upload to fail. Is there a workaround that allows me to have these properties, but still keeps the tilejson size low without having to split the files?

If this is not possible, tippecanoe should issue a warning when the tileJSON size is too large.

e-n-f commented 2 years ago

Agreed. I no longer work at Mapbox so I can't just make this change to the repository here, but you can use --no-tile-stats to keep from generating the tilestats that are bloating the TileJSON, or --tile-stats-attributes-limit=20, say, to generate tilestats about only 20 attributes.

arunasank commented 2 years ago

Thank you so much for continuing to respond here to queries! This doesn't seem to work, unfortunately. I continue to have a bloated TileJSON in spite of trying both --no-tile-stats and --tile-stats-attributes-limit=20. When I run tippecanoe-decode on the file, properties.json.vector_layers.fields has ~109k keys (since I have 109k features, each with a unique property), which is the cause of the problem I think. Is there some way to reduce the size of this obj?

Further, is this obj even necessary in the mbtiles file? If not, is there a way to delete it?

e-n-f commented 2 years ago

There isn't an existing option to do that, since I was always under the impression that it was a requirement for vector uploads to have a complete vector_layers. The loop that prepares it is here:

https://github.com/mapbox/tippecanoe/blob/master/mbtiles.cpp#L435

so if you are comfortable editing the code, you could add a count like this:

size_t count = 0;
for (auto j = fk->second.file_keys.begin(); j != fk->second.file_keys.end(); ++j) {
    count++;
    if (count > 100) {
        break;
    }

to stop after the first 100 attributes.