[tile-join with PMTiles files] Possible race condition? Different errors every time, sometimes completes successfully

[iam-spartacus]

I'm having issues joining PMTiles files with the 'tile-join' program. I get different errors every time, and sometimes the program will complete successfully. I've tested on Debian 12 and Ubuntu 24.04, and both OSes give me a wide variety of errors.

Here's some example code that I've been using for my tests.

First, using the 'go-pmtiles' program, I extract a couple of PMTiles files (roughly 200-250MB each) from the Protomaps website:

#!/bin/bash

# using go-pmtiles to download a couple of small PMTiles sections
# from https://maps.protomaps.com/builds/
./pmtiles extract https://build.protomaps.com/20240617.pmtiles debug_quebec.pmtiles --bbox=-74.134369,45.264255,-70.955200,47.007416
./pmtiles extract https://build.protomaps.com/20240617.pmtiles debug_maritimes.pmtiles --bbox=-67.478027,43.181147,-61.545410,47.100045

After that, I run the 'tile-join' program. I'm using a script to run batches of 10 trials, because I get different results every time I run it. Additionally, some trials will completely freeze my system, and I have to restart my computer.

#!/bin/bash

for i in {1..10}
do
    echo Trial $i
    tile-join --force -pk -o merged_$i.pmtiles debug_*.pmtiles
done

My results from a sample run of 10 trials (Note: Trial 7 was successful).

Trial 1: Fatal glibc error: malloc.c:4192 (_int_malloc): assertion failed: chunk_main_arena (fwd)
Trial 2: Internal error: corrupted geometry
Trial 3: Decompression error: data error
Trial 4: free(): invalid size 6785 Aborted (core dumped)
Trial 5: 7076 Segmentation fault (core dumped)
Trial 6: Decompression error: data error

Trial 7: SUCCESS!!!

Trial 8: 12027 Segmentation fault (core dumped)
Trial 9: Internal error: corrupted geometry
Trial 10: Internal error: corrupted geometry

As you can see, every once in a while, 'tile-join' will complete successfully.

Given the wide variety of outcomes, I'm wondering if there is a race condition somewhere...

[e-n-f]

Thanks for the report! I'll take a look and see if I can reproduce the problem.