SIGSEGV in `pmtiles serve`

[jamesscottbrown]

I encountered a segfault when running pmtiles serve (version 1.7.0):

$  pmtiles serve --cors="*" .

2024/02/28 16:30:10 main.go:115: served /uprn/16/32744/21777.mvt in 12.216941ms
2024/02/28 16:30:10 main.go:115: served /uprn/16/32746/21777.mvt in 5.569058ms
2024/02/28 16:30:10 main.go:115: served /uprn/16/32745/21776.mvt in 18.636425ms
2024/02/28 16:30:10 main.go:115: served /uprn/16/32746/21776.mvt in 27.242473ms

2024/02/28 16:30:36 server.go:139: fetching uprn 41561-8008
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x280 pc=0x5b762e]

goroutine 1839 [running]:
compress/gzip.(*Reader).Read(0x0?, {0xc0003a3000?, 0x7fc5d9ad8a18?, 0x0?})
        /opt/local/lib/go/src/compress/gzip/gunzip.go:247 +0x2e
bufio.(*Reader).fill(0xc0000287e0)
        /opt/local/lib/go/src/bufio/bufio.go:106 +0xff
bufio.(*Reader).ReadByte(0xc0000287e0)
        /opt/local/lib/go/src/bufio/bufio.go:265 +0x2c
encoding/binary.ReadUvarint({0x1971320, 0xc0000287e0})
        /opt/local/lib/go/src/encoding/binary/varint.go:133 +0x82
github.com/protomaps/go-pmtiles/pmtiles.deserialize_entries(0x1973a20?)
        /Users/bdon/workspace/protomaps/go-pmtiles/pmtiles/directory.go:141 +0x18c
github.com/protomaps/go-pmtiles/pmtiles.(*Server).Start.func1.1()
        /Users/bdon/workspace/protomaps/go-pmtiles/pmtiles/server.go:175 +0x9ab
created by github.com/protomaps/go-pmtiles/pmtiles.(*Server).Start.func1
        /Users/bdon/workspace/protomaps/go-pmtiles/pmtiles/server.go:127 +0x8ac

Unfortunately the uprn.pmtiles file is large and contains information that I can't re-distribute (or, at least, that I can't re-distribute in its entirety).

I'm running pmtiles inside docker inside WSL: I'm not sure whether this is relevant.

The crash occured before the URL of the requested tile was logged, and it's not obvious to me how to reconstruct the x/y tile index from the offset and size (41561-8008), so I can't easily test if the crash is reproducible in v1.18.0.

[jamesscottbrown]

Looking at the console more carefully, I think that the uprn.pmtiles file was re-written by tippecanoe without the pmtiles serve process being restarted.

pmtiles could check to see if a file has been updated and handle this gracefully, but just telling users not to edit files would also be reasonable.

[bdon]

As of v1.18 the http, s3 and file backends handle Etags gracefully. Versions prior to v1.16, and azure/gap buckets can read garbage offsets if the file changes from underneath.

[jamesscottbrown]

Great - in that case, I don't think the crash I encountered is a bug in the current version of pmtiles.