panic in fasthash/jody when race detector is enabled

[jnjackins]

I consistently get a panic in AddString64, but only when the race detector is enabled.

fatal error: checkptr: pointer arithmetic result points to invalid allocation

 goroutine 129 [running]:
 runtime.throw(0x11204c7, 0x40)
    /usr/local/go/src/runtime/panic.go:1116 +0x72 fp=0xc00046b340 sp=0xc00046b310 pc=0x46a4d2
 runtime.checkptrArithmetic(0xc000402df0, 0xc00046b3f0, 0x1, 0x1)
    /usr/local/go/src/runtime/checkptr.go:43 +0xbe fp=0xc00046b370 sp=0xc00046b340 pc=0x4397be
 github.com/segmentio/fasthash/jody.AddString64(0x1c26a908921dd5e0, 0xc000402dec, 0x4, 0x1c26a908921dd5e0)
    /build/vendor/github.com/segmentio/fasthash/jody/hash.go:111 +0x3f0 fp=0xc00046b410 sp=0xc00046b370 pc=0xf42850

[achille-roussel]

@jnjackins can you confirm which version of the package you're depending on?

[jnjackins]

Sorry - that's v1.0.3

[yerden]

It looks ok to me:

$ git rev-parse HEAD
9dc1b83dc0d8bf7d26ea69d6c6d7a11ebf6fbb89
$ (cd jody; go test -race)
PASS
ok      github.com/segmentio/fasthash/jody      0.073s

@jnjackins can you share your input data so the problem can be reproduced?

[nanmu42]

Hi, I just met the same problem. I am using github.com/segmentio/fasthash v1.0.3 and it would take several tries to produce this error when race detector is on:

fatal error: checkptr: pointer arithmetic result points to invalid allocation

goroutine 92 [running]:
runtime.throw(0x1da9a0c, 0x40)
        /usr/local/go/src/runtime/panic.go:1117 +0x72 fp=0xc000ca6158 sp=0xc000ca6128 pc=0x46aa52
runtime.checkptrArithmetic(0xc000044720, 0xc000ca6208, 0x1, 0x1)
        /usr/local/go/src/runtime/checkptr.go:43 +0xbe fp=0xc000ca6188 sp=0xc000ca6158 pc=0x43a27e
github.com/segmentio/fasthash/jody.AddString64(0x197d45f2c9aec3c1, 0xc00004471c, 0x4, 0x197d45f2c9aec3c1)
        /home/nanmu/go/pkg/mod/github.com/segmentio/fasthash@v1.0.3/jody/hash.go:111 +0x3e5 fp=0xc000ca6228 sp=0xc000ca6188 pc=0xd73bc5
github.com/segmentio/stats/v4/prometheus.labels.hash(0xc00001fd00, 0x4, 0x8, 0x8)
        /home/nanmu/go/pkg/mod/github.com/segmentio/stats/v4@v4.6.0/prometheus/label.go:52 +0xd6 fp=0xc000ca6270 sp=0xc000ca6228 pc=0xd77dd6
github.com/segmentio/stats/v4/prometheus.(*metricEntry).lookup(0xc000244f30, 0xc00001fd00, 0x4, 0x8, 0x1d7cb89)
        /home/nanmu/go/pkg/mod/github.com/segmentio/stats/v4@v4.6.0/prometheus/metric.go:166 +0x70 fp=0xc000ca6320 sp=0xc000ca6270 pc=0xd795f0
github.com/segmentio/stats/v4/prometheus.(*metricStore).update(0xc00018ebb8, 0x1, 0xc0005a4240, 0x14, 0x1d7cb89, 0x5, 0x0, 0x0, 0x3ff0000000000000, 0xc02e7c75174f163a, ...)
        /home/nanmu/go/pkg/mod/github.com/segmentio/stats/v4@v4.6.0/prometheus/metric.go:100 +0x108 fp=0xc000ca63e8 sp=0xc000ca6320 pc=0xd79088
github.com/segmentio/stats/v4/prometheus.(*Handler).HandleMeasures(0xc00018eb90, 0xc02e7c75174f163a, 0x7d2c2ab8a, 0x316d280, 0xc0004c2d80, 0x1, 0x1)
        /home/nanmu/go/pkg/mod/github.com/segmentio/stats/v4@v4.6.0/prometheus/handler.go:76 +0x2e5 fp=0xc000ca66a8 sp=0xc000ca63e8 pc=0xd76245
github.com/segmentio/stats/v4.(*Engine).measure(0xc0000bca00, 0xc02e7c75174f163a, 0x7d2c2ab8a, 0x316d280, 0x1d7cb80, 0x8, 0x1b590e0, 0x1dc0d70, 0xc000000000, 0xc000ca7450, ...)
        /home/nanmu/go/pkg/mod/github.com/segmentio/stats/v4@v4.6.0/engine.go:155 +0x599 fp=0xc000ca67b0 sp=0xc000ca66a8 pc=0xd5a4b9
github.com/segmentio/stats/v4.(*Engine).Add(0xc0000bca00, 0x1d7cb80, 0xe, 0x1b590e0, 0x1dc0d70, 0xc000ca7450, 0x4, 0x4)
        /home/nanmu/go/pkg/mod/github.com/segmentio/stats/v4@v4.6.0/engine.go:94 +0xbf fp=0xc000ca6820 sp=0xc000ca67b0 pc=0xd5957f
github.com/segmentio/stats/v4.(*Engine).Incr(...)
        /home/nanmu/go/pkg/mod/github.com/segmentio/stats/v4@v4.6.0/engine.go:84
[omitted]

I tried to isolate the enviroment that caused the error, but got no luck.

I suspect, the problem may need some criterions meet:

• race detector is on;
• jody.AddString64() is called severial times;
• jody.AddString64() is called somewhere deep in the call frame.

Just making assumptions.

[somnusfish]

@achille-roussel It would be great if you could review the PR and let me know if you have any feedback.