data race?

[didip]

Fairly typical use-case, a bunch of goroutines are performing writes and a few goroutines are performing reads.

==================
WARNING: DATA RACE
Write at 0x00c00014d8c0 by goroutine 109:
  runtime.mapassign_fast64()
      /usr/local/go/src/runtime/map_fast64.go:92 +0x0
  myapp/vendor/github.com/cznic/lldb.(*bitFiler).ReadAt()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/lldb/xact.go:161 +0x4b8
  myapp/vendor/github.com/cznic/lldb.(*RollbackFiler).ReadAt()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/lldb/xact.go:538 +0x1a4
  myapp/vendor/github.com/cznic/lldb.(*ACIDFiler0).ReadAt()
      <autogenerated>:1 +0x92
  myapp/vendor/github.com/cznic/lldb.(*Allocator).read()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/lldb/falloc.go:973 +0x85
  myapp/vendor/github.com/cznic/lldb.(*Allocator).Get()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/lldb/falloc.go:697 +0x38b
  myapp/vendor/github.com/cznic/ql.(*file).Read()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/file.go:812 +0x84
  myapp/vendor/github.com/cznic/ql.(*table).row0()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/storage.go:777 +0x101
  myapp/vendor/github.com/cznic/ql.(*tableDefaultPlan).do()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/plan.go:2401 +0x2a8
  myapp/vendor/github.com/cznic/ql.(*DB).do()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/ql.go:1479 +0x1bf
  myapp/vendor/github.com/cznic/ql.recordset.Do()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/ql.go:75 +0x95
  myapp/vendor/github.com/cznic/ql.(*recordset).Do()
      <autogenerated>:1 +0x9f
  myapp/vendor/github.com/cznic/ql.newdriverRows.func1()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/driver.go:415 +0xfc

Previous write at 0x00c00014d8c0 by goroutine 754:
  runtime.mapassign_fast64()
      /usr/local/go/src/runtime/map_fast64.go:92 +0x0
  myapp/vendor/github.com/cznic/lldb.(*bitFiler).ReadAt()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/lldb/xact.go:161 +0x4b8
  myapp/vendor/github.com/cznic/lldb.(*RollbackFiler).ReadAt()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/lldb/xact.go:538 +0x1a4
  myapp/vendor/github.com/cznic/lldb.(*ACIDFiler0).ReadAt()
      <autogenerated>:1 +0x92
  myapp/vendor/github.com/cznic/lldb.(*Allocator).read()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/lldb/falloc.go:973 +0x85
  myapp/vendor/github.com/cznic/lldb.(*Allocator).Get()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/lldb/falloc.go:697 +0x38b
  myapp/vendor/github.com/cznic/ql.(*file).Read()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/file.go:812 +0x84
  myapp/vendor/github.com/cznic/ql.(*table).row0()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/storage.go:777 +0x101
  myapp/vendor/github.com/cznic/ql.(*tableDefaultPlan).do()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/plan.go:2401 +0x2a8
  myapp/vendor/github.com/cznic/ql.(*DB).do()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/ql.go:1479 +0x1bf
  myapp/vendor/github.com/cznic/ql.recordset.Do()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/ql.go:75 +0x95
  myapp/vendor/github.com/cznic/ql.(*recordset).Do()
      <autogenerated>:1 +0x9f
  myapp/vendor/github.com/cznic/ql.newdriverRows.func1()
      /Users/didip/go/src/myapp/vendor/github.com/cznic/ql/driver.go:415 +0xfc

[cznic]

Thanks for the report. Can you please create a small reproducer? Best would be a PR that adds it as a test that fails when go test -race.

[cznic]

@didip Gentle ping.

[didip]

I’ve been trying to create a small example to reproduce this, but I am unable to :(

I haven’t seen this race problem in production again as well.

You can close this issue if you want.

[cznic]

I'll try to follow the stack trace. Do you know the exact release/commit that the trace was produced by?

[didip]

That, I can do. This is the revision that we vendored:

{
    "checksumSHA1": "NHqE67y+/xEkTTqnXt2HvwubP38=",
    "path": "github.com/cznic/b",
    "revision": "35e9bbe41f07452a183c517a5fc5f3c9f45eaa0f",
    "revisionTime": "2018-01-15T12:50:44Z"
},
{
    "checksumSHA1": "Lp/wfyPm+F2PnU2o0k3ChwKSDY0=",
    "path": "github.com/cznic/fileutil",
    "revision": "6a051e75936f623600b67c2b1116b6b6c0ffb936",
    "revisionTime": "2018-01-08T21:13:00Z"
},
{
    "checksumSHA1": "nWWlWVVxkxwjnH69xT6N1afFqqc=",
    "path": "github.com/cznic/golex/lex",
    "revision": "4ab7c5e190e49208c823ce8ec803aa39e6a4b31a",
    "revisionTime": "2017-08-03T12:31:10Z"
},
{
    "checksumSHA1": "/839m5piLe3TonKs6VGzZ668Rw0=",
    "path": "github.com/cznic/internal/buffer",
    "revision": "f44710a21d00af8b5c8d1b3ff5f141fe5cb9b9eb",
    "revisionTime": "2018-06-08T15:22:20Z"
},
{
    "checksumSHA1": "Equk0TlMj+qUyBfievlNzCs60qk=",
    "path": "github.com/cznic/internal/file",
    "revision": "f44710a21d00af8b5c8d1b3ff5f141fe5cb9b9eb",
    "revisionTime": "2018-06-08T15:22:20Z"
},
{
    "checksumSHA1": "sIFhykv6o1TIyaFHWp8uDio8sTc=",
    "path": "github.com/cznic/internal/slice",
    "revision": "f44710a21d00af8b5c8d1b3ff5f141fe5cb9b9eb",
    "revisionTime": "2018-06-08T15:22:20Z"
},
{
    "checksumSHA1": "Uiy2xCrAwApimIZHuUoSoN44fx8=",
    "path": "github.com/cznic/lldb",
    "revision": "bea8611dd5c407f3c5eab9f9c68e887a27dc6f0e",
    "revisionTime": "2016-11-02T12:47:59Z"
},
{
    "checksumSHA1": "AuwJMJvQAlHVVVgk5/ZrsLvSZrI=",
    "path": "github.com/cznic/mathutil",
    "revision": "ca4c9f2c136954238c3158b92de72078c7672ecc",
    "revisionTime": "2018-05-04T12:22:25Z"
},
{
    "checksumSHA1": "rMEAVarGnfPIczmqyr2vhu2C9m8=",
    "path": "github.com/cznic/ql",
    "revision": "47bf73cf8ed137969f12b223b7016784cdd9d42e",
    "revisionTime": "2018-08-02T13:12:42Z"
},
{
    "checksumSHA1": "/zeOOjyZpzAOAs7OhWTycaNfZsg=",
    "path": "github.com/cznic/ql/driver",
    "revision": "eccf3e9caa43fab70d7ddfdee5c3fb73fc76c2a1",
    "revisionTime": "2018-02-28T09:13:17Z"
},
{
    "checksumSHA1": "Lzy3qrO6lA6AseE2PIzFRb7bTno=",
    "path": "github.com/cznic/ql/vendored/github.com/camlistore/go4/lock",
    "revision": "eccf3e9caa43fab70d7ddfdee5c3fb73fc76c2a1",
    "revisionTime": "2018-02-28T09:13:17Z"
},
{
    "checksumSHA1": "z5JpH57/did5gbowLKrvV0V5tWo=",
    "path": "github.com/cznic/sortutil",
    "revision": "4c7342852e65c2088c981288f2c5610d10b9f7f4",
    "revisionTime": "2015-06-17T08:33:42Z"
},
{
    "checksumSHA1": "cc3pslBQu+mM91PAcKac8/RCPMQ=",
    "path": "github.com/cznic/strutil",
    "revision": "529a34b1c186b483642a7a230c67521d9aa4b0fb",
    "revisionTime": "2017-10-16T13:45:53Z"
},
{
    "checksumSHA1": "cBDjw6mDDHwvI8ji+leQmrpm5qw=",
    "path": "github.com/cznic/zappy",
    "revision": "2533cb5b45cc6c07421468ce262899ddc9d53fb7",
    "revisionTime": "2016-07-23T13:35:15Z"
}

In our use-case, the write goroutines are around 250, the read goroutines are around 10.

[cznic]

Thanks, that should enable me to pinpoint the problem, hopefully.