search

uptrace / bun

SQL-first Golang ORM
https://bun.uptrace.dev
BSD 2-Clause "Simplified" License
3.46k stars 206 forks source link

race detected in ```schema.Field.Clone()``` #970

Closed NicholasBlaskey closed 3 months ago

NicholasBlaskey commented 3 months ago

Hello,

Using Bun one of our tests spawns goroutines which have insert and select queries. Around 5% of the time the race detector will find this issue.

Please let me know if this is a known issue or if there is anything to do to workaround this. I'm also happy to contribute a fix if the project would be okay with that too. Thanks.

This is using postgres and 1.17 version of Bun and 1.22 of Go but have observed the issue on Bun 1.14.

WARNING: DATA RACE
Read at 0x00c001802a80 by goroutine 215:
  github.com/uptrace/bun/schema.(*Field).Clone()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/field.go:48 +0x2a4
  github.com/uptrace/bun/schema.(*Table).inlineFields()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/table.go:820 +0x279
  github.com/uptrace/bun/schema.(*Table).initRelations()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/table.go:423 +0x4a4
  github.com/uptrace/bun/schema.(*Table).init2()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/table.go:130 +0x44
  github.com/uptrace/bun/schema.(*tableInProgress).init2.func1()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/tables.go:34 +0x2a
  sync.(*Once).doSlow()
      /usr/local/go/src/sync/once.go:74 +0xf0
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:65 +0x44
  github.com/uptrace/bun/schema.(*tableInProgress).init2()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/tables.go:33 +0x8d
  github.com/uptrace/bun/schema.(*Tables).table()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/tables.go:104 +0x3b2
  github.com/uptrace/bun/schema.(*Tables).Get()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/tables.go:62 +0x64
  github.com/uptrace/bun.(*DB).Table()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/db.go:195 +0xe
  github.com/uptrace/bun.newStructTableModelValue()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/model_table_struct.go:47 +0xa4
  github.com/uptrace/bun._newModel()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/model.go:123 +0xbf3
  github.com/uptrace/bun.newSingleModel()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/model.go:75 +0x55
  github.com/uptrace/bun.(*baseQuery).setModel()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/query_base.go:163 +0x2e
  github.com/uptrace/bun.(*InsertQuery).Model()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/query_insert.go:47 +0x84
  github.com/determined-ai/determined/master/internal/db.AddTaskTx()
      /home/circleci/project/master/internal/db/postgres_tasks.go:34 +0x52
  github.com/determined-ai/determined/master/internal/db.AddTask()
      /home/circleci/project/master/internal/db/postgres_tasks.go:29 +0x30c
  github.com/determined-ai/determined/master/internal.(*trial).addTask()
      /home/circleci/project/master/internal/trial.go:480 +0x3f
  github.com/determined-ai/determined/master/internal.(*trial).create()
      /home/circleci/project/master/internal/trial.go:306 +0x1b5
  github.com/determined-ai/determined/master/internal.newTrial()
      /home/circleci/project/master/internal/trial.go:165 +0xad4
  github.com/determined-ai/determined/master/internal.(*internalExperiment).processOperations()
      /home/circleci/project/master/internal/experiment.go:817 +0x1224
  github.com/determined-ai/determined/master/internal.(*internalExperiment).start()
      /home/circleci/project/master/internal/experiment.go:316 +0xc35
  github.com/determined-ai/determined/master/internal.(*internalExperiment).Start()
      /home/circleci/project/master/internal/experiment.go:228 +0x148
  github.com/determined-ai/determined/master/internal.(*Master).restoreExperiment()
      /home/circleci/project/master/internal/restore.go:130 +0xfa8
  github.com/determined-ai/determined/master/internal.(*Master).tryRestoreExperiment()
      /home/circleci/project/master/internal/core.go:798 +0x144
  github.com/determined-ai/determined/master/internal.(*Master).restoreNonTerminalExperiments.gowrap1()
      /home/circleci/project/master/internal/core.go:838 +0x5d

Previous write at 0x00c001802a80 by goroutine 312:
  github.com/uptrace/bun/dialect/pgdialect.(*Dialect).onField()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun/dialect/pgdialect@v1.1.17/dialect.go:76 +0x6f
  github.com/uptrace/bun/dialect/pgdialect.(*Dialect).OnTable()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun/dialect/pgdialect@v1.1.17/dialect.go:71 +0xea
  github.com/uptrace/bun/schema.(*Tables).table()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/tables.go:113 +0x488
  github.com/uptrace/bun/schema.(*Tables).Get()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/schema/tables.go:62 +0x64
  github.com/uptrace/bun.(*DB).Table()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/db.go:195 +0xe
  github.com/uptrace/bun.newStructTableModelValue()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/model_table_struct.go:47 +0xa4
  github.com/uptrace/bun._newModel()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/model.go:123 +0xbf3
  github.com/uptrace/bun.newSingleModel()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/model.go:75 +0x55
  github.com/uptrace/bun.(*baseQuery).setModel()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/query_base.go:163 +0x2e
  github.com/uptrace/bun.(*SelectQuery).Model()
      /home/circleci/.go_workspace/pkg/mod/github.com/uptrace/bun@v1.1.17/query_select.go:60 +0x1e4
  github.com/determined-ai/determined/master/internal/db.JobByID()
      /home/circleci/project/master/internal/db/postgres_jobs.go:35 +0xba
  github.com/determined-ai/determined/master/internal.(*internalExperiment).start()
      /home/circleci/project/master/internal/experiment.go:278 +0x713
  github.com/determined-ai/determined/master/internal.(*internalExperiment).Start()
      /home/circleci/project/master/internal/experiment.go:228 +0x148
  github.com/determined-ai/determined/master/internal.(*Master).restoreExperiment()
      /home/circleci/project/master/internal/restore.go:130 +0xfa8
  github.com/determined-ai/determined/master/internal.(*Master).tryRestoreExperiment()
      /home/circleci/project/master/internal/core.go:798 +0x144
  github.com/determined-ai/determined/master/internal.(*Master).restoreNonTerminalExperiments.gowrap1()
      /home/circleci/project/master/internal/core.go:838 +0x5d

Goroutine 215 (running) created at:
  github.com/determined-ai/determined/master/internal.(*Master).restoreNonTerminalExperiments()
      /home/circleci/project/master/internal/core.go:838 +0xe4
  github.com/determined-ai/determined/master/internal.(*Master).Run()
      /home/circleci/project/master/internal/core.go:1259 +0x4698
  github.com/determined-ai/determined/master/test/testutils.RunMaster.func1()
      /home/circleci/project/master/test/testutils/fixtures.go:89 +0x8e

Goroutine 312 (running) created at:
  github.com/determined-ai/determined/master/internal.(*Master).restoreNonTerminalExperiments()
      /home/circleci/project/master/internal/core.go:838 +0xe4
  github.com/determined-ai/determined/master/internal.(*Master).Run()
      /home/circleci/project/master/internal/core.go:1259 +0x4698
  github.com/determined-ai/determined/master/test/testutils.RunMaster.func1()
      /home/circleci/project/master/test/testutils/fixtures.go:89 +0x8e

I've looked at https://github.com/uptrace/bun/issues/919 but still seeing the issue on 1.17

vmihailenco commented 3 months ago

Presumably this should be fixed by https://github.com/uptrace/bun/commit/9052fc4cdf3a1e06facae872d0f2c12d0b1e5bfb . Please try v1.2.0

NicholasBlaskey commented 3 months ago

Thanks so much! That seems to have resolved the issue!