search

sijms / go-ora

Pure go oracle client
MIT License
795 stars 176 forks source link

Upgrading v2.8.7 to v2.8.9 leads to golang race detector errors #518

Closed bobvawter closed 5 months ago

bobvawter commented 7 months ago

It looks like some recent changes have introduced a thread-safety problem when using a pooled cache of prepared statements created against a *sql.DB and then subsequently bound to a *sql.TX. When running our integration tests with go test -race, we see a number of data races identified in a high-concurrency test suite. There are also many instances of data integrity check failed errors when attempting to execute the prepared statements.

The general approach we take when executing a novel statement fingerprint is to call *sql.DB.PrepareContext() to create a *sql.Stmt. These statement objects are stored in an LRU cache. When we need to execute a statement, we call *sql.Tx.StmtContext() to bind the statement to the transaction and then execute the returned, bound *sql.Stmt.

For comparison, this same code base is used with pgx and the go-mysql drivers.

==================
WARNING: DATA RACE
Read at 0x00c0012ea218 by goroutine 588786:
  github.com/sijms/go-ora/v2/network.(*Session).writePacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:872 +0x70
  github.com/sijms/go-ora/v2/network.(*Session).BreakConnection()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:431 +0x264
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:192 +0x17e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:203 +0x47

Previous write at 0x00c0012ea218 by goroutine 4145:
  github.com/sijms/go-ora/v2/network.(*Session).resetWrite()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:164 +0x1864
  github.com/sijms/go-ora/v2/network.(*Session).ResetBuffer()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:175 +0x184d
  github.com/sijms/go-ora/v2.(*Stmt)._exec()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/command.go:1987 +0x17fc
  github.com/sijms/go-ora/v2.(*Stmt).ExecContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/command.go:1282 +0x317
  database/sql.ctxDriverStmtExec()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/ctxutil.go:65 +0xf5
  database/sql.resultFromStatement()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2635 +0x1be
  database/sql.(*Stmt).ExecContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2609 +0x10a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*Stmt).ExecContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2603 +0x19c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:496 +0x7ee
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:332 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:230 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1ea
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:85 +0xb9
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepOnce()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:265 +0x159b
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:143 +0x5f0
  github.com/cockroachdb/cdc-sink/internal/util/stopvar.DoWhenChangedOrInterval[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopvar/stopvar.go:71 +0x253
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:122 +0x1fc
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).Start.func1.1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:97 +0x2d9
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4

Goroutine 588786 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:184 +0x2da
  github.com/sijms/go-ora/v2.(*Stmt).ExecContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/command.go:1278 +0x199
  database/sql.ctxDriverStmtExec()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/ctxutil.go:65 +0xf5
  database/sql.resultFromStatement()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2635 +0x1be
  database/sql.(*Stmt).ExecContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2609 +0x10a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*Stmt).ExecContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2603 +0x19c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:496 +0x7ee
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:332 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:230 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1ea
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:85 +0xb9
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepOnce()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:265 +0x159b
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:143 +0x5f0
  github.com/cockroachdb/cdc-sink/internal/util/stopvar.DoWhenChangedOrInterval[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopvar/stopvar.go:71 +0x253
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:122 +0x1fc
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).Start.func1.1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:97 +0x2d9
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4

Goroutine 4145 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:185 +0x144
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).Start.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:91 +0xd7
  github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.LeaseGroup.func1.1.1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/lease_group.go:56 +0xbc
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
==================
==================
WARNING: DATA RACE
Read at 0x00c001119ca0 by goroutine 588786:
  runtime.growslice()
      /opt/hostedtoolcache/go/1.20.14/x64/src/runtime/slice.go:157 +0x0
  github.com/sijms/go-ora/v2/network.(*Session).writePacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:872 +0xa9
  github.com/sijms/go-ora/v2/network.(*Session).BreakConnection()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:431 +0x264
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:192 +0x17e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:203 +0x47

Previous write at 0x00c001119ca0 by goroutine 4145:
  github.com/sijms/go-ora/v2/network.(*Session).writePacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:872 +0xdb
  github.com/sijms/go-ora/v2/network.(*Session).Write()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:649 +0x4cf
  github.com/sijms/go-ora/v2.(*Stmt).write()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/command.go:559 +0x824
  github.com/sijms/go-ora/v2.(*Stmt)._exec()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/command.go:1988 +0x1984
  github.com/sijms/go-ora/v2.(*Stmt).ExecContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/command.go:1282 +0x317
  database/sql.ctxDriverStmtExec()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/ctxutil.go:65 +0xf5
  database/sql.resultFromStatement()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2635 +0x1be
  database/sql.(*Stmt).ExecContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2609 +0x10a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*Stmt).ExecContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2603 +0x19c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:496 +0x7ee
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:332 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:230 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1ea
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:85 +0xb9
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepOnce()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:265 +0x159b
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:143 +0x5f0
  github.com/cockroachdb/cdc-sink/internal/util/stopvar.DoWhenChangedOrInterval[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopvar/stopvar.go:71 +0x253
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:122 +0x1fc
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).Start.func1.1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:97 +0x2d9
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4

Goroutine 588786 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:184 +0x2da
  github.com/sijms/go-ora/v2.(*Stmt).ExecContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/command.go:1278 +0x199
  database/sql.ctxDriverStmtExec()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/ctxutil.go:65 +0xf5
  database/sql.resultFromStatement()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2635 +0x1be
  database/sql.(*Stmt).ExecContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2609 +0x10a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*Stmt).ExecContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2603 +0x19c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:496 +0x7ee
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:332 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:230 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1ea
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:85 +0xb9
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepOnce()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:265 +0x159b
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:143 +0x5f0
  github.com/cockroachdb/cdc-sink/internal/util/stopvar.DoWhenChangedOrInterval[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopvar/stopvar.go:71 +0x253
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:122 +0x1fc
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).Start.func1.1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:97 +0x2d9
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4

Goroutine 4145 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:185 +0x144
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).Start.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:91 +0xd7
  github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.LeaseGroup.func1.1.1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/lease_group.go:56 +0xbc
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
==================
==================
WARNING: DATA RACE
Read at 0x00c0012ea678 by goroutine 4146:
  github.com/sijms/go-ora/v2/network.(*Session).IsBreak()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:337 +0xcb
  github.com/sijms/go-ora/v2/network.(*Session).read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:767 +0xdb
  github.com/sijms/go-ora/v2/network.(*Session).GetByte()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:1511 +0x75
  github.com/sijms/go-ora/v2.(*Connection).read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/connection.go:357 +0x7d
  github.com/sijms/go-ora/v2.(*Stmt).ExecContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/command.go:1284 +0x394
  database/sql.ctxDriverStmtExec()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/ctxutil.go:65 +0xf5
  database/sql.resultFromStatement()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2635 +0x1be
  database/sql.(*Stmt).ExecContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2609 +0x10a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*Stmt).ExecContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2603 +0x19c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:496 +0x7ee
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:332 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:230 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1ea
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:85 +0xb9
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepOnce()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:265 +0x159b
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:143 +0x5f0
  github.com/cockroachdb/cdc-sink/internal/util/stopvar.DoWhenChangedOrInterval[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopvar/stopvar.go:71 +0x253
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:122 +0x1fc
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).Start.func1.1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:97 +0x2d9
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4

Previous write at 0x00c0012ea678 by goroutine 588767:
  github.com/sijms/go-ora/v2/network.(*Session).BreakConnection()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:436 +0x284
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:192 +0x17e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:203 +0x47

Goroutine 4146 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:185 +0x144
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).Start.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:91 +0xd7
  github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.LeaseGroup.func1.1.1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/lease_group.go:56 +0xbc
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4

Goroutine 588767 (finished) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/network/session.go:184 +0x2da
  github.com/sijms/go-ora/v2.(*Stmt).ExecContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.9/command.go:1278 +0x199
  database/sql.ctxDriverStmtExec()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/ctxutil.go:65 +0xf5
  database/sql.resultFromStatement()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2635 +0x1be
  database/sql.(*Stmt).ExecContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2609 +0x10a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*Stmt).ExecContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:2603 +0x19c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:496 +0x7ee
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:332 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:230 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1ea
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:85 +0xb9
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepOnce()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:265 +0x159b
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:143 +0x5f0
  github.com/cockroachdb/cdc-sink/internal/util/stopvar.DoWhenChangedOrInterval[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopvar/stopvar.go:71 +0x253
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).sweepTable()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:122 +0x1fc
  github.com/cockroachdb/cdc-sink/internal/sequencer/besteffort.(*BestEffort).Start.func1.1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/besteffort/best_effort.go:97 +0x2d9
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
==================
sijms commented 7 months ago

fixed in next release

sijms commented 7 months ago

would you please test it in last commit before release

bobvawter commented 7 months ago

It appears there's still a data race in the driver with commit 2ce633cb669f. I've attached the (copious) output from a test run. It appears there are several different interactions between Connect(), writePacket(), resetWrite(), and a cleanup? goroutine.

The race detector writes those "WARNING: DATA RACE" blocks as each test case exits.

26go-tests Integration Tests (v23.1, oracle-v21.3, oracle_systemSoupOrSecret@127.0.0.11521_XEPDB1).txt

bobvawter commented 7 months ago

If running that test suite locally would be helpful for you, the environment can be set up by cloning https://github.com/cockroachdb/cdc-sink/blob/master/.github/docker-compose.yml

docker-compose up cockroachdb-v23.1 and the oracle-v21.3 targets (which also sets up NNE). Then, on your development host:

export TEST_TARGET_CONNECT=oracle://system:SoupOrSecret@127.0.0.1:1521/XEPDB1
go test -race -p 4 -v ./...

I'm in crunch mode for the next couple of weeks and apologize for not being able to provide a minum-repro case. Thank you for your efforts on go-ora!

sijms commented 7 months ago

I find the issue after connection the driver will set the value of session.Context inside session.Connect function at the same time the watchdog go-routine created inside session.StartContext try to use the value of session.Context.ConnOption.Tracer

sijms commented 6 months ago

I use your guid to test it locally docker-compose up cockroachdb-v23.1 is working fine but docker-compose up oracle-v21.3 give error

sijms commented 6 months ago

I also push a new commit that fix most of race conditioning

sijms commented 6 months ago

would you please test v2.8.10. I can't do it locally

bobvawter commented 6 months ago

With v2.8.10, I see the following data races:

2024-03-19T13:52:35.6073503Z ==================
2024-03-19T13:52:35.6074135Z WARNING: DATA RACE
2024-03-19T13:52:35.6075329Z Write at 0x00c0001ae140 by goroutine 139:
2024-03-19T13:52:35.6076602Z   github.com/sijms/go-ora/v2/network.(*Session).Connect()
2024-03-19T13:52:35.6078416Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:561 +0x1909
2024-03-19T13:52:35.6080176Z   github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
2024-03-19T13:52:35.6081979Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:429 +0xdef
2024-03-19T13:52:35.6083628Z   github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
2024-03-19T13:52:35.6085659Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
2024-03-19T13:52:35.6086741Z   database/sql.(*DB).conn()
2024-03-19T13:52:35.6087946Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
2024-03-19T13:52:35.6089142Z   database/sql.(*DB).PingContext.func1()
2024-03-19T13:52:35.6090371Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:850 +0x6b
2024-03-19T13:52:35.6091863Z   database/sql.(*DB).retry()
2024-03-19T13:52:35.6093085Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
2024-03-19T13:52:35.6094176Z   database/sql.(*DB).PingContext()
2024-03-19T13:52:35.6095560Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:849 +0xc8
2024-03-19T13:52:35.6096650Z   database/sql.(*DB).Ping()
2024-03-19T13:52:35.6097832Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:867 +0x625
2024-03-19T13:52:35.6099996Z   github.com/cockroachdb/cdc-sink/internal/util/stdpool.OpenOracleAsTarget()
2024-03-19T13:52:35.6101899Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stdpool/ora.go:66 +0x5ce
2024-03-19T13:52:35.6104032Z   github.com/cockroachdb/cdc-sink/internal/util/stdpool.OpenTarget()
2024-03-19T13:52:35.6106154Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stdpool/target.go:44 +0x268
2024-03-19T13:52:35.6108247Z   github.com/cockroachdb/cdc-sink/internal/sinktest/base.ProvideTargetPool()
2024-03-19T13:52:35.6110268Z       /home/runner/work/cdc-sink/cdc-sink/internal/sinktest/base/provider.go:254 +0x5aa
2024-03-19T13:52:35.6112306Z   github.com/cockroachdb/cdc-sink/internal/sinktest/all.NewFixture()
2024-03-19T13:52:35.6114075Z       /home/runner/work/cdc-sink/cdc-sink/internal/sinktest/all/wire_gen.go:46 +0x235
2024-03-19T13:52:35.6115635Z   github.com/cockroachdb/cdc-sink/internal/script.TestScript()
2024-03-19T13:52:35.6117277Z       /home/runner/work/cdc-sink/cdc-sink/internal/script/script_test.go:55 +0x93
2024-03-19T13:52:35.6118436Z   testing.tRunner()
2024-03-19T13:52:35.6119717Z       /opt/hostedtoolcache/go/1.20.14/x64/src/testing/testing.go:1576 +0x216
2024-03-19T13:52:35.6120809Z   testing.(*T).Run.func1()
2024-03-19T13:52:35.6122115Z       /opt/hostedtoolcache/go/1.20.14/x64/src/testing/testing.go:1629 +0x47
2024-03-19T13:52:35.6122920Z 
2024-03-19T13:52:35.6123305Z Previous read at 0x00c0001ae140 by goroutine 178:
2024-03-19T13:52:35.6125134Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
2024-03-19T13:52:35.6130113Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:194 +0x84
2024-03-19T13:52:35.6132023Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
2024-03-19T13:52:35.6133772Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:211 +0x58
2024-03-19T13:52:35.6134795Z 
2024-03-19T13:52:35.6135160Z Goroutine 139 (running) created at:
2024-03-19T13:52:35.6135870Z   testing.(*T).Run()
2024-03-19T13:52:35.6137264Z       /opt/hostedtoolcache/go/1.20.14/x64/src/testing/testing.go:1629 +0x805
2024-03-19T13:52:35.6138592Z   testing.runTests.func1()
2024-03-19T13:52:35.6139886Z       /opt/hostedtoolcache/go/1.20.14/x64/src/testing/testing.go:2036 +0x8d
2024-03-19T13:52:35.6141171Z   testing.tRunner()
2024-03-19T13:52:35.6142250Z       /opt/hostedtoolcache/go/1.20.14/x64/src/testing/testing.go:1576 +0x216
2024-03-19T13:52:35.6143468Z   testing.runTests()
2024-03-19T13:52:35.6145004Z       /opt/hostedtoolcache/go/1.20.14/x64/src/testing/testing.go:2034 +0x87c
2024-03-19T13:52:35.6146312Z   testing.(*M).Run()
2024-03-19T13:52:35.6147438Z       /opt/hostedtoolcache/go/1.20.14/x64/src/testing/testing.go:1906 +0xb44
2024-03-19T13:52:35.6149012Z   main.main()
2024-03-19T13:52:35.6149949Z       _testmain.go:84 +0x2fc
2024-03-19T13:52:35.6150369Z 
2024-03-19T13:52:35.6150823Z Goroutine 178 (running) created at:
2024-03-19T13:52:35.6151933Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext()
2024-03-19T13:52:35.6153734Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:190 +0x324
2024-03-19T13:52:35.6155365Z   github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
2024-03-19T13:52:35.6157070Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:427 +0xd85
2024-03-19T13:52:35.6158540Z   github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
2024-03-19T13:52:35.6160319Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
2024-03-19T13:52:35.6161498Z   database/sql.(*DB).conn()
2024-03-19T13:52:35.6162607Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
2024-03-19T13:52:35.6163868Z   database/sql.(*DB).PingContext.func1()
2024-03-19T13:52:35.6165128Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:850 +0x6b
2024-03-19T13:52:35.6166357Z   database/sql.(*DB).retry()
2024-03-19T13:52:35.6167567Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
2024-03-19T13:52:35.6168959Z   database/sql.(*DB).PingContext()
2024-03-19T13:52:35.6170355Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:849 +0xc8
2024-03-19T13:52:35.6171666Z   database/sql.(*DB).Ping()
2024-03-19T13:52:35.6172787Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:867 +0x625
2024-03-19T13:52:35.6174476Z   github.com/cockroachdb/cdc-sink/internal/util/stdpool.OpenOracleAsTarget()
2024-03-19T13:52:35.6176173Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stdpool/ora.go:66 +0x5ce
2024-03-19T13:52:35.6178051Z   github.com/cockroachdb/cdc-sink/internal/util/stdpool.OpenTarget()
2024-03-19T13:52:35.6179824Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stdpool/target.go:44 +0x268
2024-03-19T13:52:35.6181375Z   github.com/cockroachdb/cdc-sink/internal/sinktest/base.ProvideTargetPool()
2024-03-19T13:52:35.6183369Z       /home/runner/work/cdc-sink/cdc-sink/internal/sinktest/base/provider.go:254 +0x5aa
2024-03-19T13:52:35.6184887Z   github.com/cockroachdb/cdc-sink/internal/sinktest/all.NewFixture()
2024-03-19T13:52:35.6186589Z       /home/runner/work/cdc-sink/cdc-sink/internal/sinktest/all/wire_gen.go:46 +0x235
2024-03-19T13:52:35.6188212Z   github.com/cockroachdb/cdc-sink/internal/script.TestScript()
2024-03-19T13:52:35.6189841Z       /home/runner/work/cdc-sink/cdc-sink/internal/script/script_test.go:55 +0x93
2024-03-19T13:52:35.6191045Z   testing.tRunner()
2024-03-19T13:52:35.6192150Z       /opt/hostedtoolcache/go/1.20.14/x64/src/testing/testing.go:1576 +0x216
2024-03-19T13:52:35.6193184Z   testing.(*T).Run.func1()
2024-03-19T13:52:35.6196739Z       /opt/hostedtoolcache/go/1.20.14/x64/src/testing/testing.go:1629 +0x47
2024-03-19T13:52:35.6198130Z ==================
==================
WARNING: DATA RACE
Read at 0x00c00053a2e0 by goroutine 381:
  github.com/sijms/go-ora/v2/network.newDataPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/data_packet.go:29 +0x70
  github.com/sijms/go-ora/v2/network.(*Session).WriteFinalPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:600 +0x4e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:204 +0x27e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:211 +0x58

Previous write at 0x00c00053a2e0 by goroutine 371:
  github.com/sijms/go-ora/v2/advanced_nego.(*dataIntegrityService).activateAlgorithm()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/advanced_nego/data_integrity_service.go:154 +0x85a
  github.com/sijms/go-ora/v2/advanced_nego.(*AdvNego).StartServices()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/advanced_nego/advanced_nego.go:244 +0x79
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:448 +0xfab
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).prepare()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
  database/sql.(*DB).PrepareContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).PrepareContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
  github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
  github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 381 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).prepare()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
  database/sql.(*DB).PrepareContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).PrepareContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
  github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
  github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 371 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).enqueue()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:452 +0x6f5
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).Schedule()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:216 +0x244
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:39 +0x12b
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:83 +0x23c
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.4()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:181 +0x1d8
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.6()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:257 +0x25c
  github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.(*Copier).Run()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/copier.go:73 +0x22e
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.8()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:271 +0x5c
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
==================
==================
WARNING: DATA RACE
Read at 0x00c0001c64a8 by goroutine 381:
  github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Compute()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/security/general.go:130 +0x64
  github.com/sijms/go-ora/v2/network.newDataPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/data_packet.go:30 +0xd9
  github.com/sijms/go-ora/v2/network.(*Session).WriteFinalPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:600 +0x4e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:204 +0x27e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:211 +0x58

Previous write at 0x00c0001c64a8 by goroutine 371:
  github.com/sijms/go-ora/v2/network/security.NewOracleNetworkHash2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/security/general.go:62 +0x104
  github.com/sijms/go-ora/v2/advanced_nego.(*dataIntegrityService).activateAlgorithm()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/advanced_nego/data_integrity_service.go:145 +0x712
  github.com/sijms/go-ora/v2/advanced_nego.(*AdvNego).StartServices()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/advanced_nego/advanced_nego.go:244 +0x79
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:448 +0xfab
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).prepare()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
  database/sql.(*DB).PrepareContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).PrepareContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
  github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
  github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 381 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).prepare()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
  database/sql.(*DB).PrepareContext.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).PrepareContext()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
  github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
  github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
  github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
  github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
      /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 371 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).enqueue()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:452 +0x6f5
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).Schedule()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:216 +0x244
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:39 +0x12b
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:83 +0x23c
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.4()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:181 +0x1d8
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.6()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:257 +0x25c
  github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.(*Copier).Run()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/copier.go:73 +0x22e
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.8()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:271 +0x5c
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
==================
2024-03-19T14:09:50.8744811Z ==================
2024-03-19T14:09:50.8745010Z WARNING: DATA RACE
2024-03-19T14:09:50.8745251Z Write at 0x00c000054900 by goroutine 381:
2024-03-19T14:09:50.8745481Z   crypto/sha256.(*digest).Reset()
2024-03-19T14:09:50.8746191Z       /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:124 +0x10a
2024-03-19T14:09:50.8746867Z   github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Compute()
2024-03-19T14:09:50.8747959Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/security/general.go:131 +0x11a
2024-03-19T14:09:50.8748630Z   github.com/sijms/go-ora/v2/network.newDataPacket()
2024-03-19T14:09:50.8749649Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/data_packet.go:30 +0xd9
2024-03-19T14:09:50.8750206Z   github.com/sijms/go-ora/v2/network.(*Session).WriteFinalPacket()
2024-03-19T14:09:50.8751183Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:600 +0x4e
2024-03-19T14:09:50.8751747Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
2024-03-19T14:09:50.8752723Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:204 +0x27e
2024-03-19T14:09:50.8753290Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
2024-03-19T14:09:50.8754268Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:211 +0x58
2024-03-19T14:09:50.8754280Z 
2024-03-19T14:09:50.8754582Z Previous write at 0x00c000054900 by goroutine 371:
2024-03-19T14:09:50.8754809Z   crypto/sha256.(*digest).Reset()
2024-03-19T14:09:50.8755492Z       /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:124 +0x10a
2024-03-19T14:09:50.8756180Z   github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Validate()
2024-03-19T14:09:50.8757432Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/security/general.go:166 +0x199
2024-03-19T14:09:50.8757959Z   github.com/sijms/go-ora/v2/network.newDataPacketFromData()
2024-03-19T14:09:50.8758968Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/data_packet.go:94 +0x78f
2024-03-19T14:09:50.8759467Z   github.com/sijms/go-ora/v2/network.(*Session).readPacket()
2024-03-19T14:09:50.8760474Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:1089 +0x13bb
2024-03-19T14:09:50.8760920Z   github.com/sijms/go-ora/v2/network.(*Session).read()
2024-03-19T14:09:50.8761881Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:789 +0xe9
2024-03-19T14:09:50.8762352Z   github.com/sijms/go-ora/v2/network.(*Session).GetByte()
2024-03-19T14:09:50.8763483Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:1553 +0x1eb
2024-03-19T14:09:50.8763876Z   github.com/sijms/go-ora/v2.newAuthObject()
2024-03-19T14:09:50.8764799Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/auth_object.go:52 +0x1f3
2024-03-19T14:09:50.8765146Z   github.com/sijms/go-ora/v2.(*Connection).doAuth()
2024-03-19T14:09:50.8766090Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:603 +0xa8f
2024-03-19T14:09:50.8766603Z   github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
2024-03-19T14:09:50.8767509Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:462 +0xff5
2024-03-19T14:09:50.8767976Z   github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
2024-03-19T14:09:50.8768890Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
2024-03-19T14:09:50.8769133Z   database/sql.(*DB).conn()
2024-03-19T14:09:50.8769742Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
2024-03-19T14:09:50.8769904Z   database/sql.(*DB).prepare()
2024-03-19T14:09:50.8770409Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
2024-03-19T14:09:50.8770750Z   database/sql.(*DB).PrepareContext.func1()
2024-03-19T14:09:50.8771407Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
2024-03-19T14:09:50.8771654Z   database/sql.(*DB).retry()
2024-03-19T14:09:50.8772294Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
2024-03-19T14:09:50.8772552Z   database/sql.(*DB).PrepareContext()
2024-03-19T14:09:50.8773178Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
2024-03-19T14:09:50.8773884Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
2024-03-19T14:09:50.8774796Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
2024-03-19T14:09:50.8775763Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
2024-03-19T14:09:50.8776682Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
2024-03-19T14:09:50.8777441Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
2024-03-19T14:09:50.8778269Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
2024-03-19T14:09:50.8778985Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
2024-03-19T14:09:50.8779824Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
2024-03-19T14:09:50.8780473Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
2024-03-19T14:09:50.8781326Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
2024-03-19T14:09:50.8782108Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
2024-03-19T14:09:50.8782974Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
2024-03-19T14:09:50.8783742Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
2024-03-19T14:09:50.8784737Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
2024-03-19T14:09:50.8785536Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
2024-03-19T14:09:50.8786403Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
2024-03-19T14:09:50.8787105Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
2024-03-19T14:09:50.8787968Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
2024-03-19T14:09:50.8788822Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
2024-03-19T14:09:50.8789661Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
2024-03-19T14:09:50.8790252Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
2024-03-19T14:09:50.8791034Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
2024-03-19T14:09:50.8791556Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
2024-03-19T14:09:50.8792364Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
2024-03-19T14:09:50.8792881Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
2024-03-19T14:09:50.8793663Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
2024-03-19T14:09:50.8794469Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
2024-03-19T14:09:50.8795319Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
2024-03-19T14:09:50.8796144Z   github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
2024-03-19T14:09:50.8796693Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
2024-03-19T14:09:50.8797055Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
2024-03-19T14:09:50.8797518Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
2024-03-19T14:09:50.8797939Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
2024-03-19T14:09:50.8798430Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
2024-03-19T14:09:50.8798814Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
2024-03-19T14:09:50.8799313Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
2024-03-19T14:09:50.8799742Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
2024-03-19T14:09:50.8800252Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
2024-03-19T14:09:50.8800260Z 
2024-03-19T14:09:50.8800411Z Goroutine 381 (running) created at:
2024-03-19T14:09:50.8800983Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext()
2024-03-19T14:09:50.8801974Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:190 +0x324
2024-03-19T14:09:50.8802471Z   github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
2024-03-19T14:09:50.8803540Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:427 +0xd85
2024-03-19T14:09:50.8803999Z   github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
2024-03-19T14:09:50.8804905Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
2024-03-19T14:09:50.8805123Z   database/sql.(*DB).conn()
2024-03-19T14:09:50.8805788Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
2024-03-19T14:09:50.8806008Z   database/sql.(*DB).prepare()
2024-03-19T14:09:50.8806665Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
2024-03-19T14:09:50.8806950Z   database/sql.(*DB).PrepareContext.func1()
2024-03-19T14:09:50.8807596Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
2024-03-19T14:09:50.8807810Z   database/sql.(*DB).retry()
2024-03-19T14:09:50.8808683Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
2024-03-19T14:09:50.8808938Z   database/sql.(*DB).PrepareContext()
2024-03-19T14:09:50.8809566Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
2024-03-19T14:09:50.8810226Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
2024-03-19T14:09:50.8811138Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
2024-03-19T14:09:50.8811835Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
2024-03-19T14:09:50.8812739Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
2024-03-19T14:09:50.8813462Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
2024-03-19T14:09:50.8814311Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
2024-03-19T14:09:50.8814975Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
2024-03-19T14:09:50.8815807Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
2024-03-19T14:09:50.8816414Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
2024-03-19T14:09:50.8817258Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
2024-03-19T14:09:50.8818012Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
2024-03-19T14:09:50.8818856Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
2024-03-19T14:09:50.8819595Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
2024-03-19T14:09:50.8820375Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
2024-03-19T14:09:50.8821115Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
2024-03-19T14:09:50.8821912Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
2024-03-19T14:09:50.8822579Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
2024-03-19T14:09:50.8823431Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
2024-03-19T14:09:50.8824224Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
2024-03-19T14:09:50.8825066Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
2024-03-19T14:09:50.8825617Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
2024-03-19T14:09:50.8826394Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
2024-03-19T14:09:50.8826877Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
2024-03-19T14:09:50.8827667Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
2024-03-19T14:09:50.8828364Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
2024-03-19T14:09:50.8829150Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
2024-03-19T14:09:50.8829934Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
2024-03-19T14:09:50.8830784Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
2024-03-19T14:09:50.8831577Z   github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
2024-03-19T14:09:50.8832537Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
2024-03-19T14:09:50.8833128Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
2024-03-19T14:09:50.8833968Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
2024-03-19T14:09:50.8834669Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
2024-03-19T14:09:50.8835517Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
2024-03-19T14:09:50.8836317Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
2024-03-19T14:09:50.8837247Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
2024-03-19T14:09:50.8837984Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
2024-03-19T14:09:50.8838889Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
2024-03-19T14:09:50.8838900Z 
2024-03-19T14:09:50.8839104Z Goroutine 371 (running) created at:
2024-03-19T14:09:50.8839790Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
2024-03-19T14:09:50.8840689Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
2024-03-19T14:09:50.8841291Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
2024-03-19T14:09:50.8842185Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
2024-03-19T14:09:50.8842832Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
2024-03-19T14:09:50.8843755Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
2024-03-19T14:09:50.8844156Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).enqueue()
2024-03-19T14:09:50.8844858Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:452 +0x6f5
2024-03-19T14:09:50.8845531Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).Schedule()
2024-03-19T14:09:50.8846374Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:216 +0x244
2024-03-19T14:09:50.8847092Z   github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch()
2024-03-19T14:09:50.8848050Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:39 +0x12b
2024-03-19T14:09:50.8848769Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit()
2024-03-19T14:09:50.8849598Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:83 +0x23c
2024-03-19T14:09:50.8850314Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.4()
2024-03-19T14:09:50.8851150Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:181 +0x1d8
2024-03-19T14:09:50.8851861Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.6()
2024-03-19T14:09:50.8852689Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:257 +0x25c
2024-03-19T14:09:50.8853343Z   github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.(*Copier).Run()
2024-03-19T14:09:50.8854235Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/copier.go:73 +0x22e
2024-03-19T14:09:50.8854931Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.8()
2024-03-19T14:09:50.8855865Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:271 +0x5c
2024-03-19T14:09:50.8856273Z   github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
2024-03-19T14:09:50.8856832Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
2024-03-19T14:09:50.8857008Z ==================
2024-03-19T14:09:50.7478706Z ==================
2024-03-19T14:09:50.7478969Z WARNING: DATA RACE
2024-03-19T14:09:50.7479287Z Read at 0x00c0008c9af8 by goroutine 383:
2024-03-19T14:09:50.7479753Z   crypto/cipher.(*cbcEncrypter).CryptBlocks()
2024-03-19T14:09:50.7480700Z       /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/cipher/cbc.go:83 +0x12c
2024-03-19T14:09:50.7481498Z   github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Compute()
2024-03-19T14:09:50.7482530Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/security/general.go:130 +0xf8
2024-03-19T14:09:50.7483527Z   github.com/sijms/go-ora/v2/network.newDataPacket()
2024-03-19T14:09:50.7484951Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/data_packet.go:30 +0xd9
2024-03-19T14:09:50.7486599Z   github.com/sijms/go-ora/v2/network.(*Session).WriteFinalPacket()
2024-03-19T14:09:50.7487970Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:600 +0x4e
2024-03-19T14:09:50.7489337Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
2024-03-19T14:09:50.7490907Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:204 +0x27e
2024-03-19T14:09:50.7492086Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
2024-03-19T14:09:50.7492998Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:211 +0x58
2024-03-19T14:09:50.7493489Z 
2024-03-19T14:09:50.7493677Z Previous write at 0x00c0008c9af8 by goroutine 358:
2024-03-19T14:09:50.7494284Z   crypto/cipher.newCBC()
2024-03-19T14:09:50.7494953Z       /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/cipher/cbc.go:31 +0x193
2024-03-19T14:09:50.7495912Z   crypto/cipher.NewCBCEncrypter()
2024-03-19T14:09:50.7496976Z       /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/cipher/cbc.go:56 +0xcc
2024-03-19T14:09:50.7498325Z   github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Init()
2024-03-19T14:09:50.7499532Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/security/general.go:98 +0x285
2024-03-19T14:09:50.7500441Z   github.com/sijms/go-ora/v2/network/security.NewOracleNetworkHash2()
2024-03-19T14:09:50.7502917Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/security/general.go:76 +0x34e
2024-03-19T14:09:50.7504522Z   github.com/sijms/go-ora/v2/advanced_nego.(*dataIntegrityService).activateAlgorithm()
2024-03-19T14:09:50.7506740Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/advanced_nego/data_integrity_service.go:145 +0x712
2024-03-19T14:09:50.7508751Z   github.com/sijms/go-ora/v2/advanced_nego.(*AdvNego).StartServices()
2024-03-19T14:09:50.7510636Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/advanced_nego/advanced_nego.go:244 +0x79
2024-03-19T14:09:50.7512043Z   github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
2024-03-19T14:09:50.7513729Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:448 +0xfab
2024-03-19T14:09:50.7515011Z   github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
2024-03-19T14:09:50.7516491Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
2024-03-19T14:09:50.7517513Z   database/sql.(*DB).conn()
2024-03-19T14:09:50.7518491Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
2024-03-19T14:09:50.7519405Z   database/sql.(*DB).prepare()
2024-03-19T14:09:50.7520385Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
2024-03-19T14:09:50.7521340Z   database/sql.(*DB).PrepareContext.func1()
2024-03-19T14:09:50.7522420Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
2024-03-19T14:09:50.7523731Z   database/sql.(*DB).retry()
2024-03-19T14:09:50.7524735Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
2024-03-19T14:09:50.7525656Z   database/sql.(*DB).PrepareContext()
2024-03-19T14:09:50.7526686Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
2024-03-19T14:09:50.7528016Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
2024-03-19T14:09:50.7529650Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
2024-03-19T14:09:50.7531152Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
2024-03-19T14:09:50.7532799Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
2024-03-19T14:09:50.7534365Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
2024-03-19T14:09:50.7535895Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
2024-03-19T14:09:50.7537264Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
2024-03-19T14:09:50.7538788Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
2024-03-19T14:09:50.7539981Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
2024-03-19T14:09:50.7541476Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
2024-03-19T14:09:50.7543006Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
2024-03-19T14:09:50.7544686Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
2024-03-19T14:09:50.7546199Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
2024-03-19T14:09:50.7547796Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
2024-03-19T14:09:50.7549240Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
2024-03-19T14:09:50.7550832Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
2024-03-19T14:09:50.7552241Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
2024-03-19T14:09:50.7553843Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
2024-03-19T14:09:50.7555428Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
2024-03-19T14:09:50.7557112Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
2024-03-19T14:09:50.7558423Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
2024-03-19T14:09:50.7559822Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
2024-03-19T14:09:50.7561039Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
2024-03-19T14:09:50.7562674Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
2024-03-19T14:09:50.7564032Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
2024-03-19T14:09:50.7565430Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
2024-03-19T14:09:50.7566953Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
2024-03-19T14:09:50.7568656Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
2024-03-19T14:09:50.7570207Z   github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
2024-03-19T14:09:50.7572027Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
2024-03-19T14:09:50.7573460Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
2024-03-19T14:09:50.7574973Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
2024-03-19T14:09:50.7576478Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
2024-03-19T14:09:50.7578098Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
2024-03-19T14:09:50.7579692Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
2024-03-19T14:09:50.7581341Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
2024-03-19T14:09:50.7582913Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
2024-03-19T14:09:50.7584618Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
2024-03-19T14:09:50.7585409Z 
2024-03-19T14:09:50.7585627Z Goroutine 383 (running) created at:
2024-03-19T14:09:50.7586601Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext()
2024-03-19T14:09:50.7588189Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:190 +0x324
2024-03-19T14:09:50.7589526Z   github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
2024-03-19T14:09:50.7591026Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:427 +0xd85
2024-03-19T14:09:50.7592275Z   github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
2024-03-19T14:09:50.7593736Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
2024-03-19T14:09:50.7594760Z   database/sql.(*DB).conn()
2024-03-19T14:09:50.7595754Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
2024-03-19T14:09:50.7596666Z   database/sql.(*DB).prepare()
2024-03-19T14:09:50.7597664Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
2024-03-19T14:09:50.7598611Z   database/sql.(*DB).PrepareContext.func1()
2024-03-19T14:09:50.7599714Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
2024-03-19T14:09:50.7600578Z   database/sql.(*DB).retry()
2024-03-19T14:09:50.7601553Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
2024-03-19T14:09:50.7602435Z   database/sql.(*DB).PrepareContext()
2024-03-19T14:09:50.7603611Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
2024-03-19T14:09:50.7604967Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
2024-03-19T14:09:50.7606584Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
2024-03-19T14:09:50.7608079Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
2024-03-19T14:09:50.7609752Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
2024-03-19T14:09:50.7611277Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
2024-03-19T14:09:50.7612884Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
2024-03-19T14:09:50.7614294Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
2024-03-19T14:09:50.7616090Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
2024-03-19T14:09:50.7617433Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
2024-03-19T14:09:50.7618966Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
2024-03-19T14:09:50.7620480Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
2024-03-19T14:09:50.7622136Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
2024-03-19T14:09:50.7623681Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
2024-03-19T14:09:50.7625246Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
2024-03-19T14:09:50.7626712Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
2024-03-19T14:09:50.7628564Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
2024-03-19T14:09:50.7629966Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
2024-03-19T14:09:50.7631598Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
2024-03-19T14:09:50.7633389Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
2024-03-19T14:09:50.7635100Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
2024-03-19T14:09:50.7636406Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
2024-03-19T14:09:50.7637814Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
2024-03-19T14:09:50.7639004Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
2024-03-19T14:09:50.7640386Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
2024-03-19T14:09:50.7641609Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
2024-03-19T14:09:50.7642951Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
2024-03-19T14:09:50.7644625Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
2024-03-19T14:09:50.7646314Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
2024-03-19T14:09:50.7647872Z   github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
2024-03-19T14:09:50.7649692Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
2024-03-19T14:09:50.7651146Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
2024-03-19T14:09:50.7652663Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
2024-03-19T14:09:50.7654128Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
2024-03-19T14:09:50.7655747Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
2024-03-19T14:09:50.7657142Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
2024-03-19T14:09:50.7658745Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
2024-03-19T14:09:50.7660310Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
2024-03-19T14:09:50.7662073Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
2024-03-19T14:09:50.7662891Z 
2024-03-19T14:09:50.7663100Z Goroutine 358 (running) created at:
2024-03-19T14:09:50.7664196Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
2024-03-19T14:09:50.7665880Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
2024-03-19T14:09:50.7667303Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
2024-03-19T14:09:50.7668884Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
2024-03-19T14:09:50.7670376Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
2024-03-19T14:09:50.7672201Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
2024-03-19T14:09:50.7673637Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).enqueue()
2024-03-19T14:09:50.7675244Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:452 +0x6f5
2024-03-19T14:09:50.7676702Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).Schedule()
2024-03-19T14:09:50.7678298Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:216 +0x244
2024-03-19T14:09:50.7679805Z   github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch()
2024-03-19T14:09:50.7681589Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:39 +0x12b
2024-03-19T14:09:50.7683381Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit()
2024-03-19T14:09:50.7685036Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:83 +0x23c
2024-03-19T14:09:50.7686564Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.4()
2024-03-19T14:09:50.7688169Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:181 +0x1d8
2024-03-19T14:09:50.7689833Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.6()
2024-03-19T14:09:50.7691424Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:257 +0x25c
2024-03-19T14:09:50.7692861Z   github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.(*Copier).Run()
2024-03-19T14:09:50.7694486Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/copier.go:73 +0x22e
2024-03-19T14:09:50.7696013Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.8()
2024-03-19T14:09:50.7697637Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:271 +0x5c
2024-03-19T14:09:50.7699048Z   github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
2024-03-19T14:09:50.7700639Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
2024-03-19T14:09:50.7701571Z ==================
2024-03-19T14:09:51.0038300Z ==================
2024-03-19T14:09:51.0038418Z WARNING: DATA RACE
2024-03-19T14:09:51.0038567Z Write at 0x00c0008e6020 by goroutine 383:
2024-03-19T14:09:51.0038712Z   runtime.slicecopy()
2024-03-19T14:09:51.0039060Z       /opt/hostedtoolcache/go/1.20.14/x64/src/runtime/slice.go:310 +0x0
2024-03-19T14:09:51.0039227Z   crypto/sha256.(*digest).Write()
2024-03-19T14:09:51.0039604Z       /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:198 +0x2ae
2024-03-19T14:09:51.0040004Z   github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Compute()
2024-03-19T14:09:51.0040609Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/security/general.go:133 +0x1aa
2024-03-19T14:09:51.0040879Z   github.com/sijms/go-ora/v2/network.newDataPacket()
2024-03-19T14:09:51.0041432Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/data_packet.go:30 +0xd9
2024-03-19T14:09:51.0041770Z   github.com/sijms/go-ora/v2/network.(*Session).WriteFinalPacket()
2024-03-19T14:09:51.0042313Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:600 +0x4e
2024-03-19T14:09:51.0042655Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
2024-03-19T14:09:51.0043500Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:204 +0x27e
2024-03-19T14:09:51.0043851Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
2024-03-19T14:09:51.0044388Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:211 +0x58
2024-03-19T14:09:51.0044394Z 
2024-03-19T14:09:51.0044594Z Previous write at 0x00c0008e6020 by goroutine 358:
2024-03-19T14:09:51.0044724Z   runtime.slicecopy()
2024-03-19T14:09:51.0045087Z       /opt/hostedtoolcache/go/1.20.14/x64/src/runtime/slice.go:310 +0x0
2024-03-19T14:09:51.0045222Z   crypto/sha256.(*digest).Write()
2024-03-19T14:09:51.0045617Z       /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:198 +0x2ae
2024-03-19T14:09:51.0045999Z   github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Compute()
2024-03-19T14:09:51.0046620Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/security/general.go:132 +0x154
2024-03-19T14:09:51.0046875Z   github.com/sijms/go-ora/v2/network.newDataPacket()
2024-03-19T14:09:51.0047444Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/data_packet.go:30 +0xd9
2024-03-19T14:09:51.0047860Z   github.com/sijms/go-ora/v2/network.(*Session).Write()
2024-03-19T14:09:51.0048424Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:658 +0x4af
2024-03-19T14:09:51.0048671Z   github.com/sijms/go-ora/v2.(*DataTypeNego).write()
2024-03-19T14:09:51.0049207Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/data_type_nego.go:544 +0x77a
2024-03-19T14:09:51.0049512Z   github.com/sijms/go-ora/v2.(*Connection).dataTypeNegotiation()
2024-03-19T14:09:51.0050044Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:1258 +0x1e4
2024-03-19T14:09:51.0050335Z   github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
2024-03-19T14:09:51.0050852Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:458 +0xfda
2024-03-19T14:09:51.0051126Z   github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
2024-03-19T14:09:51.0051639Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
2024-03-19T14:09:51.0051776Z   database/sql.(*DB).conn()
2024-03-19T14:09:51.0052162Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
2024-03-19T14:09:51.0052294Z   database/sql.(*DB).prepare()
2024-03-19T14:09:51.0052666Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
2024-03-19T14:09:51.0052844Z   database/sql.(*DB).PrepareContext.func1()
2024-03-19T14:09:51.0053208Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
2024-03-19T14:09:51.0053335Z   database/sql.(*DB).retry()
2024-03-19T14:09:51.0053699Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
2024-03-19T14:09:51.0053849Z   database/sql.(*DB).PrepareContext()
2024-03-19T14:09:51.0054232Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
2024-03-19T14:09:51.0054619Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
2024-03-19T14:09:51.0055138Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
2024-03-19T14:09:51.0055534Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
2024-03-19T14:09:51.0056047Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
2024-03-19T14:09:51.0056461Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
2024-03-19T14:09:51.0056922Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
2024-03-19T14:09:51.0057310Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
2024-03-19T14:09:51.0057787Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
2024-03-19T14:09:51.0058135Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
2024-03-19T14:09:51.0058769Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
2024-03-19T14:09:51.0059201Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
2024-03-19T14:09:51.0059698Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
2024-03-19T14:09:51.0060120Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
2024-03-19T14:09:51.0060572Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
2024-03-19T14:09:51.0061483Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
2024-03-19T14:09:51.0062391Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
2024-03-19T14:09:51.0063189Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
2024-03-19T14:09:51.0064078Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
2024-03-19T14:09:51.0064987Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
2024-03-19T14:09:51.0066088Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
2024-03-19T14:09:51.0066838Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
2024-03-19T14:09:51.0067666Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
2024-03-19T14:09:51.0068363Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
2024-03-19T14:09:51.0069153Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
2024-03-19T14:09:51.0069868Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
2024-03-19T14:09:51.0070652Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
2024-03-19T14:09:51.0071515Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
2024-03-19T14:09:51.0072471Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
2024-03-19T14:09:51.0073366Z   github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
2024-03-19T14:09:51.0074457Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
2024-03-19T14:09:51.0075290Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
2024-03-19T14:09:51.0076168Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
2024-03-19T14:09:51.0077012Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
2024-03-19T14:09:51.0077928Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
2024-03-19T14:09:51.0078714Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
2024-03-19T14:09:51.0079645Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
2024-03-19T14:09:51.0080533Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
2024-03-19T14:09:51.0081498Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
2024-03-19T14:09:51.0081983Z 
2024-03-19T14:09:51.0082125Z Goroutine 383 (running) created at:
2024-03-19T14:09:51.0082659Z   github.com/sijms/go-ora/v2/network.(*Session).StartContext()
2024-03-19T14:09:51.0083730Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/network/session.go:190 +0x324
2024-03-19T14:09:51.0084535Z   github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
2024-03-19T14:09:51.0085401Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:427 +0xd85
2024-03-19T14:09:51.0086126Z   github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
2024-03-19T14:09:51.0086977Z       /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.10/connection.go:133 +0x337
2024-03-19T14:09:51.0087581Z   database/sql.(*DB).conn()
2024-03-19T14:09:51.0088171Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
2024-03-19T14:09:51.0088887Z   database/sql.(*DB).prepare()
2024-03-19T14:09:51.0089467Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1587 +0x7b
2024-03-19T14:09:51.0090014Z   database/sql.(*DB).PrepareContext.func1()
2024-03-19T14:09:51.0090657Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1561 +0xa4
2024-03-19T14:09:51.0091172Z   database/sql.(*DB).retry()
2024-03-19T14:09:51.0091746Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
2024-03-19T14:09:51.0092296Z   database/sql.(*DB).PrepareContext()
2024-03-19T14:09:51.0092905Z       /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1560 +0xec
2024-03-19T14:09:51.0093822Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).get()
2024-03-19T14:09:51.0094863Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:113 +0x2a4
2024-03-19T14:09:51.0095729Z   github.com/cockroachdb/cdc-sink/internal/util/stmtcache.(*Cache[...]).Prepare()
2024-03-19T14:09:51.0096667Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stmtcache/stmtcache.go:84 +0x104
2024-03-19T14:09:51.0097702Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertBagsLocked()
2024-03-19T14:09:51.0098620Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:466 +0x58d
2024-03-19T14:09:51.0099432Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).upsertLocked()
2024-03-19T14:09:51.0100323Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:326 +0x30c
2024-03-19T14:09:51.0101092Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*apply).Apply()
2024-03-19T14:09:51.0101955Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/apply.go:224 +0x1069
2024-03-19T14:09:51.0102809Z   github.com/cockroachdb/cdc-sink/internal/target/apply.(*Acceptor).AcceptTableBatch()
2024-03-19T14:09:51.0103763Z       /home/runner/work/cdc-sink/cdc-sink/internal/target/apply/acceptor.go:48 +0x1e4
2024-03-19T14:09:51.0104622Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptTableBatch()
2024-03-19T14:09:51.0105514Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:84 +0xa5e
2024-03-19T14:09:51.0106350Z   github.com/cockroachdb/cdc-sink/internal/types.(*orderedAdapter).AcceptMultiBatch()
2024-03-19T14:09:51.0107234Z       /home/runner/work/cdc-sink/cdc-sink/internal/types/acceptors.go:136 +0xa60
2024-03-19T14:09:51.0108041Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
2024-03-19T14:09:51.0108959Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:155 +0x69a
2024-03-19T14:09:51.0109868Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
2024-03-19T14:09:51.0110826Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
2024-03-19T14:09:51.0111596Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
2024-03-19T14:09:51.0112407Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
2024-03-19T14:09:51.0113108Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
2024-03-19T14:09:51.0113890Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
2024-03-19T14:09:51.0114599Z   github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
2024-03-19T14:09:51.0115377Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
2024-03-19T14:09:51.0116242Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
2024-03-19T14:09:51.0117191Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
2024-03-19T14:09:51.0118075Z   github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
2024-03-19T14:09:51.0119105Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
2024-03-19T14:09:51.0120096Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
2024-03-19T14:09:51.0120987Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
2024-03-19T14:09:51.0121842Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
2024-03-19T14:09:51.0122754Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
2024-03-19T14:09:51.0123724Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
2024-03-19T14:09:51.0124643Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
2024-03-19T14:09:51.0125529Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
2024-03-19T14:09:51.0126504Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
2024-03-19T14:09:51.0126950Z 
2024-03-19T14:09:51.0127082Z Goroutine 358 (running) created at:
2024-03-19T14:09:51.0127733Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
2024-03-19T14:09:51.0128685Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
2024-03-19T14:09:51.0129670Z   github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
2024-03-19T14:09:51.0130581Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
2024-03-19T14:09:51.0131429Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
2024-03-19T14:09:51.0132320Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
2024-03-19T14:09:51.0133144Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).enqueue()
2024-03-19T14:09:51.0134019Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:452 +0x6f5
2024-03-19T14:09:51.0134828Z   github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).Schedule()
2024-03-19T14:09:51.0135754Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:216 +0x244
2024-03-19T14:09:51.0136641Z   github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch()
2024-03-19T14:09:51.0137658Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:39 +0x12b
2024-03-19T14:09:51.0138596Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit()
2024-03-19T14:09:51.0139542Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:83 +0x23c
2024-03-19T14:09:51.0140417Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.4()
2024-03-19T14:09:51.0141359Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:181 +0x1d8
2024-03-19T14:09:51.0142231Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.6()
2024-03-19T14:09:51.0143188Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:257 +0x25c
2024-03-19T14:09:51.0144020Z   github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.(*Copier).Run()
2024-03-19T14:09:51.0144981Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/copier.go:73 +0x22e
2024-03-19T14:09:51.0145879Z   github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.8()
2024-03-19T14:09:51.0146821Z       /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:271 +0x5c
2024-03-19T14:09:51.0147634Z   github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
2024-03-19T14:09:51.0148560Z       /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
2024-03-19T14:09:51.0149105Z ==================
sijms commented 6 months ago

would you please help me set up this test locally to accelerated the solution

bobvawter commented 6 months ago

I'm working on cutting our test suite down to a minimum-reproduction case that can be run in isolation.

sijms commented 6 months ago

another commit would you please test

bobvawter commented 6 months ago

I ran the test suite a few more times. Below are the distinct data-race conditions I saw with the latest commit.

Many of these traces pass through the cleanup helper in Session.StartContext(). Something that may be particular about our test case is that we have very fine-grained uses of nested Context objects to impose per-action timeouts through the application. There's quite a bit of code that ends with executing a database interaction and immediately canceling the relevant context.

func DoSomething(ctx context.Context) error {
  ctx, cancel := context.WithTimeout(ctx, config.ApplyTimeout)
  defer cancel()

  // This could wind up dialing a new connection, too.
  return sqlDbPool.ExecContext(ctx, sqlStatement, args)
}

This leads me to ask about the Session.doneContext slice. Does that represent the lifecycle of a single database interaction, let's say the ExecContext() call above?

func (session *Session) StartContext(ctx context.Context) {
go func(idone chan struct{}, mu *sync.Mutex) {
  select {
    case <-idone:
      return
    case <-ctx.Done():
      session.Disconnect()
  }
}()
}

I wonder if the cleanup helper is firing incorrectly if the helper's goroutine is slow to be scheduled (the test runners are CPU-bound) and both idone and ctx.Done() are closed. The specific case chosen is undefined if two channels are ready. Could that lead to session.Disconnect() being called on a Session that was returned to the pool and picked up by another goroutine?

==================
WARNING: DATA RACE
Write at 0x00c000a8a778 by goroutine 13167:
  github.com/sijms/go-ora/v2/network.(*Session).resetRead()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:172 +0x189
  github.com/sijms/go-ora/v2/network.(*Session).ResetBuffer()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:182 +0x178
  github.com/sijms/go-ora/v2/network.(*Session).Disconnect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:610 +0x30
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:208 +0x39e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:211 +0x58

Previous write at 0x00c000a8a778 by goroutine 13157:
  github.com/sijms/go-ora/v2/network.(*Session).readPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:1100 +0x1829
  github.com/sijms/go-ora/v2/network.(*Session).read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:789 +0xe9
  github.com/sijms/go-ora/v2/network.(*Session).GetInt64()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:1581 +0xf5
  github.com/sijms/go-ora/v2/advanced_nego.(*AdvNego).readHeader()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/advanced_nego/advanced_nego.go:54 +0x64
  github.com/sijms/go-ora/v2/advanced_nego.(*AdvNego).Read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/advanced_nego/advanced_nego.go:99 +0x4a
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:444 +0xf95
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 13167 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.202403[2303](https://github.com/cockroachdb/cdc-sink/actions/runs/8422905255/job/23063416801?pr=785#step:6:2304)1236-4984576d94fc/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 13157 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:407 +0x72e
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
==================
==================
WARNING: DATA RACE
Write at 0x00c000a8a7b8 by goroutine 13167:
  github.com/sijms/go-ora/v2/network.(*Session).resetRead()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:173 +0x1c7
  github.com/sijms/go-ora/v2/network.(*Session).ResetBuffer()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:182 +0x178
  github.com/sijms/go-ora/v2/network.(*Session).Disconnect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:610 +0x30
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:208 +0x39e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:211 +0x58

Previous write at 0x00c000a8a7b8 by goroutine 13157:
  github.com/sijms/go-ora/v2/network.(*Session).read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:878 +0x3bb
  github.com/sijms/go-ora/v2/network.(*Session).GetInt64()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:1581 +0xf5
  github.com/sijms/go-ora/v2/advanced_nego.(*AdvNego).readHeader()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/advanced_nego/advanced_nego.go:54 +0x64
  github.com/sijms/go-ora/v2/advanced_nego.(*AdvNego).Read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20[2403](https://github.com/cockroachdb/cdc-sink/actions/runs/8422905255/job/23063416801?pr=785#step:6:2404)23031236-4984576d94fc/advanced_nego/advanced_nego.go:99 +0x4a
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:444 +0xf95
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 13167 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 13157 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:407 +0x72e
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
==================
==================
WARNING: DATA RACE
Read at 0x00c000a8a708 by goroutine 13157:
  github.com/sijms/go-ora/v2/network.(*Session).initWrite()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:264 +0x1cc
  github.com/sijms/go-ora/v2/network.(*Session).writePacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:893 +0x2a8
  github.com/sijms/go-ora/v2/network.(*Session).Write()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:662 +0x4d5
  github.com/sijms/go-ora/v2/advanced_nego.(*AdvNego).Read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/advanced_nego/advanced_nego.go:214 +0x12fd
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:444 +0xf95
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Previous write at 0x00c000a8a708 by goroutine 13167:
  github.com/sijms/go-ora/v2/network.(*Session).Disconnect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:617 +0xe5
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:208 +0x39e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:211 +0x58

Goroutine 13157 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:407 +0x72e
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 13167 (finished) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
==================
==================
WARNING: DATA RACE
Read at 0x00c000229c08 by goroutine 13157:
  github.com/sijms/go-ora/v2/network.(*Session).initWrite()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:264 +0x1cc
  github.com/sijms/go-ora/v2/network.(*Session).writePacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:893 +0x2a8
  github.com/sijms/go-ora/v2/network.(*Session).Write()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:662 +0x4d5
  github.com/sijms/go-ora/v2.(*AuthObject).Write()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/auth_object.go:336 +0x3457
  github.com/sijms/go-ora/v2.(*Connection).doAuth()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:615 +0xe16
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:462 +0xff5
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:150 +0x4c6
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Previous write at 0x00c000229c08 by goroutine 14777:
  github.com/sijms/go-ora/v2/network.(*Session).Disconnect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:617 +0xe5
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:208 +0x39e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:211 +0x58

Goroutine 13157 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:407 +0x72e
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64

Goroutine 14777 (finished) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:150 +0x4c6
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
==================
==================
WARNING: DATA RACE
Write at 0x00c000464080 by goroutine 43885:
  crypto/sha256.(*digest).Reset()
      /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:124 +0x10a
  github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Compute()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/security/general.go:131 +0x11a
  github.com/sijms/go-ora/v2/network.newDataPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/data_packet.go:33 +0x152
  github.com/sijms/go-ora/v2/network.(*Session).WriteFinalPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:600 +0x4e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:204 +0x29e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:211 +0x58
Previous write at 0x00c000464080 by goroutine 43884:
  crypto/sha256.(*digest).Reset()
      /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:124 +0x10a
  github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Validate()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/security/general.go:166 +0x199
  github.com/sijms/go-ora/v2/network.newDataPacketFromData()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/data_packet.go:97 +0x78f
  github.com/sijms/go-ora/v2/network.(*Session).readPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:1089 +0x13bb
  github.com/sijms/go-ora/v2/network.(*Session).read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:789 +0xe9
  github.com/sijms/go-ora/v2/network.(*Session).GetByte()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:1553 +0x1eb
  github.com/sijms/go-ora/v2.newAuthObject()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/auth_object.go:52 +0x1f3
  github.com/sijms/go-ora/v2.(*Connection).doAuth()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:603 +0xa8f
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:462 +0xff5
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
Goroutine 43885 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
Goroutine 43884 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).enqueue()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:452 +0x6f5
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).Schedule()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:216 +0x244
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:39 +0x12b
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:83 +0x23c
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.4()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:181 +0x1d8
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.6()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:257 +0x25c
  github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.(*Copier).Run()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/copier.go:73 +0x22e
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.8()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:271 +0x5c
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
==================
==================
WARNING: DATA RACE
Write at 0x00c000464088 by goroutine 43885:
  crypto/sha256.(*digest).Reset()
      /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:126 +0x13c
  github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Compute()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/security/general.go:131 +0x11a
  github.com/sijms/go-ora/v2/network.newDataPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/data_packet.go:33 +0x152
  github.com/sijms/go-ora/v2/network.(*Session).WriteFinalPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:600 +0x4e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:204 +0x29e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:211 +0x58
Previous write at 0x00c000464088 by goroutine 43884:
  crypto/sha256.(*digest).Reset()
      /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:126 +0x13c
  github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Validate()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/security/general.go:166 +0x199
  github.com/sijms/go-ora/v2/network.newDataPacketFromData()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/data_packet.go:97 +0x78f
  github.com/sijms/go-ora/v2/network.(*Session).readPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:1089 +0x13bb
  github.com/sijms/go-ora/v2/network.(*Session).read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:789 +0xe9
  github.com/sijms/go-ora/v2/network.(*Session).GetByte()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:1553 +0x1eb
  github.com/sijms/go-ora/v2.newAuthObject()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/auth_object.go:52 +0x1f3
  github.com/sijms/go-ora/v2.(*Connection).doAuth()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:603 +0xa8f
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:462 +0xff5
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
Goroutine 43885 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
Goroutine 43884 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).enqueue()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:452 +0x6f5
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).Schedule()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:216 +0x244
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:39 +0x12b
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:83 +0x23c
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.4()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:181 +0x1d8
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.6()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:257 +0x25c
  github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.(*Copier).Run()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/copier.go:73 +0x22e
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.8()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:271 +0x5c
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
==================
==================
WARNING: DATA RACE
Write at 0x00c000464098 by goroutine 43885:
  crypto/sha256.(*digest).Reset()
      /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:130 +0x19c
  github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Compute()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/security/general.go:131 +0x11a
  github.com/sijms/go-ora/v2/network.newDataPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/data_packet.go:33 +0x152
  github.com/sijms/go-ora/v2/network.(*Session).WriteFinalPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:600 +0x4e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:204 +0x29e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:211 +0x58
Previous write at 0x00c000464098 by goroutine 43884:
  crypto/sha256.(*digest).Reset()
      /opt/hostedtoolcache/go/1.20.14/x64/src/crypto/sha256/sha256.go:130 +0x19c
  github.com/sijms/go-ora/v2/network/security.(*OracleNetworkHash2).Validate()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/security/general.go:166 +0x199
  github.com/sijms/go-ora/v2/network.newDataPacketFromData()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/data_packet.go:97 +0x78f
  github.com/sijms/go-ora/v2/network.(*Session).readPacket()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:1089 +0x13bb
  github.com/sijms/go-ora/v2/network.(*Session).read()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:789 +0xe9
  github.com/sijms/go-ora/v2/network.(*Session).GetByte()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:1553 +0x1eb
  github.com/sijms/go-ora/v2.newAuthObject()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/auth_object.go:52 +0x1f3
  github.com/sijms/go-ora/v2.(*Connection).doAuth()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:603 +0xa8f
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:462 +0xff5
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
Goroutine 43885 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
Goroutine 43884 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).enqueue()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:452 +0x6f5
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).Schedule()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:216 +0x244
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:39 +0x12b
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:83 +0x23c
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.4()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:181 +0x1d8
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.6()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:257 +0x25c
  github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.(*Copier).Run()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/copier.go:73 +0x22e
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.8()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:271 +0x5c
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
==================
==================
WARNING: DATA RACE
Write at 0x00c000164f00 by goroutine 43885:
  github.com/sijms/go-ora/v2/network.(*Session).ResetBuffer()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:180 +0x92
  github.com/sijms/go-ora/v2/network.(*Session).Disconnect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:610 +0x30
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func1()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:208 +0x39e
  github.com/sijms/go-ora/v2/network.(*Session).StartContext.func2()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:211 +0x58
Previous write at 0x00c000164f00 by goroutine 43884:
  github.com/sijms/go-ora/v2.(*Connection).readMsg()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:1113 +0x424
  github.com/sijms/go-ora/v2.newAuthObject()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/auth_object.go:141 +0x27b
  github.com/sijms/go-ora/v2.(*Connection).doAuth()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:603 +0xa8f
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:462 +0xff5
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
Goroutine 43885 (running) created at:
  github.com/sijms/go-ora/v2/network.(*Session).StartContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/network/session.go:190 +0x324
  github.com/sijms/go-ora/v2.(*Connection).OpenWithContext()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:427 +0xd85
  github.com/sijms/go-ora/v2.(*OracleConnector).Connect()
      /home/runner/go/pkg/mod/github.com/sijms/go-ora/v2@v2.8.11-0.20240323031236-4984576d94fc/connection.go:133 +0x337
  database/sql.(*DB).conn()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1387 +0xb0f
  database/sql.(*DB).begin()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1853 +0x70
  database/sql.(*DB).BeginTx.func1()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1836 +0x8a
  database/sql.(*DB).retry()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1538 +0x4c
  database/sql.(*DB).BeginTx()
      /opt/hostedtoolcache/go/1.20.14/x64/src/database/sql/sql.go:1835 +0xc9
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).tryCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:149 +0x446
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1.2()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:102 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:69 +0x58
  github.com/cockroachdb/cdc-sink/internal/util/retry.Loop()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:98 +0x221
  github.com/cockroachdb/cdc-sink/internal/util/retry.Retry()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/retry/retry.go:68 +0xbe
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:101 +0x213
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:45 +0x164
  github.com/cockroachdb/cdc-sink/internal/util/lockset.tryCall[...]()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:511 +0x141
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:352 +0x20d
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).worker()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:133 +0x161
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart.func2()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x64
Goroutine 43884 (running) created at:
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).maybeStart()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:114 +0x2c5
  github.com/cockroachdb/cdc-sink/internal/util/workgroup.(*Group).Go()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/workgroup/workgroup.go:79 +0x12f
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).dispose()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:416 +0x1fb
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).enqueue()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:452 +0x6f5
  github.com/cockroachdb/cdc-sink/internal/util/lockset.(*Set[...]).Schedule()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/lockset/lockset.go:216 +0x244
  github.com/cockroachdb/cdc-sink/internal/sequencer/scheduler.(*Scheduler).Batch()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/scheduler/scheduler.go:39 +0x12b
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*round).scheduleCommit()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/round.go:83 +0x23c
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.4()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:181 +0x1d8
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.6()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:257 +0x25c
  github.com/cockroachdb/cdc-sink/internal/sequencer/sequtil.(*Copier).Run()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/sequtil/copier.go:73 +0x22e
  github.com/cockroachdb/cdc-sink/internal/sequencer/core.(*Core).Start.func1.8()
      /home/runner/work/cdc-sink/cdc-sink/internal/sequencer/core/core.go:271 +0x5c
  github.com/cockroachdb/cdc-sink/internal/util/stopper.(*Context).Go.func1()
      /home/runner/work/cdc-sink/cdc-sink/internal/util/stopper/stopper.go:187 +0xb4
==================
sijms commented 6 months ago

hi @bobvawter when run test locally I get 

Error:          Received unexpected error:
                            ORA-01938: IDENTIFIED BY must be specified for CREATE USER

                            could not create user "src_11410_2"

I use oracle 12c with setting the following environmental variable:

export TEST_TARGET_CONNECT=oracle://sys:sys@localhost/db
export TEST_SOURCE_CONNECT =oracle://sys:sys@localhost/db
export TEST_STAGING_CONNECT=oracle://sys:sys@localhost/db
sijms commented 6 months ago

I bypass the above error still getting error related to the testing files use postgresql as default db

sijms commented 6 months ago

I should fix this issue otherwise I will return back to one thread model

sijms commented 6 months ago

last commit I fix regression occur in data integrety and test the code locally no DATA RACE would you please make another test

bobvawter commented 5 months ago

I'm still seeing races in the OracleNetworkHash2 type.

race-v2.8.11.log

sijms commented 5 months ago

would you please test again (last commit) I fix nearly all possible cause of data race I think now it is clean

sijms commented 5 months ago

I use the following code to produce the error

db, err := getDB()
if err != nil {
  t.Error(err)
  return
}
defer func() {
  err = db.Close()
  if err != nil {
    t.Error(err)
  }
}()
ctx, cancel := context.WithTimeout(context.Background(), 850*time.Millisecond)
defer cancel()
err = db.PingContext(ctx)
if err != nil {
  t.Error(err)
  return
}

by fine adjust the timeout I can but the interrupt after authentication, db version and getDatabaseTimeStamp all of them produce different types of data race

AndreyRomashchenko commented 5 months ago

Hi @sijms I'm not sure if this is the issue you're talking about, but there can be race conditions when the same slice is passed to multiple goroutines. To avoid this, I always create a copy of the options before invoking this function from goroutines.

// BuildJDBC create url from user, password and JDBC description string
func BuildJDBC(user, password, connStr string, options map[string]string) string {
    if options == nil {
        options = make(map[string]string)
    }
    options["connStr"] = connStr
    return BuildUrl("", 0, "", user, password, options)
}
sijms commented 5 months ago

the issue is old related to context (timeout and cancel). driver should stop the current operation once timeout occur or cancel called 1- previous solution using read/write SetDeadline: this partially solve the context timeout but not cancel 2- current solution using another go-routine (the source of data-race) work as watchdog and send signal to main routine to stop or cancel current operation