search

nikepan / clickhouse-bulk

Collects many small inserts to ClickHouse and send in big inserts
Apache License 2.0
474 stars 87 forks source link

collector: fix data race #9

Closed lenstr closed 5 years ago

lenstr commented 5 years ago

Hello and thank you for the great project!

Just found that Collector.Push is not "goroutine safe". You can check it by passing -race flag to the go test command:

> go test -race .
2019/02/18 17:41:29 send 1 rows to http://127.0.0.1:8124 of 
2019/02/18 17:41:29 send 1 rows to http://127.0.0.1:8125 of 
2019/02/18 17:41:29 send 1 rows to http://127.0.0.1:8123 of 
==================
WARNING: DATA RACE
Read at 0x00c000184810 by goroutine 39:
  runtime.mapaccess2_faststr()
      /home/lenstr/sdk/go1.11.5/src/runtime/map_faststr.go:101 +0x0
  github.com/lenstr/clickhouse-bulk.(*Collector).Push()
      /home/lenstr/Projects/clickhouse-bulk/collector.go:149 +0x6f

Previous write at 0x00c000184810 by goroutine 37:
  runtime.mapassign_faststr()
      /home/lenstr/sdk/go1.11.5/src/runtime/map_faststr.go:190 +0x0
  github.com/lenstr/clickhouse-bulk.(*Collector).AddTable()
      /home/lenstr/Projects/clickhouse-bulk/collector.go:143 +0x1c2
  github.com/lenstr/clickhouse-bulk.(*Collector).Push()
      /home/lenstr/Projects/clickhouse-bulk/collector.go:152 +0x125
...
codecov-io commented 5 years ago

Codecov Report

Merging #9 into master will increase coverage by 0.44%. The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master       #9      +/-   ##
==========================================
+ Coverage    87.2%   87.64%   +0.44%     
==========================================
  Files           6        6              
  Lines         336      348      +12     
==========================================
+ Hits          293      305      +12     
  Misses         32       32              
  Partials       11       11
Impacted Files Coverage Δ
collector.go 100% <100%> (ø) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 2d2164e...7b93c4b. Read the comment docs.