Segmentation fault when writing csv

slumbi commented 1 week ago

When dumping a large table from postgres to clickhouse using sling 1.2.19, I bump into the following error:

Sling version (sling --version): 1.2.19

Operating System (linux, mac, windows): linux

Logs:

2024-09-24 16:02:59 INF streaming data
2024-09-24 16:02:59 DBG use `dbt_akos`
unexpected fault address 0xc003000000
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0xc003000000 pc=0x562f72]

goroutine 307 gp=0xc00154b180 m=11 mp=0xc000600808 [running]:
runtime.throw({0x34ec85e?, 0x5?})
        /github/sling-cli/_work/_tool/go/1.22.6/x64/src/runtime/panic.go:1023 +0x5c fp=0xc002189d60 sp=0xc002189d30 pc=0x448ddc
runtime.sigpanic()
        /github/sling-cli/_work/_tool/go/1.22.6/x64/src/runtime/signal_unix.go:895 +0x285 fp=0xc002189dc0 sp=0xc002189d60 pc=0x461965
strings.IndexAny({0xc002d1e691?, 0xc002d1e689?}, {0x34eafa6?, 0x0?})
        /github/sling-cli/_work/_tool/go/1.22.6/x64/src/strings/strings.go:161 +0x172 fp=0xc002189e38 sp=0xc002189dc0 pc=0x562f72
github.com/flarco/g/csv.(*Writer).Write(0xc0018eca68, {0xc001d8f500?, 0x17, 0xc001d8f500?})
        /home/larco/go/pkg/mod/github.com/flarco/g@v0.1.119/csv/writer.go:84 +0x334 fp=0xc002189ee0 sp=0xc002189e38 pc=0xa70254
github.com/slingdata-io/sling-cli/core/dbio/iop.(*Datastream).writeBwCsv(0xc001691ba0, {0xc001d8f500?, 0x17?, 0x17?})
        /github/sling-cli/_work/sling-cli/sling-cli/core/dbio/iop/datastream.go:318 +0x27 fp=0xc002189f10 sp=0xc002189ee0 pc=0x11ad7c7
github.com/slingdata-io/sling-cli/core/dbio/iop.(*Datastream).processBwRows.func1()
        /github/sling-cli/_work/sling-cli/sling-cli/core/dbio/iop/datastream.go:225 +0xc5 fp=0xc002189f80 sp=0xc002189f10 pc=0x11acd45
github.com/slingdata-io/sling-cli/core/dbio/iop.(*Datastream).processBwRows(0xc001691ba0)
        /github/sling-cli/_work/sling-cli/sling-cli/core/dbio/iop/datastream.go:234 +0x91 fp=0xc002189fc8 sp=0xc002189f80 pc=0x11acc51
github.com/slingdata-io/sling-cli/core/dbio/iop.(*Datastream).Start.gowrap1()
        /github/sling-cli/_work/sling-cli/sling-cli/core/dbio/iop/datastream.go:819 +0x25 fp=0xc002189fe0 sp=0xc002189fc8 pc=0x11b3505
runtime.goexit({})
        /github/sling-cli/_work/_tool/go/1.22.6/x64/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc002189fe8 sp=0xc002189fe0 pc=0x483f61
created by github.com/slingdata-io/sling-cli/core/dbio/iop.(*Datastream).Start in goroutine 286
        /github/sling-cli/_work/sling-cli/sling-cli/core/dbio/iop/datastream.go:819 +0x1445

flarco commented 6 days ago

This is something that has been an issue for a while. It has to do with clickhouse's decimal value. Try with env var SLING_PROCESS_BW=false.

slumbi commented 5 days ago

It worked, thanks!

flarco / g

Segmentation fault when writing csv #1