apache / arrow-adbc

Database connectivity API standard and libraries for Apache Arrow
https://arrow.apache.org/adbc/
Apache License 2.0
385 stars 98 forks source link

go/adbc/driver/snowflake: Panic when running `show warehouses` with Snowflake driver #863

Closed jonashaag closed 1 year ago

jonashaag commented 1 year ago

While testing https://github.com/pola-rs/polars/pull/9600

cur.execute("show warehouses")
Snowflake driver panicked, stack traces:
goroutine 17 [running, locked to thread]:
main.poison(0x0?, {0x127697e8c, 0x19}, {0x127a6e3e0, 0x14000038600})
    /Users/runner/work/arrow-adbc/arrow-adbc/adbc/go/adbc/pkg/snowflake/driver.go:97 +0xa8
main.SnowflakeStatementExecuteQuery.func1()
    /Users/runner/work/arrow-adbc/arrow-adbc/adbc/go/adbc/pkg/snowflake/driver.go:691 +0x58
panic({0x127a6e3e0, 0x14000038600})
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/runtime/panic.go:838 +0x20c
github.com/apache/arrow-adbc/go/adbc/driver/snowflake.newRecordReader({0x127b03bf0, 0x14000122000}, {0x127b01340?, 0x14000486140}, {0x127b045c8, 0x140004d6240}, 0x0)
    /Users/runner/work/arrow-adbc/arrow-adbc/adbc/go/adbc/driver/snowflake/record_reader.go:249 +0x7a8
github.com/apache/arrow-adbc/go/adbc/driver/snowflake.(*statement).ExecuteQuery(0x14000458700, {0x127b03bf0, 0x14000122000})
    /Users/runner/work/arrow-adbc/arrow-adbc/adbc/go/adbc/driver/snowflake/statement.go:439 +0x130
main.SnowflakeStatementExecuteQuery(0x1400010e000?, 0x12b121380, 0x16b8f0328, 0x16b8f0330)
    /Users/runner/work/arrow-adbc/arrow-adbc/adbc/go/adbc/pkg/snowflake/driver.go:709 +0xa4

goroutine 72 [IO wait]:
internal/poll.runtime_pollWait(0x1261a4d98, 0x72)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/runtime/netpoll.go:302 +0xa4
internal/poll.(*pollDesc).wait(0x140007ae100?, 0x14000738000?, 0x0)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/internal/poll/fd_poll_runtime.go:83 +0x2c
internal/poll.(*pollDesc).waitRead(...)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Read(0x140007ae100, {0x14000738000, 0x15e7, 0x15e7})
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/internal/poll/fd_unix.go:167 +0x1ec
net.(*netFD).Read(0x140007ae100, {0x14000738000?, 0x1400072a198?, 0x8?})
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/net/fd_posix.go:55 +0x2c
net.(*conn).Read(0x14000010028, {0x14000738000?, 0x140004ce7c8?, 0x126c434b8?})
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/net/net.go:183 +0x38
crypto/tls.(*atLeastReader).Read(0x140006e0150, {0x14000738000?, 0x0?, 0x126c39518?})
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/crypto/tls/conn.go:785 +0x44
bytes.(*Buffer).ReadFrom(0x1400072a278, {0x127af34e0, 0x140006e0150})
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/bytes/buffer.go:204 +0x94
crypto/tls.(*Conn).readFromUntil(0x1400072a000, {0x1261a4e88?, 0x14000010028}, 0xa0000000000009b?)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/crypto/tls/conn.go:807 +0xd8
crypto/tls.(*Conn).readRecordOrCCS(0x1400072a000, 0x0)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/crypto/tls/conn.go:614 +0xdc
crypto/tls.(*Conn).readRecord(...)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/crypto/tls/conn.go:582
crypto/tls.(*Conn).Read(0x1400072a000, {0x1400076a000, 0x1000, 0x0?})
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/crypto/tls/conn.go:1285 +0x180
net/http.(*persistConn).Read(0x14000722000, {0x1400076a000?, 0x126c345a0?, 0x140005302a0?})
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/net/http/transport.go:1929 +0x54
bufio.(*Reader).fill(0x140007356e0)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/bufio/bufio.go:106 +0x100
bufio.(*Reader).Peek(0x140007356e0, 0x1)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/bufio/bufio.go:144 +0x6c
net/http.(*persistConn).readLoop(0x14000722000)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/net/http/transport.go:2093 +0x160
created by net/http.(*Transport).dialConn
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/net/http/transport.go:1750 +0x11e0

goroutine 73 [select]:
net/http.(*persistConn).writeLoop(0x14000722000)
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/net/http/transport.go:2392 +0xa4
created by net/http.(*Transport).dialConn
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/net/http/transport.go:1751 +0x122c

goroutine 26 [select, 2 minutes]:
database/sql.(*DB).connectionOpener(0x140004ea9c0, {0x127b03bb8, 0x1400061e1c0})
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/database/sql/sql.go:1226 +0x84
created by database/sql.OpenDB
    /Users/runner/hostedtoolcache/go/1.18.10/x64/src/database/sql/sql.go:794 +0x188

InternalError: ADBC_STATUS_INTERNAL (9): [Snowflake] AdbcStatementExecuteQuery: Go panic in Snowflake driver (see stderr): runtime.boundsError{x:0, y:0, signed:true, code:0x0}
lidavidm commented 1 year ago

Thanks for the report! It looks like that line doesn't check a slice's length before indexing it

lidavidm commented 1 year ago

Hmm, weird. Snowflake returns no Arrow batches for SHOW WAREHOUSES but also claims the result set is non-empty...

lidavidm commented 1 year ago

So indeed, it seems Snowflake doesn't support getting Arrow data for certain queries.

(dlv) p scd
*github.com/snowflakedb/gosnowflake.snowflakeArrowStreamChunkDownloader {
    sc: *github.com/snowflakedb/gosnowflake.snowflakeConn {
        ctx: context.Context(*context.valueCtx) ...,
        cfg: *(*"github.com/snowflakedb/gosnowflake.Config")(0xc0004a88c0),
        rest: *(*"github.com/snowflakedb/gosnowflake.snowflakeRestful")(0xc000a72180),
        SequenceCounter: 1,
        QueryID: "01ad4ad4-0001-2b93-0001-8206045595ce",
        SQLState: "",
        telemetry: *(*"github.com/snowflakedb/gosnowflake.snowflakeTelemetry")(0xc0000e6f40),
        internal: github.com/snowflakedb/gosnowflake.InternalClient(*github.com/snowflakedb/gosnowflake.httpClient) ...,},
    ChunkMetas: []github.com/snowflakedb/gosnowflake.execResponseChunk len: 0, cap: 0, nil,
    Total: 12,
    Qrmk: "",
    ChunkHeader: map[string]string nil,
    FuncGet: github.com/snowflakedb/gosnowflake.getChunk,
    RowSet: github.com/snowflakedb/gosnowflake.rowSetType {
            ...
lidavidm commented 1 year ago

864 will prevent it from panicking at least, but I think we'll have to bring this upstream.

jonashaag commented 1 year ago

upstream

You mean upstream = Snowflake? Good luck with that...

zeroshade commented 1 year ago

I'll see if I can find an engineer here at snowflake summit that I can bug to figure out what's up with this. :smile: