search

housepower / clickhouse_sinker

Easily load data from kafka to ClickHouse
https://housepower.github.io/clickhouse_sinker
Apache License 2.0
515 stars 118 forks source link

panic: reflect: call of reflect.Value.Interface on zero Value #103

Closed fzyzcjy closed 3 years ago

fzyzcjy commented 3 years ago

logs:

    /usr/local/go/src/reflect/value.go:1016
github.com/ClickHouse/clickhouse-go/lib/data.(*Block).AppendRow(0xc000176320, 0xc00024c150, 0x7, 0x7, 0xc00001e001, 0xc00024c150)
    /root/go/pkg/mod/github.com/!click!house/clickhouse-go@v1.4.3/lib/data/block.go:138 +0x2bd
github.com/ClickHouse/clickhouse-go.(*stmt).execContext(0xc0001e6060, 0x11252c0, 0xc00003e018, 0xc00024c150, 0x7, 0x7, 0x176c090, 0x0, 0x0, 0x7f68eed40028)
    /root/go/pkg/mod/github.com/!click!house/clickhouse-go@v1.4.3/stmt.go:47 +0x65
github.com/ClickHouse/clickhouse-go.(*stmt).ExecContext(0xc0001e6060, 0x11252c0, 0xc00003e018, 0xc000120480, 0x7, 0x7, 0x10, 0xe3b700, 0x40da01, 0xc00023a040)
    /root/go/pkg/mod/github.com/!click!house/clickhouse-go@v1.4.3/stmt.go:41 +0x145
database/sql.ctxDriverStmtExec(0x11252c0, 0xc00003e018, 0x1125480, 0xc0001e6060, 0xc000120480, 0x7, 0x7, 0x7, 0x7, 0x0, ...)
    /usr/local/go/src/database/sql/ctxutil.go:65 +0x216
database/sql.resultFromStatement(0x11252c0, 0xc00003e018, 0x1121980, 0xc00016ee10, 0xc0000a2740, 0xc00032a100, 0x7, 0x8, 0x0, 0x0, ...)
    /usr/local/go/src/database/sql/sql.go:2519 +0x150
database/sql.(*Stmt).ExecContext(0xc0003e8090, 0x11252c0, 0xc00003e018, 0xc00032a100, 0x7, 0x8, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/database/sql/sql.go:2495 +0x205
database/sql.(*Stmt).Exec(...)
    /usr/local/go/src/database/sql/sql.go:2507
github.com/housepower/clickhouse_sinker/output.(*ClickHouse).write(0xc00007b220, 0xc0002a61c0, 0x0, 0x0)
    /root/go/src/github.com/housepower/clickhouse_sinker/output/clickhouse.go:100 +0x32d
github.com/housepower/clickhouse_sinker/output.(*ClickHouse).loopWrite(0xc00007b220, 0xc0002a61c0, 0xfb1860)
    /root/go/src/github.com/housepower/clickhouse_sinker/output/clickhouse.go:139 +0x5b
github.com/housepower/clickhouse_sinker/output.(*ClickHouse).Send.func1()
    /root/go/src/github.com/housepower/clickhouse_sinker/output/clickhouse.go:76 +0x45
github.com/housepower/clickhouse_sinker/util.(*WorkerPool).wokerFunc(0xc0001886c0)
    /root/go/src/github.com/housepower/clickhouse_sinker/util/workerpool.go:56 +0xae
created by github.com/housepower/clickhouse_sinker/util.(*WorkerPool).start
    /root/go/src/github.com/housepower/clickhouse_sinker/util/workerpool.go:76 +0x45

My schema:

DROP TABLE IF EXISTS frontend_event;;
CREATE TABLE frontend_event
(
    `UserId`         UInt64,
    `EventTime`      DateTime,
    `Action`         LowCardinality(String),
    `ArgIntName`     Array(String),
    `ArgIntValue`    Array(Int64),
    `ArgStringName`  Array(String),
    `ArgStringValue` Array(String)
)
    ENGINE = MergeTree()
        PARTITION BY toYYYYMM(EventTime)
        ORDER BY (EventTime, intHash32(UserId))
        SAMPLE BY intHash32(UserId)

my data:

{"UserId":12345,"EventTime":"2020-12-18T03:38:39.000Z","Action":"APP_START","ArgIntName":["AA","BB"],"ArgIntValue":[1000,-2000],"ArgStringName":[],"ArgStringValue":[]}

if formatted, it looks like:


{
   "UserId": 12345,
   "EventTime": "2020-12-18T03:38:39.000Z",
   "Action": "APP_START",
   "ArgIntName": [
      "AA",
      "BB"
   ],
   "ArgIntValue": [
      1000,
      -2000
   ],
   "ArgStringName": [],
   "ArgStringValue": []
}
yuzhichang commented 3 years ago

I have reproduced the issue.

yuzhichang commented 3 years ago

fixed by master 318f21c