search

pingcap / tidb-lightning

This repository has been moved to https://github.com/pingcap/br
Apache License 2.0
143 stars 66 forks source link

panic in tidb backend in strict sql-mode #550

Open glorv opened 3 years ago

glorv commented 3 years ago

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do? If possible, provide a recipe for reproducing the error. Use lightning tidb backend to import data with config: 
    [tidb]
sql-mode = "STRICT_ALL_TABLES"

    panic backtrace:

    goroutine 578 [running]:
github.com/pingcap/tidb/types.(*Datum).ConvertTo(0xc00a0fd220, 0xc0001dcdc0, 0xc0004a0be8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20201126095401-55c106afb89b/types/datum.go:843 +0xcff
github.com/pingcap/tidb/table.CastValue(0x2b48760, 0xc000c6e000, 0x5, 0x0, 0x25442e2, 0xb, 0xc00cb05371, 0x8, 0x8, 0x0, ...)
/go/pkg/mod/github.com/pingcap/tidb@v1.1.0-beta.0.20201126095401-55c106afb89b/table/column.go:244 +0xf2
github.com/pingcap/tidb-lightning/lightning/backend.(*tidbEncoder).appendSQL(0xc00810e040, 0xc000c22120, 0xc00a0fd660, 0xc008097770, 0x0, 0x0)
/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb-lightning/lightning/backend/tidb.go:181 +0x594
github.com/pingcap/tidb-lightning/lightning/backend.(*tidbEncoder).Encode(0xc00810e040, 0xc019f0a3c0, 0xc00062a480, 0xa, 0x10, 0x1, 0xc00a0ec060, 0xa, 0xb, 0x1f37685, ...)
/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb-lightning/lightning/backend/tidb.go:251 +0x32d
github.com/pingcap/tidb-lightning/lightning/restore.(*chunkRestore).encodeLoop(0xc000c46040, 0x2b02ec0, 0xc008097680, 0xc019f0a360, 0xc008063aa0, 0xc019f0a3c0, 0x2adba00, 0xc00810e040, 0xc00a0ec000, 0xc0192fc000, ...)
/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:1902 +0x350
github.com/pingcap/tidb-lightning/lightning/restore.(*chunkRestore).restore(0xc000c46040, 0x2b02ec0, 0xc008097680, 0xc008063aa0, 0x0, 0xc000170e00, 0xc000bb2180, 0xc0192fc000, 0x0, 0x0)
/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:1976 +0x7a4
github.com/pingcap/tidb-lightning/lightning/restore.(*TableRestore).restoreEngine.func1(0xc00c3f8fe0, 0xc0192fc000, 0x2b02ec0, 0xc008097680, 0xc008063aa0, 0xc000000000, 0xc000170e00, 0xc000bb2180, 0xc000c46020, 0xc014f65060, ...)
/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:1086 +0x175
created by github.com/pingcap/tidb-lightning/lightning/restore.(*TableRestore).restoreEngine
/home/jenkins/agent/workspace/optimization-build-tidb-linux-amd/go/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:1078 +0x64c
panic: should never happen

    Root Cause: The tidb backend FetchRemoteTableModels implementation is not accurate, it only set Flag in The FieldType and ignore other fields. So when run tidb backend with strict sql-mode, the table.CastValue panic because the fieldtype.Tp is 0(undefined).