search

pingcap / br

A command-line tool for distributed backup and restoration of the TiDB cluster data
https://pingcap.com/docs/dev/how-to/maintain/backup-and-restore/br/
Apache License 2.0
124 stars 102 forks source link

Go1.16 compilation failed #1034

Closed CadmusJiang closed 1 year ago

CadmusJiang commented 3 years ago

The PingCAP company is upgrading the go version from 1.13 to 1.16. Is this a compilation error caused by go version? go1.13 image go1.15 image go1.16 image

glorv commented 3 years ago

@CadmusJiang Seems some unit tests were failed. Could you please add more info about which tests were failed? I test locally with the newest master branch and go 1.16.3, the result is ok.

YuJuncen commented 3 years ago
image

It works with Darwin/arm64. If possible, could you upload the full log? (There might be an unstable unit test.)

CadmusJiang commented 3 years ago

I make again, one success and one failure.It seems that there is something wrong with the lightning test. failure

cp go.mod1 go.mod && cp go.sum1 go.sum
CGO_ENABLED=1 GO111MODULE=on go build -trimpath -ldflags '-X "github.com/pingcap/br/pkg/version/build.ReleaseVersion=v5.0.0-master-dirty" -X "github.com/pingcap/br/pkg/version/build.BuildTS=2021-04-19 06:12:46" -X "github.com/pingcap/br/pkg/version/build.GitHash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a" -X "github.com/pingcap/br/pkg/version/build.GitBranch=master"'  -o bin/br cmd/br/*.go
cp go.mod1 go.mod && cp go.sum1 go.sum
CGO_ENABLED=1 GO111MODULE=on go build -trimpath -ldflags '-X "github.com/pingcap/br/pkg/version/build.ReleaseVersion=v5.0.0-master-dirty" -X "github.com/pingcap/br/pkg/version/build.BuildTS=2021-04-19 06:12:46" -X "github.com/pingcap/br/pkg/version/build.GitHash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a" -X "github.com/pingcap/br/pkg/version/build.GitBranch=master"'  -o bin/tidb-lightning cmd/tidb-lightning/main.go
cp go.mod1 go.mod && cp go.sum1 go.sum
CGO_ENABLED=1 GO111MODULE=on go build -trimpath -ldflags '-X "github.com/pingcap/br/pkg/version/build.ReleaseVersion=v5.0.0-master-dirty" -X "github.com/pingcap/br/pkg/version/build.BuildTS=2021-04-19 06:12:46" -X "github.com/pingcap/br/pkg/version/build.GitHash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a" -X "github.com/pingcap/br/pkg/version/build.GitBranch=master"'  -o bin/tidb-lightning-ctl cmd/tidb-lightning-ctl/main.go
make[1]: 进入目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br”
go mod tidy
cp go.mod1 go.mod && cp go.sum1 go.sum
GO111MODULE=on go mod tidy
cp go.mod go.mod1 && cp go.sum go.sum1
cd tests && GO111MODULE=on go mod tidy
git diff --exit-code go.mod1 go.sum1 tools/go.mod tools/go.sum
make[1]: 离开目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br”
make[1]: 进入目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br”
install tools...
make[2]: 进入目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/tools”
make[2]: 对“all”无需做任何事。
make[2]: 离开目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/tools”
generator errors.toml
./tools/check-errdoc.sh
Generate successfully, output:
Non-normalized error: proto: negative length found during unmarshaling
Non-normalized error: proto: integer overflow
Non-normalized error: proto: unexpected end of group
Non-normalized error: insert statement not found

cp go.mod1 go.mod && cp go.sum1 go.sum
tools/bin/gofumports -w -d -format-only -local github.com/pingcap/br $(find . -name '*.go' -type f | grep -vE 'vendor|\.pb\.go|mock|tools|res_vfsdata') 2>&1 | awk '{ print } END { if (NR > 0) { exit 1 } }'
# TODO: go vet lightning packages too.
tools/bin/govet --shadow $(go list ./... | grep -vE 'vendor|tools' | sed 's/github.com\/pingcap\/br\/*//' | grep -v "lightning") 2>&1 | awk '{ print } END { if (NR > 0) { exit 1 } }'
# TODO: lint lightning packages too.
CGO_ENABLED=0 tools/bin/golangci-lint run --enable-all --deadline 120s \
    --disable gochecknoglobals \
    --disable goimports \
    --disable gofmt \
    --disable gci \
    --disable wsl \
    --disable funlen \
    --disable gocognit \
    --disable godox \
    --disable gomnd \
    --disable testpackage \
    --disable nestif \
    --disable goerr113 \
    --disable lll \
    --disable paralleltest \
    --disable nlreturn \
    --disable exhaustivestruct \
    --disable exhaustive \
    --disable godot \
    --disable gosec \
    --disable errorlint \
    --disable wrapcheck \
    go list ./... | grep -vE 'vendor|tools' | sed 's/github.com\/pingcap\/br\/*//'
ERRO [linters context] typechecking error: stat /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/go: directory not found
ERRO [linters context] typechecking error: stat /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/list: directory not found
WARN [runner] Can't run linter goanalysis_metalinter: bodyclose: failed prerequisites: [buildssa@./go: analysis skipped: errors in package: [-: stat /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/go: directory not found]]
WARN [runner] Can't run linter unused: buildir: analysis skipped: errors in package: [-: stat /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/go: directory not found]
ERRO Running error: buildir: analysis skipped: errors in package: [-: stat /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/go: directory not found]
# pingcap/errors APIs are mixed with multiple patterns 'pkg/errors',
# 'juju/errors' and 'pingcap/parser'. To avoid confusion and mistake,
# we only allow a subset of APIs, that's "Normalize|Annotate|Trace|Cause|Find".
# TODO: check lightning packages.
grep -Rn --include="*.go" --exclude="*_test.go" -E "(\t| )errors\.[A-Z]" \
    $(go list ./... | grep -vE 'vendor|tools' | sed 's/github.com\/pingcap\/br\/*//' | grep -vE "tests|lightning") | \
    grep -vE "Normalize|Annotate|Trace|Cause|RedactLogEnabled|Find" 2>&1 | awk '{ print } END { if (NR > 0) { exit 1 } }'
# The package name of "github.com/pingcap/kvproto/pkg/backup" collides
# "github.com/pingcap/br/pkg/backup", so we rename kvproto to backuppb.
grep -Rn --include="*.go" -E '"github.com/pingcap/kvproto/pkg/backup"' \
    $(go list ./... | grep -vE 'vendor|tools' | sed 's/github.com\/pingcap\/br\/*//') | \
    grep -vE "backuppb" | awk '{ print } END { if (NR > 0) { exit 1 } }'
linting
# TODO: lint lightning packages.
CGO_ENABLED=0 tools/bin/revive -formatter friendly -config revive.toml $(go list ./... | grep -vE 'vendor|tools' | grep -v "lightning")
  ⚠  https://revive.run/r#flag-parameter  parameter 'isLast' seems to be a control flag, avoid control coupling
  /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/trace/tracing.go:66:13

  ⚠  https://revive.run/r#flag-parameter  parameter 'checkRequirements' seems to be a control flag, avoid control coupling
  /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/conn/conn.go:171:12

  ⚠  https://revive.run/r#flag-parameter  parameter 'needDomain' seems to be a control flag, avoid control coupling
  /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/conn/conn.go:171:12

  ⚠  https://revive.run/r#flag-parameter  parameter 'skipChecksum' seems to be a control flag, avoid control coupling
  /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/backup/schema.go:68:33

  ⚠  https://revive.run/r#flag-parameter  parameter 'useTLS' seems to be a control flag, avoid control coupling
  /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/task/common.go:515:20

  ⚠  https://revive.run/r#flag-parameter  parameter 'useTLS' seems to be a control flag, avoid control coupling
  /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/task/common.go:515:20

  ⚠  https://revive.run/r#confusing-naming  Method 'newS3Storage' differs only by capitalization to function 'NewS3Storage' in the same source file
  /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/storage/s3.go:236:6

  ⚠  https://revive.run/r#confusing-naming  Method 'open' differs only by capitalization to method 'Open' in the same source file
  /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/storage/s3.go:474:22

  ⚠  https://revive.run/r#flag-parameter  parameter 'enable' seems to be a control flag, avoid control coupling
  /root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/restore/client.go:1072:25

⚠ 9 problems (0 errors, 9 warnings)

Warnings:
  7  flag-parameter
  2  confusing-naming

make[1]: 离开目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br”
cp go.mod1 go.mod && cp go.sum1 go.sum
make[1]: 进入目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br”
install tools...
make[2]: 进入目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/tools”
make[2]: 对“all”无需做任何事。
make[2]: 离开目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/tools”
tools/bin/failpoint-ctl enable
make[1]: 离开目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br”
CGO_ENABLED=1 GO111MODULE=on go test -ldflags '-X "github.com/pingcap/br/pkg/version/build.ReleaseVersion=v5.0.0-master-dirty" -X "github.com/pingcap/br/pkg/version/build.BuildTS=2021-04-19 06:12:46" -X "github.com/pingcap/br/pkg/version/build.GitHash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a" -X "github.com/pingcap/br/pkg/version/build.GitBranch=master"'  -tags leak $(go list ./... | grep -vE 'vendor|tools') || ( make failpoint-disable && exit 1 )
ok      github.com/pingcap/br/cmd/br    0.078s
ok      github.com/pingcap/br/cmd/tidb-lightning    0.155s
ok      github.com/pingcap/br/cmd/tidb-lightning-ctl    0.142s
ok      github.com/pingcap/br/pkg/backup    0.393s
ok      github.com/pingcap/br/pkg/cdclog    (cached)
ok      github.com/pingcap/br/pkg/checksum  0.387s
ok      github.com/pingcap/br/pkg/conn  (cached)
?       github.com/pingcap/br/pkg/errors    [no test files]
?       github.com/pingcap/br/pkg/glue  [no test files]
?       github.com/pingcap/br/pkg/gluetidb  [no test files]
ok      github.com/pingcap/br/pkg/gluetikv  0.098s
?       github.com/pingcap/br/pkg/httputil  [no test files]
ok      github.com/pingcap/br/pkg/kv    (cached)
[2021/04/19 14:14:07.131 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.132 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":0,\"lightning\":{\"table-concurrency\":0,\"index-concurrency\":0,\"region-concurrency\":0,\"io-concurrency\":0,\"check-requirements\":false},\"tidb\":{\"host\":\"\",\"port\":0,\"user\":\"\",\"status-port\":0,\"pd-addr\":\"\",\"sql-mode\":\"\",\"tls\":\"\",\"security\":null,\"max-allowed-packet\":0,\"distsql-scan-concurrency\":0,\"build-stats-concurrency\":0,\"index-serial-scan-concurrency\":0,\"checksum-table-concurrency\":0},\"checkpoint\":{\"schema\":\"\",\"driver\":\"invalid\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":0,\"batch-size\":0,\"batch-import-ratio\":0,\"data-source-dir\":\"file:///root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/pkg/lightning\",\"character-set\":\"\",\"csv\":{\"separator\":\"\",\"delimiter\":\"\",\"null\":\"\",\"header\":false,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":false},\"max-region-size\":0,\"filter\":[\"*.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"\",\"on-duplicate\":\"\",\"max-kv-pairs\":0,\"send-kv-pairs\":0,\"region-split-size\":0,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":0,\"engine-mem-cache-size\":0,\"local-writer-mem-cache-size\":0},\"post-restore\":{\"checksum\":\"off\",\"analyze\":\"off\",\"level-1-compact\":false,\"post-process-at-last\":false,\"compact\":false},\"cron\":{\"switch-mode\":\"0s\",\"log-progress\":\"0s\",\"check-disk-quota\":\"0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.132 +08:00] [INFO] [lightning.go:285] ["load data source start"]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/backend.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/backend.go__failpoint_stash__]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/backend_test.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/binding__failpoint_binding__.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/importer/importer.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/importer/importer_test.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/allocator.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/session.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/session_test.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/sql2kv.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/sql2kv_test.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/types.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/binding__failpoint_binding__.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local.go__failpoint_stash__]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local_test.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local_unix.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local_unix.go__failpoint_stash__]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local_windows.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/localhelper.go]
[2021/04/19 14:14:07.133 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/localhelper_test.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/noop/noop.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/tidb/binding__failpoint_binding__.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/tidb/tidb.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/tidb/tidb.go__failpoint_stash__]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/tidb/tidb_test.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=binding__failpoint_binding__.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpoints.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpoints_file_test.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpoints_sql_test.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpoints_test.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpointspb/file_checkpoints.pb.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpointspb/file_checkpoints.proto]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/glue_checkpoint.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/tidb.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/once_error.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/once_error_test.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/pause.go]
[2021/04/19 14:14:07.134 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/pause_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/security.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/security_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/storage.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/storage_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/storage_unix.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/storage_windows.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/util.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/util_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/bytesize.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/bytesize_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/config.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/config_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/configlist.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/configlist_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/const.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/global.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=glue/glue.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=lightning.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=lightning.go__failpoint_stash__]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=lightning_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/filter.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/filter_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/log.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/log_test.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/redact.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/testlogger.go]
[2021/04/19 14:14:07.135 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=manual/manual.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=manual/manual_nocgo.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=metric/metric.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=metric/metric_test.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/bytes.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/csv/split_large_file.csv]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/csv_parser.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/csv_parser_test.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/metadata]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/loader.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/loader_test.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parquet_parser.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parquet_parser_test.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parser.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parser.rl]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parser_generated.go]
[2021/04/19 14:14:07.136 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parser_test.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/reader.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/reader_test.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/region.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/region_test.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/router.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/router_test.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/binding__failpoint_binding__.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/checksum.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/checksum.go__failpoint_stash__]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/checksum_test.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/restore.go]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/restore.go__failpoint_stash__]
[2021/04/19 14:14:07.140 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/restore_test.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/tidb.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/tidb_test.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=sigusr1_other.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=sigusr1_unix.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=tikv/tikv.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=tikv/tikv_test.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=verification/checksum.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=verification/checksum_test.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=web/progress.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=web/res.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=web/res_vfsdata.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=worker/worker.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=worker/worker_test.go]
[2021/04/19 14:14:07.141 +08:00] [INFO] [lightning.go:288] ["load data source completed"] [takeTime=8.898251ms] []
[2021/04/19 14:14:07.141 +08:00] [ERROR] [lightning.go:310] ["restore failed"] [error="open checkpoint db failed: Unknown checkpoint driver invalid"]
[2021/04/19 14:14:07.141 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.141 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":0,\"lightning\":{\"table-concurrency\":0,\"index-concurrency\":0,\"region-concurrency\":0,\"io-concurrency\":0,\"check-requirements\":false},\"tidb\":{\"host\":\"\",\"port\":0,\"user\":\"\",\"status-port\":0,\"pd-addr\":\"\",\"sql-mode\":\"\",\"tls\":\"\",\"security\":null,\"max-allowed-packet\":0,\"distsql-scan-concurrency\":0,\"build-stats-concurrency\":0,\"index-serial-scan-concurrency\":0,\"checksum-table-concurrency\":0},\"checkpoint\":{\"schema\":\"\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":0,\"batch-size\":0,\"batch-import-ratio\":0,\"data-source-dir\":\".\",\"character-set\":\"\",\"csv\":{\"separator\":\"\",\"delimiter\":\"\",\"null\":\"\",\"header\":false,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":false},\"max-region-size\":0,\"filter\":[\"*.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":false},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"\",\"on-duplicate\":\"\",\"max-kv-pairs\":0,\"send-kv-pairs\":0,\"region-split-size\":0,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":0,\"engine-mem-cache-size\":0,\"local-writer-mem-cache-size\":0},\"post-restore\":{\"checksum\":\"off\",\"analyze\":\"off\",\"level-1-compact\":false,\"post-process-at-last\":false,\"compact\":false},\"cron\":{\"switch-mode\":\"0s\",\"log-progress\":\"0s\",\"check-disk-quota\":\"0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.142 +08:00] [INFO] [lightning.go:285] ["load data source start"]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/backend.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/backend.go__failpoint_stash__]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/backend_test.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/binding__failpoint_binding__.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/importer/importer.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/importer/importer_test.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/allocator.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/session.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/session_test.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/sql2kv.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/sql2kv_test.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/kv/types.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/binding__failpoint_binding__.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local.go__failpoint_stash__]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local_test.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local_unix.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local_unix.go__failpoint_stash__]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/local_windows.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/localhelper.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/local/localhelper_test.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/noop/noop.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/tidb/binding__failpoint_binding__.go]
[2021/04/19 14:14:07.142 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/tidb/tidb.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/tidb/tidb.go__failpoint_stash__]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=backend/tidb/tidb_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=binding__failpoint_binding__.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpoints.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpoints_file_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpoints_sql_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpoints_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpointspb/file_checkpoints.pb.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/checkpointspb/file_checkpoints.proto]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/glue_checkpoint.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=checkpoints/tidb.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/once_error.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/once_error_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/pause.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/pause_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/security.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/security_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/storage.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/storage_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/storage_unix.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/storage_windows.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/util.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=common/util_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/bytesize.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/bytesize_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/config.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/config_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/configlist.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/configlist_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/const.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=config/global.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=glue/glue.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=lightning.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=lightning.go__failpoint_stash__]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=lightning_test.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/filter.go]
[2021/04/19 14:14:07.143 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/filter_test.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/log.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/log_test.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/redact.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=log/testlogger.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=manual/manual.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=manual/manual_nocgo.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=metric/metric.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=metric/metric_test.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/bytes.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/csv/split_large_file.csv]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/csv_parser.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/csv_parser_test.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/metadata]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/mocker_test-schema-create.sql]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/mocker_test.i-schema.sql]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/mocker_test.i.sql]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/mocker_test.report_case_high_risk-schema.sql]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/mocker_test.report_case_high_risk.sql]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/mocker_test.tbl_autoid-schema.sql]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/mocker_test.tbl_autoid.sql]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/mocker_test.tbl_multi_index-schema.sql]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/examples/mocker_test.tbl_multi_index.sql]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/loader.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/loader_test.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parquet_parser.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parquet_parser_test.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parser.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parser.rl]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parser_generated.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/parser_test.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/reader.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/reader_test.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/region.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/region_test.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/router.go]
[2021/04/19 14:14:07.144 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=mydump/router_test.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/binding__failpoint_binding__.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/checksum.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/checksum.go__failpoint_stash__]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/checksum_test.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/restore.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/restore.go__failpoint_stash__]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/restore_test.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/tidb.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=restore/tidb_test.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=sigusr1_other.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=sigusr1_unix.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=tikv/tikv.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=tikv/tikv_test.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=verification/checksum.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=verification/checksum_test.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=web/progress.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=web/res.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=web/res_vfsdata.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=worker/worker.go]
[2021/04/19 14:14:07.145 +08:00] [INFO] [loader.go:299] ["[loader] file is filtered by file router"] [path=worker/worker_test.go]
[2021/04/19 14:14:07.145 +08:00] [ERROR] [lightning.go:288] ["load data source failed"] [takeTime=3.474407ms] [error="no schema create sql files found. Please either set `mydumper.no-schema` to true or add schema sql file for each database"]
[2021/04/19 14:14:07.146 +08:00] [INFO] [lightning.go:163] ["starting HTTP server"] [address=127.0.0.1:40264]
[2021/04/19 14:14:07.146 +08:00] [INFO] [lightning.go:170] ["stopped HTTP server"] [error="http: Server closed"]
[2021/04/19 14:14:07.155 +08:00] [INFO] [lightning.go:163] ["starting HTTP server"] [address=127.0.0.1:32785]
[2021/04/19 14:14:07.156 +08:00] [INFO] [lightning.go:170] ["stopped HTTP server"] [error="http: Server closed"]
[2021/04/19 14:14:07.157 +08:00] [INFO] [lightning.go:163] ["starting HTTP server"] [address=127.0.0.1:38713]
[2021/04/19 14:14:07.157 +08:00] [INFO] [lightning.go:196] ["Lightning server is running, post to /tasks to start an import task"] [address=127.0.0.1:38713]
[2021/04/19 14:14:07.259 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.259 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847259872452,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-1\",\"character-set\":\"auto\",\"csv\":{\"separator\":\",\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.373 +08:00] [INFO] [lightning.go:507] ["canceled task"] [taskID=123456] [success=false]
[2021/04/19 14:14:07.373 +08:00] [INFO] [lightning.go:507] ["canceled task"] [taskID=1618812847260267651] [success=true]
[2021/04/19 14:14:07.374 +08:00] [INFO] [lightning.go:507] ["canceled task"] [taskID=1618812847259872452] [success=true]
[2021/04/19 14:14:07.374 +08:00] [ERROR] [lightning.go:209] ["tidb lightning encountered error"] [error="context canceled"]
[2021/04/19 14:14:07.374 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.374 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847260504691,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-3\",\"character-set\":\"auto\",\"csv\":{\"separator\":\",\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.475 +08:00] [ERROR] [lightning.go:209] ["tidb lightning encountered error"] [error="context canceled"]
[2021/04/19 14:14:07.475 +08:00] [INFO] [lightning.go:170] ["stopped HTTP server"] [error="http: Server closed"]
[2021/04/19 14:14:07.478 +08:00] [INFO] [lightning.go:163] ["starting HTTP server"] [address=127.0.0.1:35047]
[2021/04/19 14:14:07.478 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.479 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847478944651,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://.\",\"character-set\":\"auto\",\"csv\":{\"separator\":\",\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.597 +08:00] [INFO] [lightning.go:507] ["canceled task"] [taskID=123456] [success=false]
[2021/04/19 14:14:07.597 +08:00] [INFO] [lightning.go:507] ["canceled task"] [taskID=1618812847478944651] [success=true]
[2021/04/19 14:14:07.597 +08:00] [INFO] [lightning.go:170] ["stopped HTTP server"] [error="http: Server closed"]
[2021/04/19 14:14:07.599 +08:00] [INFO] [lightning.go:163] ["starting HTTP server"] [address=127.0.0.1:41103]
[2021/04/19 14:14:07.600 +08:00] [INFO] [lightning.go:196] ["Lightning server is running, post to /tasks to start an import task"] [address=127.0.0.1:41103]
[2021/04/19 14:14:07.706 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.706 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847706280891,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-0\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.706 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.706 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847706720163,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-1\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.707 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.707 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847706990132,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-2\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.707 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.707 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847707290505,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-3\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.707 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.707 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847707691747,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-4\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.708 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.708 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847708459413,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-5\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.708 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.708 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847708840943,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-6\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.709 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.709 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847709163637,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-7\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.709 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.709 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847709492984,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-8\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.709 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.709 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847709850111,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-9\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.710 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.710 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847710164494,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-10\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.710 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.710 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847710507533,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-11\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.710 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.710 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847710806640,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-12\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.711 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.711 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847711142894,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-13\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.711 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.711 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847711458741,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-14\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.711 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.711 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847711752950,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-15\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.712 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.712 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847712079035,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-16\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.712 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.712 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847712423381,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-17\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.712 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.712 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847712731818,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-18\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.713 +08:00] [INFO] [info.go:40] ["Welcome to TiDB-Lightning"] [release-version=v5.0.0-master-dirty] [git-hash=db863d52a070eb5050b65a2d5fe3678d50b6bc6a] [git-branch=master] [go-version=go1.16] [utc-build-time="2021-04-19 06:12:46"] [race-enabled=false]
[2021/04/19 14:14:07.713 +08:00] [INFO] [lightning.go:218] [cfg] [cfg="{\"id\":1618812847713023270,\"lightning\":{\"table-concurrency\":6,\"index-concurrency\":2,\"region-concurrency\":8,\"io-concurrency\":5,\"check-requirements\":true},\"tidb\":{\"host\":\"test.invalid\",\"port\":4000,\"user\":\"root\",\"status-port\":10080,\"pd-addr\":\"test.invalid:2379\",\"sql-mode\":\"ONLY_FULL_GROUP_BY,NO_AUTO_CREATE_USER\",\"tls\":\"false\",\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"max-allowed-packet\":67108864,\"distsql-scan-concurrency\":15,\"build-stats-concurrency\":20,\"index-serial-scan-concurrency\":20,\"checksum-table-concurrency\":2},\"checkpoint\":{\"schema\":\"tidb_lightning_checkpoint\",\"driver\":\"file\",\"enable\":true,\"keep-after-success\":false},\"mydumper\":{\"read-block-size\":65536,\"batch-size\":107374182400,\"batch-import-ratio\":0,\"data-source-dir\":\"file://demo-path-19\",\"character-set\":\"auto\",\"csv\":{\"separator\":\"/\",\"delimiter\":\"\\\"\",\"null\":\"\\\\N\",\"header\":true,\"trim-last-separator\":false,\"not-null\":false,\"backslash-escape\":true},\"max-region-size\":268435456,\"filter\":[\"*.*\",\"!mysql.*\",\"!sys.*\",\"!INFORMATION_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!INSPECTION_SCHEMA.*\"],\"files\":null,\"no-schema\":false,\"case-sensitive\":false,\"strict-format\":false,\"default-file-rules\":true},\"tikv-importer\":{\"addr\":\"\",\"backend\":\"importer\",\"on-duplicate\":\"replace\",\"max-kv-pairs\":4096,\"send-kv-pairs\":32768,\"region-split-size\":100663296,\"sorted-kv-dir\":\"\",\"disk-quota\":0,\"range-concurrency\":16,\"engine-mem-cache-size\":536870912,\"local-writer-mem-cache-size\":134217728},\"post-restore\":{\"checksum\":\"required\",\"analyze\":\"optional\",\"level-1-compact\":false,\"post-process-at-last\":true,\"compact\":false},\"cron\":{\"switch-mode\":\"5m0s\",\"log-progress\":\"5m0s\",\"check-disk-quota\":\"1m0s\"},\"routes\":null,\"security\":{\"ca-path\":\"\",\"cert-path\":\"\",\"key-path\":\"\",\"redact-info-log\":false},\"black-white-list\":{\"do-tables\":null,\"do-dbs\":null,\"ignore-tables\":null,\"ignore-dbs\":null}}"]
[2021/04/19 14:14:07.717 +08:00] [INFO] [lightning.go:170] ["stopped HTTP server"] [error="http: Server closed"]
OOPS: 6 passed, 1 FAILED
--- FAIL: TestLightning (0.59s)
FAIL
FAIL    github.com/pingcap/br/pkg/lightning 0.700s
ok      github.com/pingcap/br/pkg/lightning/backend 0.072s
ok      github.com/pingcap/br/pkg/lightning/backend/importer    0.060s [no tests to run]
ok      github.com/pingcap/br/pkg/lightning/backend/kv  (cached)
ok      github.com/pingcap/br/pkg/lightning/backend/local   1.594s
?       github.com/pingcap/br/pkg/lightning/backend/noop    [no test files]
ok      github.com/pingcap/br/pkg/lightning/backend/tidb    (cached)
ok      github.com/pingcap/br/pkg/lightning/checkpoints (cached)
?       github.com/pingcap/br/pkg/lightning/checkpoints/checkpointspb   [no test files]
ok      github.com/pingcap/br/pkg/lightning/common  (cached)
ok      github.com/pingcap/br/pkg/lightning/config  0.847s
?       github.com/pingcap/br/pkg/lightning/glue    [no test files]
ok      github.com/pingcap/br/pkg/lightning/log (cached)
?       github.com/pingcap/br/pkg/lightning/manual  [no test files]
ok      github.com/pingcap/br/pkg/lightning/metric  (cached)
ok      github.com/pingcap/br/pkg/lightning/mydump  0.210s
ok      github.com/pingcap/br/pkg/lightning/restore 30.277s
ok      github.com/pingcap/br/pkg/lightning/tikv    (cached) [no tests to run]
ok      github.com/pingcap/br/pkg/lightning/verification    (cached)
?       github.com/pingcap/br/pkg/lightning/web [no test files]
ok      github.com/pingcap/br/pkg/lightning/worker  (cached)
ok      github.com/pingcap/br/pkg/logutil   (cached)
?       github.com/pingcap/br/pkg/manual    [no test files]
ok      github.com/pingcap/br/pkg/mock  0.254s
ok      github.com/pingcap/br/pkg/pdutil    (cached)
ok      github.com/pingcap/br/pkg/redact    (cached)
ok      github.com/pingcap/br/pkg/restore   1.620s
ok      github.com/pingcap/br/pkg/rtree (cached)
ok      github.com/pingcap/br/pkg/storage   1.918s
ok      github.com/pingcap/br/pkg/summary   (cached)
ok      github.com/pingcap/br/pkg/task  (cached)
ok      github.com/pingcap/br/pkg/trace (cached)
ok      github.com/pingcap/br/pkg/utils (cached)
ok      github.com/pingcap/br/pkg/version   (cached)
ok      github.com/pingcap/br/pkg/version/build 0.030s
?       github.com/pingcap/br/tests/br_gcs  [no test files]
?       github.com/pingcap/br/tests/br_key_locked   [no test files]
?       github.com/pingcap/br/tests/br_rawkv    [no test files]
?       github.com/pingcap/br/tests/br_z_gc_safepoint   [no test files]
?       github.com/pingcap/br/tests/lightning_checkpoint_parquet    [no test files]
FAIL
make[1]: 进入目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br”
install tools...
make[2]: 进入目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/tools”
make[2]: 对“all”无需做任何事。
make[2]: 离开目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br/tools”
tools/bin/failpoint-ctl disable
make[1]: 离开目录“/root/.gvm/pkgsets/go1.16/global/src/github.com/pingcap/br”
make: *** [test] 错误 1
glorv commented 3 years ago

I see. This is an unstable unit test which was fixed in #1030, please update the newest code and try again, thanks~

wuhuizuo commented 1 year ago

It's fixed, close it.