Flaky integration test `http_api`

[pingyu]

Bug Report

1. Describe the bug

2. Minimal reproduce step (Required)

3. What did you see instead (Required)

4. What did you expect to see? (Required)

https://do.pingcap.net/jenkins/blue/organizations/jenkins/tikv%2Fmigration%2Fpull_integration_test/detail/pull_integration_test/125/pipeline/237/

run put data
[2023/08/25 11:44:10.150 +08:00] [INFO] [client.go:397] ["[pd] create pd client with endpoints"] [pd-address="[https://127.0.0.1:2579]"]
[2023/08/25 11:44:10.159 +08:00] [INFO] [base_client.go:378] ["[pd] switch leader"] [new-leader=https://127.0.0.1:2579] [old-leader=]
[2023/08/25 11:44:10.159 +08:00] [INFO] [base_client.go:105] ["[pd] init cluster id"] [cluster-id=7271100523333625434]
[2023/08/25 11:44:10.159 +08:00] [INFO] [client.go:690] ["[pd] tso dispatcher created"] [dc-location=global]
[2023/08/25 11:44:13.332 +08:00] [INFO] [region_cache.go:2529] ["[health check] check health error"] [store=127.0.0.1:22160] [error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:22160: connect: connection refused\""]
[2023/08/25 11:44:13.332 +08:00] [INFO] [region_request.go:794] ["mark store's regions need be refill"] [id=1] [addr=127.0.0.1:22160] [error="rpc error: code = Unavailable desc = error reading from server: read tcp 127.0.0.1:44978->127.0.0.1:22160: read: connection reset by peer"] [errorVerbose="rpc error: code = Unavailable desc = error reading from server: read tcp 127.0.0.1:44978->127.0.0.1:22160: read: connection reset by peer\ngithub.com/tikv/client-go/v2/internal/client.sendBatchRequest\n\tgithub.com/tikv/client-go/v2@v2.0.1-0.20220818084834-0d0ae0dcfb1f/internal/client/client_batch.go:789\ngithub.com/tikv/client-go/v2/internal/client.(*RPCClient).sendRequest\n\tgithub.com/tikv/client-go/v2@v2.0.1-0.20220818084834-0d0ae0dcfb1f/internal/client/client.go:498\ngithub.com/tikv/client-go/v2/internal/client.(*RPCClient).SendRequest\n\tgithub.com/tikv/client-go/v2@v2.0.1-0.20220818084834-0d0ae0dcfb1f/internal/client/client.go:541\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionRequestSender).sendReqToRegion\n\tgithub.com/tikv/client-go/v2@v2.0.1-0.20220818084834-0d0ae0dcfb1f/internal/locate/region_request.go:1184\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionRequestSender).SendReqCtx\n\tgithub.com/tikv/client-go/v2@v2.0.1-0.20220818084834-0d0ae0dcfb1f/internal/locate/region_request.go:1017\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionRequestSender).SendReq\n\tgithub.com/tikv/client-go/v2@v2.0.1-0.20220818084834-0d0ae0dcfb1f/internal/locate/region_request.go:233\ngithub.com/tikv/client-go/v2/rawkv.(*Client).sendReq\n\tgithub.com/tikv/client-go/v2@v2.0.1-0.20220818084834-0d0ae0dcfb1f/rawkv/rawkv.go:668\ngithub.com/tikv/client-go/v2/rawkv.(*Client).Delete\n\tgithub.com/tikv/client-go/v2@v2.0.1-0.20220818084834-0d0ae0dcfb1f/rawkv/rawkv.go:404\nmain.runDeleteCmd.func1\n\tgithub.com/tikv/migration/cdc/tests/utils/rawkv_data/gen_data.go:103\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:75\nruntime.goexit\n\truntime/asm_amd64.s:1594"]
[2023/08/25 11:44:13.334 +08:00] [INFO] [region_cache.go:2529] ["[health check] check health error"] [store=127.0.0.1:22160] [error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:22160: connect: connection refused\""]
[2023/08/25 11:44:13.335 +08:00] [INFO] [region_cache.go:2529] ["[health check] check health error"] [store=127.0.0.1:22160] [error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:22160: connect: connection refused\""]
[2023/08/25 11:44:14.333 +08:00] [INFO] [region_cache.go:2529] ["[health check] check health error"] [store=127.0.0.1:22160] [error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:22160: connect: connection refused\""]
[2023/08/25 11:44:15.333 +08:00] [INFO] [region_cache.go:2529] ["[health check] check health error"] [store=127.0.0.1:22160] [error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:22160: connect: connection refused\""]
[2023/08/25 11:44:16.333 +08:00] [INFO] [region_cache.go:2529] ["[health check] check health error"] [store=127.0.0.1:22160] [error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:22160: connect: connection refused\""]
[2023/08/25 11:44:17.333 +08:00] [INFO] [region_cache.go:2529] ["[health check] check health error"] [store=127.0.0.1:22160] [error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:22160: connect: connection refused\""]
[2023/08/25 11:44:18.332 +08:00] [INFO] [client_batch.go:581] ["batchRecvLoop re-create streaming fail"] [target=127.0.0.1:22160] [forwardedHost=] [error="context deadline exceeded"]
[2023/08/25 11:44:18.332 +08:00] [INFO] [client_batch.go:581] ["batchRecvLoop re-create streaming fail"] [target=127.0.0.1:22160] [forwardedHost=] [error="context deadline exceeded"]
[2023/08/25 11:44:18.332 +08:00] [INFO] [client_batch.go:581] ["batchRecvLoop re-create streaming fail"] [target=127.0.0.1:22160] [forwardedHost=] [error="context deadline exceeded"]
[2023/08/25 11:44:18.332 +08:00] [INFO] [client_batch.go:581] ["batchRecvLoop re-create streaming fail"] [target=127.0.0.1:22160] [forwardedHost=] [error="context deadline exceeded"]

The tikv-server seems to be killed.

5. What is your migration tool and TiKV version? (Required)

• TiKV:
• TiKV CDC: https://github.com/tikv/migration/pull/349/commits/c305708a78118f919d9bfdc239a8af1dde14bbdf
• TiKV BR:
• TiKV Online Bulk Load:

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity.