Ng启动失败 (Ng startup failure) ，提示 Arena too small, toWrite:126 newTotal:xxx limit:xxx

手动杀死Prometheus和ng后，ng重启失败 (Description ng fails to restart after prometheus and ng are manually killed)

tidb版本：6.5 ng版本： Git Commit Hash"=f1c05e221155c2c95d391957971defbcbbf56832

错误日志位于：/tidb-deploy/prometheus-9090/log/tsdb.log

2023-03-07T06:36:53.570Z        info    /go/pkg/mod/github.com/!victoria!metrics/!victoria!metrics@v1.65.0/app/vmselect/promql/rollup_result_cache.go:106       loading rollupResult cache from "/tidb-data/prometheus-9090/tsdb/cache/rollupResult"...
2023-03-07T06:36:53.571Z        info    /go/pkg/mod/github.com/!victoria!metrics/!victoria!metrics@v1.65.0/app/vmselect/promql/rollup_result_cache.go:132       loaded rollupResult cache from "/tidb-data/prometheus-9090/tsdb/cache/rollupResult" in 0.001 seconds; entriesCount: 0, sizeBytes: 0
2023-03-07T06:36:53.571Z        info    /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/ng-monitoring/database/timeseries/vm.go:45 started VictoriaMetrics in 0.094 seconds
2023/03/07 14:36:53 Arena too small, toWrite:126 newTotal:21810362 limit:21810342
github.com/dgraph-io/badger/v3/y.AssertTruef
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/y/error.go:62
github.com/dgraph-io/badger/v3/skl.(*Arena).putVal
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/skl/arena.go:83
github.com/dgraph-io/badger/v3/skl.newNode
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/skl/skl.go:114
github.com/dgraph-io/badger/v3/skl.(*Skiplist).Put
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/skl/skl.go:306
github.com/dgraph-io/badger/v3.(*memTable).replayFunction.func1
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:258
github.com/dgraph-io/badger/v3.(*logFile).iterate
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:521
github.com/dgraph-io/badger/v3.(*memTable).UpdateSkipList
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:219
github.com/dgraph-io/badger/v3.(*DB).openMemTable
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:146
github.com/dgraph-io/badger/v3.(*DB).openMemTables
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/memtable.go:88
github.com/dgraph-io/badger/v3.Open
        /go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.1/db.go:333
github.com/genjidb/genji/engine/badgerengine.NewEngine
        /go/pkg/mod/github.com/crazycs520/genji/engine/badgerengine@v0.12.1-0.20220328082424-727a2d089bde/engine.go:25
github.com/pingcap/ng-monitoring/database/document.Init
        /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/ng-monitoring/database/document/document.go:34
github.com/pingcap/ng-monitoring/database.Init
        /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/ng-monitoring/database/database.go:14
main.main
        /home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/ng-monitoring/main.go:68
runtime.main
        /usr/local/go/src/runtime/proc.go:250
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1594

ng.log 无报错最后的日志如下


[2023/03/07 14:41:44.272 +08:00] [INFO] [printer.go:25] ["Welcome to ng-monitoring."] ["Git Commit Hash"=f1c05e221155c2c95d391957971defbcbbf56832] ["Git Branch"=heads/refs/tags/v6.5.0] ["UTC Build Time"="2022-12-16 08:18:47"] [GoVersion=go1.19.3]
[2023/03/07 14:41:44.273 +08:00] [INFO] [main.go:64] [config] [config="{\"address\":\"0.0.0.0:12020\",\"advertise_address\":\"10.0.0.80:12020\",\"pd\":{\"endpoints\":[\"10.0.0.80:2379\",\"10.0.0.76:2379\",\"10.0.0.74:2379\"]},\"log\":{\"path\":\"/tidb-deploy/prometheus-9090/log\",\"level\":\"INFO\"},\"storage\":{\"path\":\"/tidb-data/prometheus-9090\"},\"continuous_profiling\":{\"enable\":false,\"profile_seconds\":10,\"interval_seconds\":60,\"timeout_seconds\":120,\"data_retention_seconds\":259200},\"security\":{\"ca_path\":\"\",\"cert_path\":\"\",\"key_path\":\"\"}}"]

pingcap / ng-monitoring

Ng启动失败 (Ng startup failure) ，提示 Arena too small, toWrite:126 newTotal:xxx limit:xxx #182