TDengine 3.0.3.1单机版服务器断电后文件损坏

Bug Description TDengine单机版服务器断电后vnode文件损坏，启动失败。如何修复损坏文件

To Reproduce 服务器断电导致文件损坏查看TDengine日志如下： sysctl: setting key "kernel.core_pattern": Read-only file system 08/09 13:17:30.287889 00000035 TAOS_ADAPTER info "start server:00000035" model=main [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.

using env: export GIN_MODE=release
using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] GET /debug/pprof/ --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers) [GIN-debug] GET /debug/pprof/cmdline --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers) [GIN-debug] GET /debug/pprof/profile --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers) [GIN-debug] POST /debug/pprof/symbol --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers) [GIN-debug] GET /debug/pprof/symbol --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers) [GIN-debug] GET /debug/pprof/trace --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers) [GIN-debug] GET /debug/pprof/allocs --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers) [GIN-debug] GET /debug/pprof/block --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers) [GIN-debug] GET /debug/pprof/goroutine --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers) [GIN-debug] GET /debug/pprof/heap --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers) [GIN-debug] GET /debug/pprof/mutex --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers) [GIN-debug] GET /debug/pprof/threadcreate --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers) [GIN-debug] GET /metrics --> github.com/gin-gonic/gin.WrapH.func1 (5 handlers) [GIN-debug] GET /-/ping --> github.com/taosdata/taosadapter/v3/controller/ping.Controller.Init.func1 (5 handlers) [GIN-debug] POST /rest/sql --> github.com/taosdata/taosadapter/v3/controller/rest.(Restful).sql-fm (7 handlers) [GIN-debug] POST /rest/sql/:db --> github.com/taosdata/taosadapter/v3/controller/rest.(Restful).sql-fm (7 handlers) [GIN-debug] POST /rest/sql/:db/vgid --> github.com/taosdata/taosadapter/v3/controller/rest.(Restful).tableVgID-fm (7 handlers) [GIN-debug] GET /rest/login/:user/:password --> github.com/taosdata/taosadapter/v3/controller/rest.(Restful).des-fm (6 handlers) [GIN-debug] GET /rest/ws --> github.com/taosdata/taosadapter/v3/controller/rest.(Restful).ws-fm (6 handlers) [GIN-debug] GET /rest/stmt --> github.com/taosdata/taosadapter/v3/controller/rest.(Restful).stmt-fm (6 handlers) [GIN-debug] GET /rest/tmq --> github.com/taosdata/taosadapter/v3/controller/rest.(Restful).tmq-fm (6 handlers) [GIN-debug] POST /rest/upload --> github.com/taosdata/taosadapter/v3/controller/rest.(Restful).upload-fm (7 handlers) [GIN-debug] GET /rest/schemaless --> github.com/taosdata/taosadapter/v3/controller/rest.(Restful).schemalessWs-fm (6 handlers) [GIN-debug] GET /genauth/:user/:password/:key --> github.com/taosdata/taosadapter/v3/plugin.RegisterGenerateAuth.func1 (5 handlers) 08/09 13:17:30.288171 00000035 TAOS_ADAPTER info "init plugin collectd/v1" model=plugin 08/09 13:17:30.288854 00000035 TAOS_ADAPTER info "collectd disabled" model=collectd 08/09 13:17:30.288861 00000035 TAOS_ADAPTER info "init plugin influxdb/v1" model=plugin [GIN-debug] POST /influxdb/v1/write --> github.com/taosdata/taosadapter/v3/plugin/influxdb.(Influxdb).write-fm (7 handlers) 08/09 13:17:30.288970 00000035 TAOS_ADAPTER info "init plugin node_exporter/v1" model=plugin 08/09 13:17:30.290361 00000035 TAOS_ADAPTER info "node_exporter disabled" model=NodeExporter 08/09 13:17:30.290367 00000035 TAOS_ADAPTER info "init plugin opentsdb/v1" model=plugin [GIN-debug] POST /opentsdb/v1/put/json/:db --> github.com/taosdata/taosadapter/v3/plugin/opentsdb.(Plugin).insertJson-fm (7 handlers) [GIN-debug] POST /opentsdb/v1/put/telnet/:db --> github.com/taosdata/taosadapter/v3/plugin/opentsdb.(Plugin).insertTelnet-fm (7 handlers) 08/09 13:17:30.290478 00000035 TAOS_ADAPTER info "init plugin opentsdb_telnet/v1" model=plugin 08/09 13:17:30.291405 00000035 TAOS_ADAPTER info "opentsdb_telnet disabled" model=opentsdb_telnet 08/09 13:17:30.291411 00000035 TAOS_ADAPTER info "init plugin prometheus/v1" model=plugin [GIN-debug] POST /prometheus/v1/remote_read/:db --> github.com/taosdata/taosadapter/v3/plugin/prometheus.(Plugin).Read-fm (7 handlers) [GIN-debug] POST /prometheus/v1/remote_write/:db --> github.com/taosdata/taosadapter/v3/plugin/prometheus.(Plugin).Write-fm (7 handlers) 08/09 13:17:30.291536 00000035 TAOS_ADAPTER info "init plugin statsd/v1" model=plugin 08/09 13:17:30.292998 00000035 TAOS_ADAPTER info "statsd disabled" model=statsd 08/09 13:17:30.293260 00000035 TAOS_ADAPTER info "all plugin init finish" model=plugin 08/09 13:17:30.293265 00000035 TAOS_ADAPTER info "all plugin start finish" model=plugin 08/09 13:17:30.293363 00000035 TAOS_ADAPTER info "Running in terminal." model=main 08/09 13:17:30.295760 00000035 TAOS_ADAPTER info "server on : 6041" model=main 08/09 13:17:30.748843 00000084 UTL apoll url not load 08/09 13:17:30.748874 00000084 UTL load from cfg file /etc/taos//taos.cfg success 08/09 13:17:30.748881 00000084 UTL env file:.env not load 08/09 13:17:30.748894 00000084 UTL load from env variables cfg success 08/09 13:17:30.749175 00000084 DND startup path:/root args:taosd 08/09 13:17:30.749408 00000084 UTL apoll url not load 08/09 13:17:30.749437 00000084 UTL load from cfg file /etc/taos//taos.cfg success 08/09 13:17:30.749442 00000084 UTL env file:.env not load 08/09 13:17:30.749453 00000084 UTL load from env variables cfg success 08/09 13:17:30.749547 00000084 UTL global config 08/09 13:17:30.749551 00000084 UTL ================================================================= 08/09 13:17:30.749553 00000084 UTL default firstEp taosd.base-component.svc.cluster.local:6030 08/09 13:17:30.749556 00000084 UTL default secondEp taosd.base-component.svc.cluster.local:6030 08/09 13:17:30.749558 00000084 UTL env_var fqdn taosd.base-component.svc.cluster.local 08/09 13:17:30.749562 00000084 UTL env_var serverPort 6030 08/09 13:17:30.749564 00000084 UTL default tempDir /tmp/ 08/09 13:17:30.749566 00000084 UTL default minimalTmpDirGB 1.00 08/09 13:17:30.749572 00000084 UTL default shellActivityTimer 3 08/09 13:17:30.749574 00000084 UTL default compressMsgSize -1 08/09 13:17:30.749577 00000084 UTL default compressColData -1 08/09 13:17:30.749579 00000084 UTL default queryPolicy 1 08/09 13:17:30.749581 00000084 UTL default enableQueryHb 0 08/09 13:17:30.749583 00000084 UTL default querySmaOptimize 0 08/09 13:17:30.749585 00000084 UTL default queryPlannerTrace 0 08/09 13:17:30.749587 00000084 UTL default queryNodeChunkSize 32768 08/09 13:17:30.749590 00000084 UTL default queryUseNodeAllocator 1 08/09 13:17:30.749592 00000084 UTL default keepColumnName 0 08/09 13:17:30.749594 00000084 UTL default smlChildTableName
08/09 13:17:30.749596 00000084 UTL default smlTagName _tag_null 08/09 13:17:30.749599 00000084 UTL default maxMemUsedByInsert 1024 08/09 13:17:30.749601 00000084 UTL default maxRetryWaitTime 10000 08/09 13:17:30.749604 00000084 UTL default useAdapter 0 08/09 13:17:30.749606 00000084 UTL default crashReporting 1 08/09 13:17:30.749608 00000084 UTL default numOfRpcThreads 10 08/09 13:17:30.749610 00000084 UTL default numOfRpcSessions 10000 08/09 13:17:30.749613 00000084 UTL default timeToGetAvailableConn 500000 08/09 13:17:30.749615 00000084 UTL default numOfTaskQueueThreads 10 08/09 13:17:30.749617 00000084 UTL default dataDir /var/lib/taos/ 08/09 13:17:30.749620 00000084 UTL default minimalDataDirGB 2.00 08/09 13:17:30.749622 00000084 UTL default supportVnodes 96 08/09 13:17:30.749624 00000084 UTL default maxShellConns 50000 08/09 13:17:30.749626 00000084 UTL default statusInterval 1 08/09 13:17:30.749628 00000084 UTL default minSlidingTime 10 08/09 13:17:30.749630 00000084 UTL default minIntervalTime 1 08/09 13:17:30.749632 00000084 UTL default maxNumOfDistinctRes 10000000 08/09 13:17:30.749634 00000084 UTL default countAlwaysReturnValue 1 08/09 13:17:30.749636 00000084 UTL default queryBufferSize -1 08/09 13:17:30.749638 00000084 UTL default printAuth 0 08/09 13:17:30.749640 00000084 UTL default queryRspPolicy 0 08/09 13:17:30.749643 00000084 UTL default numOfRpcThreads 10 08/09 13:17:30.749644 00000084 UTL default numOfRpcSessions 10000 08/09 13:17:30.749647 00000084 UTL default timeToGetAvailableConn 10000 08/09 13:17:30.749649 00000084 UTL default numOfCommitThreads 4 08/09 13:17:30.749651 00000084 UTL default numOfMnodeReadThreads 4 08/09 13:17:30.749652 00000084 UTL default numOfVnodeQueryThreads 96 08/09 13:17:30.749654 00000084 UTL default ratioOfVnodeStreamThrea 1.00 08/09 13:17:30.749656 00000084 UTL default numOfVnodeFetchThreads 12 08/09 13:17:30.749658 00000084 UTL default numOfVnodeRsmaThreads 48 08/09 13:17:30.749659 00000084 UTL default numOfQnodeQueryThreads 96 08/09 13:17:30.749662 00000084 UTL default numOfSnodeSharedThreads 4 08/09 13:17:30.749664 00000084 UTL default numOfSnodeUniqueThreads 4 08/09 13:17:30.749666 00000084 UTL default rpcQueueMemoryAllowed 13460113817 08/09 13:17:30.749669 00000084 UTL default syncElectInterval 25000 08/09 13:17:30.749670 00000084 UTL default syncHeartbeatInterval 1000 08/09 13:17:30.749672 00000084 UTL default syncHeartbeatTimeout 20000 08/09 13:17:30.749675 00000084 UTL default vndCommitMaxInterval 600000 08/09 13:17:30.749678 00000084 UTL default mndSdbWriteDelta 200 08/09 13:17:30.749681 00000084 UTL default mndLogRetention 2000 08/09 13:17:30.749684 00000084 UTL default monitor 1 08/09 13:17:30.749686 00000084 UTL default monitorInterval 30 08/09 13:17:30.749687 00000084 UTL default monitorFqdn
08/09 13:17:30.749690 00000084 UTL default monitorPort 6043 08/09 13:17:30.749692 00000084 UTL default monitorMaxLogs 100 08/09 13:17:30.749694 00000084 UTL default monitorComp 0 08/09 13:17:30.749697 00000084 UTL default crashReporting 1 08/09 13:17:30.749699 00000084 UTL default telemetryReporting 1 08/09 13:17:30.749701 00000084 UTL default telemetryInterval 43200 08/09 13:17:30.749703 00000084 UTL default telemetryServer telemetry.taosdata.com 08/09 13:17:30.749705 00000084 UTL default telemetryPort 80 08/09 13:17:30.749708 00000084 UTL default transPullupInterval 2 08/09 13:17:30.749709 00000084 UTL default mqRebalanceInterval 2 08/09 13:17:30.749711 00000084 UTL default ttlUnit 86400 08/09 13:17:30.749713 00000084 UTL default ttlPushInterval 86400 08/09 13:17:30.749716 00000084 UTL default uptimeInterval 300 08/09 13:17:30.749718 00000084 UTL default queryRsmaTolerance 1000 08/09 13:17:30.749720 00000084 UTL default walFsyncDataSizeLimit 104857600 08/09 13:17:30.749722 00000084 UTL default udf 1 08/09 13:17:30.749724 00000084 UTL default udfdResFuncs
08/09 13:17:30.749726 00000084 UTL default udfdLdLibPath
08/09 13:17:30.749728 00000084 UTL default disableStream 0 08/09 13:17:30.749730 00000084 UTL default cacheLazyLoadThreshold 500 08/09 13:17:30.749732 00000084 UTL default configDir /etc/taos/ 08/09 13:17:30.749735 00000084 UTL default scriptDir /etc/taos/ 08/09 13:17:30.749737 00000084 UTL default logDir /var/log/taos/ 08/09 13:17:30.749739 00000084 UTL default minimalLogDirGB 1.00 08/09 13:17:30.749741 00000084 UTL default numOfLogLines 10000000 08/09 13:17:30.749743 00000084 UTL default asyncLog 1 08/09 13:17:30.749745 00000084 UTL default logKeepDays 0 08/09 13:17:30.749747 00000084 UTL default debugFlag 0 08/09 13:17:30.749749 00000084 UTL default simDebugFlag 143 08/09 13:17:30.749751 00000084 UTL default tmrDebugFlag 131 08/09 13:17:30.749754 00000084 UTL default uDebugFlag 131 08/09 13:17:30.749756 00000084 UTL default rpcDebugFlag 131 08/09 13:17:30.749758 00000084 UTL default jniDebugFlag 131 08/09 13:17:30.749759 00000084 UTL default qDebugFlag 131 08/09 13:17:30.749761 00000084 UTL default cDebugFlag 131 08/09 13:17:30.749763 00000084 UTL default dDebugFlag 131 08/09 13:17:30.749766 00000084 UTL default vDebugFlag 131 08/09 13:17:30.749768 00000084 UTL default mDebugFlag 131 08/09 13:17:30.749770 00000084 UTL default wDebugFlag 131 08/09 13:17:30.749772 00000084 UTL default sDebugFlag 131 08/09 13:17:30.749774 00000084 UTL default tsdbDebugFlag 131 08/09 13:17:30.749776 00000084 UTL default tqDebugFlag 131 08/09 13:17:30.749779 00000084 UTL default fsDebugFlag 131 08/09 13:17:30.749781 00000084 UTL default udfDebugFlag 131 08/09 13:17:30.749783 00000084 UTL default smaDebugFlag 131 08/09 13:17:30.749784 00000084 UTL default idxDebugFlag 131 08/09 13:17:30.749786 00000084 UTL default tdbDebugFlag 131 08/09 13:17:30.749789 00000084 UTL default metaDebugFlag 131 08/09 13:17:30.749795 00000084 UTL default timezone Asia/Shanghai (CST, +0800) 08/09 13:17:30.749797 00000084 UTL default locale en_US.UTF-8 08/09 13:17:30.749799 00000084 UTL default charset UTF-8 08/09 13:17:30.749800 00000084 UTL default assert 1 08/09 13:17:30.749802 00000084 UTL default enableCoreFile 1 08/09 13:17:30.749804 00000084 UTL default numOfCores 48.00 08/09 13:17:30.749808 00000084 UTL default SSE42 0 08/09 13:17:30.749810 00000084 UTL default AVX 0 08/09 13:17:30.749812 00000084 UTL default AVX2 0 08/09 13:17:30.749814 00000084 UTL default FMA 0 08/09 13:17:30.749815 00000084 UTL default SIMD-builtins 0 08/09 13:17:30.749817 00000084 UTL default tagFilterCache 0 08/09 13:17:30.749819 00000084 UTL default openMax 1048576 08/09 13:17:30.749821 00000084 UTL default streamMax 16 08/09 13:17:30.749823 00000084 UTL default pageSizeKB 4 08/09 13:17:30.749825 00000084 UTL default totalMemoryKB 131446424 08/09 13:17:30.749827 00000084 UTL default os sysname Linux 08/09 13:17:30.749829 00000084 UTL default os nodename tdengine-58cbb4b45b-dgk2g 08/09 13:17:30.749832 00000084 UTL default os release 5.4.0-155-generic 08/09 13:17:30.749834 00000084 UTL default os version #172-Ubuntu SMP Fri Jul 7 16:10:02 UTC 2023 08/09 13:17:30.749836 00000084 UTL default os machine x86_64 08/09 13:17:30.749838 00000084 UTL default version 3.0.3.1 08/09 13:17:30.749840 00000084 UTL default compatible_version 3.0.0.0 08/09 13:17:30.749842 00000084 UTL default gitinfo ac5f91a9f6726d698a534dd694fb573db669657d 08/09 13:17:30.749844 00000084 UTL default buildinfo Built at 2023-03-21 20:49 08/09 13:17:30.749846 00000084 UTL ================================================================= 08/09 13:17:30.751200 00000084 DND start to init dnode env 08/09 13:17:30.751685 00000084 DND succceed to read dnode file /var/lib/taos//dnode/dnode.json 08/09 13:17:30.752193 00000084 DND succceed to read mnode file /var/lib/taos//mnode/mnode.json 08/09 13:17:30.752314 00000084 DND file:/var/lib/taos//qnode/qnode.json not exist 08/09 13:17:30.752321 00000084 DND file:/var/lib/taos//snode/snode.json not exist 08/09 13:17:30.779855 00000084 DND dnode env is initialized 08/09 13:17:30.779867 00000084 DND start to init service 08/09 13:17:30.779878 00000084 DND node:dnode, start to open 08/09 13:17:30.779884 00000084 UTL worker:dnode-mgmt is initialized, min:1 max:1 08/09 13:17:30.779906 00000084 UTL worker:dnode-mgmt:0 is launched, total:1 08/09 13:17:30.779910 00000084 UTL worker:dnode-mgmt, queue:0x7f694dad5a80 is allocated, ahandle:0x7f694da6eb80 08/09 13:17:30.779915 00000107 UTL worker:dnode-mgmt:0 is running, thread:00000107 08/09 13:17:30.780072 00000108 UDF start to init udfd 08/09 13:17:30.780097 00000108 UDF udfd LD_LIBRARY_PATH: ::/usr/lib 08/09 13:17:30.808162 00000108 UDF udfd is initialized 08/09 13:17:30.808186 00000084 DND node:dnode, has been opened 08/09 13:17:30.808201 00000084 DND node:mnode, start to open 08/09 13:17:30.808242 00000084 WAL wal module is initialized, rsetId:3 08/09 13:17:30.808385 00000084 DND succceed to read mnode file /var/lib/taos//mnode/mnode.json 08/09 13:17:30.808395 00000084 DND mnode start to open 08/09 13:17:30.808399 00000084 MND start to open mnode in /var/lib/taos//mnode 08/09 13:17:30.810668 00000084 WAL vgId:1, reset commitVer to 4410 08/09 13:17:30.811994 00000084 MND mnode-wal is initialized 08/09 13:17:30.812003 00000084 MND start to init sdb in /var/lib/taos//mnode 08/09 13:17:30.812016 00000084 MND sdb init success 08/09 13:17:30.812021 00000084 MND mnode-sdb is initialized 08/09 13:17:30.812029 00000084 MND sdb table:trans is initialized 08/09 13:17:30.812034 00000084 MND mnode-trans is initialized 08/09 13:17:30.812038 00000084 MND sdb table:cluster is initialized 08/09 13:17:30.812042 00000084 MND mnode-cluster is initialized 08/09 13:17:30.812045 00000084 MND sdb table:mnode is initialized 08/09 13:17:30.812049 00000084 MND mnode-mnode is initialized 08/09 13:17:30.812053 00000084 MND sdb table:qnode is initialized 08/09 13:17:30.812057 00000084 MND mnode-qnode is initialized 08/09 13:17:30.812085 00000084 MND sdb table:snode is initialized 08/09 13:17:30.812088 00000084 MND mnode-snode is initialized 08/09 13:17:30.812092 00000084 MND sdb table:dnode is initialized 08/09 13:17:30.812095 00000084 MND mnode-dnode is initialized 08/09 13:17:30.812098 00000084 MND sdb table:user is initialized 08/09 13:17:30.812100 00000084 MND mnode-user is initialized 08/09 13:17:30.812103 00000084 MND mnode-grant is initialized 08/09 13:17:30.812105 00000084 MND mnode-privilege is initialized 08/09 13:17:30.812109 00000084 MND sdb table:acct is initialized 08/09 13:17:30.812114 00000084 MND mnode-acct is initialized 08/09 13:17:30.812120 00000084 MND sdb table:stream is initialized 08/09 13:17:30.812122 00000084 MND mnode-stream is initialized 08/09 13:17:30.812128 00000084 MND sdb table:topic is initialized 08/09 13:17:30.812132 00000084 MND mnode-topic is initialized 08/09 13:17:30.812137 00000084 MND sdb table:consumer is initialized 08/09 13:17:30.812140 00000084 MND mnode-consumer is initialized 08/09 13:17:30.812143 00000084 MND sdb table:subscribe is initialized 08/09 13:17:30.812146 00000084 MND mnode-subscribe is initialized 08/09 13:17:30.812149 00000084 MND sdb table:vgroup is initialized 08/09 13:17:30.812151 00000084 MND mnode-vgroup is initialized 08/09 13:17:30.812156 00000084 MND sdb table:stb is initialized 08/09 13:17:30.812159 00000084 MND mnode-stb is initialized 08/09 13:17:30.812163 00000084 MND sdb table:sma is initialized 08/09 13:17:30.812164 00000084 MND mnode-sma is initialized 08/09 13:17:30.812167 00000084 MND sdb table:idx is initialized 08/09 13:17:30.812170 00000084 MND mnode-idx is initialized 08/09 13:17:30.812198 00000084 MND mnode-infos is initialized 08/09 13:17:30.812211 00000084 MND mnode-perfs is initialized 08/09 13:17:30.812217 00000084 MND sdb table:db is initialized 08/09 13:17:30.812220 00000084 MND mnode-db is initialized 08/09 13:17:30.812223 00000084 MND sdb table:func is initialized 08/09 13:17:30.812225 00000084 MND mnode-func is initialized 08/09 13:17:30.812229 00000084 MND start to reset sdb 08/09 13:17:30.812233 00000084 MND sdb:trans is reset 08/09 13:17:30.812237 00000084 MND sdb:cluster is reset 08/09 13:17:30.812239 00000084 MND sdb:mnode is reset 08/09 13:17:30.812241 00000084 MND sdb:qnode is reset 08/09 13:17:30.812244 00000084 MND sdb:snode is reset 08/09 13:17:30.812247 00000084 MND sdb:dnode is reset 08/09 13:17:30.812248 00000084 MND sdb:user is reset 08/09 13:17:30.812251 00000084 MND sdb:acct is reset 08/09 13:17:30.812254 00000084 MND sdb:stream is reset 08/09 13:17:30.812260 00000084 MND sdb:subscribe is reset 08/09 13:17:30.812262 00000084 MND sdb:consumer is reset 08/09 13:17:30.812264 00000084 MND sdb:topic is reset 08/09 13:17:30.812268 00000084 MND sdb:vgroup is reset 08/09 13:17:30.812270 00000084 MND sdb:sma is reset 08/09 13:17:30.812273 00000084 MND sdb:stb is reset 08/09 13:17:30.812275 00000084 MND sdb:db is reset 08/09 13:17:30.812279 00000084 MND sdb:func is reset 08/09 13:17:30.812281 00000084 MND sdb:idx is reset 08/09 13:17:30.812283 00000084 MND sdb reset success 08/09 13:17:30.812287 00000084 MND start to read sdb file:/var/lib/taos//mnode/data/sdb.data 08/09 13:17:30.812580 00000084 MND db:1.weather_repo, tsdbPageSize set from 4 to default 4 08/09 13:17:30.812606 00000084 MND vgId:1, has ready mnode:1, status:ready 08/09 13:17:30.812610 00000084 MND vgId:1, ep:taosd.base-component.svc.cluster.local:6030 dnode:1 08/09 13:17:30.812619 00000084 MND vgId:1, mnode sync not reconfig since readyMnodes:1 updatingMnodes:0 08/09 13:17:30.812624 00000084 MND read sdb file:/var/lib/taos//mnode/data/sdb.data success, commit index:4410 term:7 config:-1 08/09 13:17:30.812748 00000084 MND mnode-sdb is initialized 08/09 13:17:30.812789 00000084 MND mnode-profile is initialized 08/09 13:17:30.812796 00000084 MND mnode-show is initialized 08/09 13:17:30.813054 00000084 MND mnode-query is initialized 08/09 13:17:30.813058 00000084 MND vgId:1, start to open sync, replica:0 selfIndex:0 08/09 13:17:30.813351 00000084 SYN vgId:0, succceed to read sync cfg file /var/lib/taos//mnode/sync/raft_config.json 08/09 13:17:30.813470 00000084 SYN vgId:0, use sync config from sync cfg file 08/09 13:17:30.813474 00000084 SYN vgId:1, start to open sync node, replica:1 selfIndex:0 08/09 13:17:30.813476 00000084 SYN vgId:1, index:0 ep:taosd.base-component.svc.cluster.local:6030 dnode:1 cluster:4271663898657404236 08/09 13:17:30.813502 00000084 SYN vgId:1, sync addr:18398811190714171393, dnode:1 cluster:4271663898657404236 fqdn:taosd.base-component.svc.cluster.local ip:127.0.0.1 port:6030 ipv4:16777343 08/09 13:17:30.813514 00000084 SYN vgId:1, sync addr:18398811190714171393, dnode:1 cluster:4271663898657404236 fqdn:taosd.base-component.svc.cluster.local ip:127.0.0.1 port:6030 ipv4:16777343 08/09 13:17:30.813770 00000084 SYN vgId:1, succceed to read raft store file /var/lib/taos//mnode/sync/raft_store.json 08/09 13:17:30.813906 00000084 SYN vgId:1, sync node commitIndex initialized as 4410 08/09 13:17:30.818977 00000084 SYN vgId:1, init sync log buffer. buffer: [4410 4410 4600, 4601) 08/09 13:17:30.818989 00000084 SYN vgId:1, sync open, node:0x7f6949171000 electInterval:25000 heartbeatInterval:1000 heartbeatTimeout:20000, sync:follower, term:7, commit-index:4410, first-ver:2411, last-ver:4600, min:-1, snap:4410, snap-term:7, elect-times:0, as-leader-times:0, cfg-ch-times:0, hb-slow:0, hbr-slow:0, aq-items:-1, snaping:-1, replicas:1, last-cfg:-1, chging:0, restore:0, quorum:1, elect-lc-timer:0, hb:0, buffer:[4410 4410 4600, 4601), repl-mgrs:{0:0 [0 0, 0)}, members:{num:1, as:0, [taosd.base-component.svc.cluster.local:6030]}, hb:{0:1691558250813}, hb-reply:{0:1691558250813} 08/09 13:17:30.818997 00000084 MND mnode-sync is opened, id:2 08/09 13:17:30.818999 00000084 MND mnode-sync is initialized 08/09 13:17:30.819009 00000084 MND mnode-telem is initialized 08/09 13:17:30.819016 00000084 MND mnode open successfully 08/09 13:17:30.819020 00000084 UTL worker:mnode-query is initialized, min:4 max:4 08/09 13:17:30.819051 00000084 UTL worker:mnode-query:0 is launched, total:1 08/09 13:17:30.819058 00000117 UTL worker:mnode-query:0 is running, thread:00000117 08/09 13:17:30.819072 00000084 UTL worker:mnode-query:1 is launched, total:2 08/09 13:17:30.819083 00000118 UTL worker:mnode-query:1 is running, thread:00000118 08/09 13:17:30.819092 00000084 UTL worker:mnode-query:2 is launched, total:3 08/09 13:17:30.819101 00000119 UTL worker:mnode-query:2 is running, thread:00000119 08/09 13:17:30.819118 00000084 UTL worker:mnode-query:3 is launched, total:4 08/09 13:17:30.819120 00000084 UTL worker:mnode-query, queue:0x7f694dad5e80 is allocated, ahandle:0x7f694dabb300 08/09 13:17:30.819123 00000084 UTL worker:mnode-fetch is initialized, min:1 max:1 08/09 13:17:30.819125 00000120 UTL worker:mnode-query:3 is running, thread:00000120 08/09 13:17:30.819140 00000084 UTL worker:mnode-fetch:0 is launched, total:1 08/09 13:17:30.819143 00000084 UTL worker:mnode-fetch, queue:0x7f694dad5f00 is allocated, ahandle:0x7f694dabb300 08/09 13:17:30.819145 00000084 UTL worker:mnode-read is initialized, min:4 max:4 08/09 13:17:30.819149 00000121 UTL worker:mnode-fetch:0 is running, thread:00000121 08/09 13:17:30.819165 00000084 UTL worker:mnode-read:0 is launched, total:1 08/09 13:17:30.819176 00000122 UTL worker:mnode-read:0 is running, thread:00000122 08/09 13:17:30.819184 00000084 UTL worker:mnode-read:1 is launched, total:2 08/09 13:17:30.819193 00000123 UTL worker:mnode-read:1 is running, thread:00000123 08/09 13:17:30.819208 00000084 UTL worker:mnode-read:2 is launched, total:3 08/09 13:17:30.819218 00000124 UTL worker:mnode-read:2 is running, thread:00000124 08/09 13:17:30.819230 00000084 UTL worker:mnode-read:3 is launched, total:4 08/09 13:17:30.819232 00000084 UTL worker:mnode-read, queue:0x7f694a4eb000 is allocated, ahandle:0x7f694dabb300 08/09 13:17:30.819234 00000084 UTL worker:mnode-write is initialized, min:1 max:1 08/09 13:17:30.819240 00000125 UTL worker:mnode-read:3 is running, thread:00000125 08/09 13:17:30.819251 00000084 UTL worker:mnode-write:0 is launched, total:1 08/09 13:17:30.819253 00000084 UTL worker:mnode-write, queue:0x7f694a4eb080 is allocated, ahandle:0x7f694dabb300 08/09 13:17:30.819255 00000084 UTL worker:mnode-sync is initialized, min:1 max:1 08/09 13:17:30.819259 00000126 UTL worker:mnode-write:0 is running, thread:00000126 08/09 13:17:30.819276 00000084 UTL worker:mnode-sync:0 is launched, total:1 08/09 13:17:30.819279 00000084 UTL worker:mnode-sync, queue:0x7f694a4eb100 is allocated, ahandle:0x7f694dabb300 08/09 13:17:30.819281 00000084 UTL worker:mnode-sync-ctrl is initialized, min:1 max:1 08/09 13:17:30.819284 00000127 UTL worker:mnode-sync:0 is running, thread:00000127 08/09 13:17:30.819308 00000084 UTL worker:mnode-sync-ctrl:0 is launched, total:1 08/09 13:17:30.819311 00000084 UTL worker:mnode-sync-ctrl, queue:0x7f694a4eb180 is allocated, ahandle:0x7f694dabb300 08/09 13:17:30.819313 00000084 DND node:mnode, has been opened 08/09 13:17:30.819316 00000128 UTL worker:mnode-sync-ctrl:0 is running, thread:00000128 08/09 13:17:30.819321 00000084 DND node:vnode, start to open 08/09 13:17:30.819886 00000084 UTL worker:vnode-query is initialized, min:96 max:96 08/09 13:17:30.819892 00000084 UTL worker:vnode-stream is initialized as auto 08/09 13:17:30.819895 00000084 UTL worker:vnode-fetch is initialized, max:12 08/09 13:17:30.819897 00000084 UTL worker:vnode-mgmt is initialized, min:1 max:1 08/09 13:17:30.819916 00000084 UTL worker:vnode-mgmt:0 is launched, total:1 08/09 13:17:30.819918 00000084 UTL worker:vnode-mgmt, queue:0x7f694a4eb200 is allocated, ahandle:0x7f69490aee00 08/09 13:17:30.819946 00000133 UTL worker:vnode-mgmt:0 is running, thread:00000133 08/09 13:17:30.820308 00000084 DND succceed to read vnode file /var/lib/taos//vnode/vnodes.json 08/09 13:17:30.820482 00000084 DND open 2 vnodes with 24 threads 08/09 13:17:30.820520 00000134 DND thread:0, start to open 1 vnodes 08/09 13:17:30.820528 00000135 DND thread:1, start to open 1 vnodes 08/09 13:17:30.919417 00000134 TSD ERROR vgId:2, tsdbScanAndTryFixFS failed at line 214 since Data file corrupted, fid:1955 08/09 13:17:30.919430 00000134 TSD ERROR vgId:2, tsdbFSOpen failed at line 794 since Data file corrupted 08/09 13:17:30.919434 00000134 VND ERROR vgId:2, failed to open vnode tsdb since success 08/09 13:17:30.919598 00000135 TSD ERROR vgId:3, tsdbScanAndTryFixFS failed at line 229 since Data file corrupted, fid:1955 08/09 13:17:30.919609 00000135 TSD ERROR vgId:3, tsdbFSOpen failed at line 794 since Data file corrupted 08/09 13:17:30.919613 00000135 VND ERROR vgId:3, failed to open vnode tsdb since success 08/09 13:17:30.919748 00000134 DND ERROR vgId:2, failed to open vnode by thread:0 08/09 13:17:30.919752 00000134 DND thread:0, numOfVnodes:1, opened:0 failed:1 08/09 13:17:30.919855 00000135 DND ERROR vgId:3, failed to open vnode by thread:1 08/09 13:17:30.919860 00000135 DND thread:1, numOfVnodes:1, opened:0 failed:1 08/09 13:17:30.919968 00000084 DND ERROR there are total vnodes:2, opened:0 08/09 13:17:30.919975 00000084 DND ERROR failed to open vnode since success 08/09 13:17:30.919977 00000084 DND ERROR failed to init vnodes-mgmt since success 08/09 13:17:30.919979 00000084 DND start to close all vnodes 08/09 13:17:30.919987 00000084 UTL worker:vnode-mgmt:0 is stopping 08/09 13:17:30.919991 00000133 UTL worker:vnode-mgmt:0 qset:0x7f694da47360, got no message and exiting, thread:00000133 08/09 13:17:30.920009 00000084 UTL worker:vnode-mgmt:0 is stopped 08/09 13:17:30.920015 00000084 UTL worker:vnode-mgmt is closed 08/09 13:17:30.920017 00000084 UTL worker:vnode-mgmt, queue:0x7f694a4eb200 is freed 08/09 13:17:30.920019 00000084 DND vnodes mgmt worker is stopped 08/09 13:17:30.920025 00000084 DND close 0 vnodes with 24 threads 08/09 13:17:30.920030 00000084 DND total vnodes:0 are all closed 08/09 13:17:30.920034 00000084 UTL worker:vnode-query is closed 08/09 13:17:30.920036 00000084 UTL worker:vnode-stream is closed 08/09 13:17:30.920038 00000084 UTL worker:vnode-fetch is closed 08/09 13:17:31.808341 00000084 WAL wal module is cleaned up 08/09 13:17:31.808377 00000084 DND ERROR node:vnode, failed to open since success 08/09 13:17:31.808381 00000084 DND ERROR node:vnode, failed to open since success 08/09 13:17:31.808383 00000084 DND ERROR failed to open nodes since success 08/09 13:17:31.808385 00000084 DND shutting down the service 08/09 13:17:32.010549 00000084 UDF udfd start to stop, need cleanup:1, spawn err:0 08/09 13:17:32.010666 00000084 UDF udfd is cleaned up 08/09 13:17:32.313003 00000084 DND dnode env is cleaned up

根据日志，发现/var/lib/taos/vnode 下的vnode2和vnode3的tsdb下的文件都有损坏，导致无法启动

Screenshots

Environment

K8S v1.25.5
Memory 125G, CPU 48, current Disk Space 16T
TDengine Version 3.0.3.1

taosdata / TDengine

TDengine 3.0.3.1单机版服务器断电后文件损坏 #22378