apache / rocketmq-operator

Apache RocketMQ Operator
https://rocketmq.apache.org/
Apache License 2.0
314 stars 127 forks source link

rocketmq-console broker-0(master)经常不显示 #158

Open songchaohua opened 1 year ago

songchaohua commented 1 year ago

使用rocketmq-operator 0.3.0 部署rocketmq 4.9.4集群2master2slave异步复制模式,部署没有问题,集群没有问题,但是显示有问题,如下图: 集群状态正常: image 控制台经常少broker-0(master) image 也有正确显示的时候: image

caigy commented 1 year ago

It seems something is wrong in registration of broker-0 slave, you can check if there are some error messages in the broker logs.

songchaohua commented 1 year ago

看哪个日志文件? image

caigy commented 1 year ago

看哪个日志文件? image

try broker.log first.

songchaohua commented 1 year ago

2023-05-27 14:02:26 INFO main - rocketmqHome=/root/rocketmq/broker 2023-05-27 14:02:26 INFO main - namesrvAddr=name-server-service.adk:9876 2023-05-27 14:02:26 INFO main - brokerIP1=10.241.115.63 2023-05-27 14:02:26 INFO main - brokerIP2=10.241.115.63 2023-05-27 14:02:26 INFO main - brokerName=broker-0 2023-05-27 14:02:26 INFO main - brokerClusterName=broker 2023-05-27 14:02:26 INFO main - brokerId=0 2023-05-27 14:02:26 INFO main - brokerPermission=6 2023-05-27 14:02:26 INFO main - defaultTopicQueueNums=8 2023-05-27 14:02:26 INFO main - autoCreateTopicEnable=true 2023-05-27 14:02:26 INFO main - clusterTopicEnable=true 2023-05-27 14:02:26 INFO main - brokerTopicEnable=true 2023-05-27 14:02:26 INFO main - autoCreateSubscriptionGroup=true 2023-05-27 14:02:26 INFO main - messageStorePlugIn= 2023-05-27 14:02:26 INFO main - msgTraceTopicName=RMQ_SYS_TRACE_TOPIC 2023-05-27 14:02:26 INFO main - traceTopicEnable=false 2023-05-27 14:02:26 INFO main - sendMessageThreadPoolNums=1 2023-05-27 14:02:26 INFO main - putMessageFutureThreadPoolNums=1 2023-05-27 14:02:26 INFO main - pullMessageThreadPoolNums=18 2023-05-27 14:02:26 INFO main - processReplyMessageThreadPoolNums=18 2023-05-27 14:02:26 INFO main - queryMessageThreadPoolNums=9 2023-05-27 14:02:26 INFO main - adminBrokerThreadPoolNums=16 2023-05-27 14:02:26 INFO main - clientManageThreadPoolNums=32 2023-05-27 14:02:26 INFO main - consumerManageThreadPoolNums=32 2023-05-27 14:02:26 INFO main - heartbeatThreadPoolNums=1 2023-05-27 14:02:26 INFO main - endTransactionThreadPoolNums=10 2023-05-27 14:02:26 INFO main - flushConsumerOffsetInterval=5000 2023-05-27 14:02:26 INFO main - flushConsumerOffsetHistoryInterval=60000 2023-05-27 14:02:26 INFO main - rejectTransactionMessage=false 2023-05-27 14:02:26 INFO main - fetchNamesrvAddrByAddressServer=false 2023-05-27 14:02:26 INFO main - sendThreadPoolQueueCapacity=10000 2023-05-27 14:02:26 INFO main - putThreadPoolQueueCapacity=10000 2023-05-27 14:02:26 INFO main - pullThreadPoolQueueCapacity=100000 2023-05-27 14:02:26 INFO main - replyThreadPoolQueueCapacity=10000 2023-05-27 14:02:26 INFO main - queryThreadPoolQueueCapacity=20000 2023-05-27 14:02:26 INFO main - clientManagerThreadPoolQueueCapacity=1000000 2023-05-27 14:02:26 INFO main - consumerManagerThreadPoolQueueCapacity=1000000 2023-05-27 14:02:26 INFO main - heartbeatThreadPoolQueueCapacity=50000 2023-05-27 14:02:26 INFO main - endTransactionPoolQueueCapacity=100000 2023-05-27 14:02:26 INFO main - filterServerNums=0 2023-05-27 14:02:26 INFO main - longPollingEnable=true 2023-05-27 14:02:26 INFO main - shortPollingTimeMills=1000 2023-05-27 14:02:26 INFO main - notifyConsumerIdsChangedEnable=true 2023-05-27 14:02:26 INFO main - highSpeedMode=false 2023-05-27 14:02:26 INFO main - commercialEnable=true 2023-05-27 14:02:26 INFO main - commercialTimerCount=1 2023-05-27 14:02:26 INFO main - commercialTransCount=1 2023-05-27 14:02:26 INFO main - commercialBigCount=1 2023-05-27 14:02:26 INFO main - commercialBaseCount=1 2023-05-27 14:02:26 INFO main - transferMsgByHeap=true 2023-05-27 14:02:26 INFO main - maxDelayTime=40 2023-05-27 14:02:26 INFO main - regionId=DefaultRegion 2023-05-27 14:02:26 INFO main - registerBrokerTimeoutMills=6000 2023-05-27 14:02:26 INFO main - slaveReadEnable=false 2023-05-27 14:02:26 INFO main - disableConsumeIfConsumerReadSlowly=false 2023-05-27 14:02:26 INFO main - consumerFallbehindThreshold=17179869184 2023-05-27 14:02:26 INFO main - brokerFastFailureEnable=true 2023-05-27 14:02:26 INFO main - waitTimeMillsInSendQueue=200 2023-05-27 14:02:26 INFO main - waitTimeMillsInPullQueue=5000 2023-05-27 14:02:26 INFO main - waitTimeMillsInHeartbeatQueue=31000 2023-05-27 14:02:26 INFO main - waitTimeMillsInTransactionQueue=3000 2023-05-27 14:02:26 INFO main - startAcceptSendRequestTimeStamp=0 2023-05-27 14:02:26 INFO main - traceOn=true 2023-05-27 14:02:26 INFO main - enableCalcFilterBitMap=false 2023-05-27 14:02:26 INFO main - expectConsumerNumUseFilter=32 2023-05-27 14:02:26 INFO main - maxErrorRateOfBloomFilter=20 2023-05-27 14:02:26 INFO main - filterDataCleanTimeSpan=86400000 2023-05-27 14:02:26 INFO main - filterSupportRetry=false 2023-05-27 14:02:26 INFO main - enablePropertyFilter=false 2023-05-27 14:02:26 INFO main - compressedRegister=false 2023-05-27 14:02:26 INFO main - forceRegister=true 2023-05-27 14:02:26 INFO main - registerNameServerPeriod=30000 2023-05-27 14:02:26 INFO main - transactionTimeOut=6000 2023-05-27 14:02:26 INFO main - transactionCheckMax=15 2023-05-27 14:02:26 INFO main - transactionCheckInterval=60000 2023-05-27 14:02:26 INFO main - aclEnable=false 2023-05-27 14:02:26 INFO main - storeReplyMessageEnable=true 2023-05-27 14:02:26 INFO main - enableDetailStat=true 2023-05-27 14:02:26 INFO main - autoDeleteUnusedStats=false 2023-05-27 14:02:26 INFO main - isolateLogEnable=false 2023-05-27 14:02:26 INFO main - listenPort=10911 2023-05-27 14:02:26 INFO main - serverWorkerThreads=8 2023-05-27 14:02:26 INFO main - serverCallbackExecutorThreads=0 2023-05-27 14:02:26 INFO main - serverSelectorThreads=3 2023-05-27 14:02:26 INFO main - serverOnewaySemaphoreValue=256 2023-05-27 14:02:26 INFO main - serverAsyncSemaphoreValue=64 2023-05-27 14:02:26 INFO main - serverChannelMaxIdleTimeSeconds=120 2023-05-27 14:02:26 INFO main - serverSocketSndBufSize=0 2023-05-27 14:02:26 INFO main - serverSocketRcvBufSize=0 2023-05-27 14:02:26 INFO main - writeBufferHighWaterMark=0 2023-05-27 14:02:26 INFO main - writeBufferLowWaterMark=0 2023-05-27 14:02:26 INFO main - serverSocketBacklog=1024 2023-05-27 14:02:26 INFO main - serverPooledByteBufAllocatorEnable=true 2023-05-27 14:02:26 INFO main - useEpollNativeSelector=false 2023-05-27 14:02:26 INFO main - clientWorkerThreads=4 2023-05-27 14:02:26 INFO main - clientCallbackExecutorThreads=1 2023-05-27 14:02:26 INFO main - clientOnewaySemaphoreValue=65535 2023-05-27 14:02:26 INFO main - clientAsyncSemaphoreValue=65535 2023-05-27 14:02:26 INFO main - connectTimeoutMillis=3000 2023-05-27 14:02:26 INFO main - channelNotActiveInterval=60000 2023-05-27 14:02:26 INFO main - clientChannelMaxIdleTimeSeconds=120 2023-05-27 14:02:26 INFO main - clientSocketSndBufSize=0 2023-05-27 14:02:26 INFO main - clientSocketRcvBufSize=0 2023-05-27 14:02:26 INFO main - clientPooledByteBufAllocatorEnable=false 2023-05-27 14:02:26 INFO main - clientCloseSocketIfTimeout=true 2023-05-27 14:02:26 INFO main - useTLS=false 2023-05-27 14:02:26 INFO main - writeBufferHighWaterMark=0 2023-05-27 14:02:26 INFO main - writeBufferLowWaterMark=0 2023-05-27 14:02:26 INFO main - storePathRootDir=/root/store 2023-05-27 14:02:26 INFO main - storePathCommitLog= 2023-05-27 14:02:26 INFO main - storePathDLedgerCommitLog= 2023-05-27 14:02:26 INFO main - readOnlyCommitLogStorePaths= 2023-05-27 14:02:26 INFO main - mappedFileSizeCommitLog=1073741824 2023-05-27 14:02:26 INFO main - mappedFileSizeConsumeQueue=6000000 2023-05-27 14:02:26 INFO main - enableConsumeQueueExt=false 2023-05-27 14:02:26 INFO main - mappedFileSizeConsumeQueueExt=50331648 2023-05-27 14:02:26 INFO main - bitMapLengthConsumeQueueExt=64 2023-05-27 14:02:26 INFO main - flushIntervalCommitLog=500 2023-05-27 14:02:26 INFO main - commitIntervalCommitLog=200 2023-05-27 14:02:26 INFO main - useReentrantLockWhenPutMessage=true 2023-05-27 14:02:26 INFO main - flushCommitLogTimed=true 2023-05-27 14:02:26 INFO main - flushIntervalConsumeQueue=1000 2023-05-27 14:02:26 INFO main - cleanResourceInterval=10000 2023-05-27 14:02:26 INFO main - deleteCommitLogFilesInterval=100 2023-05-27 14:02:26 INFO main - deleteConsumeQueueFilesInterval=100 2023-05-27 14:02:26 INFO main - destroyMapedFileIntervalForcibly=120000 2023-05-27 14:02:26 INFO main - redeleteHangedFileInterval=120000 2023-05-27 14:02:26 INFO main - deleteWhen=04 2023-05-27 14:02:26 INFO main - diskMaxUsedSpaceRatio=75 2023-05-27 14:02:26 INFO main - fileReservedTime=48 2023-05-27 14:02:26 INFO main - putMsgIndexHightWater=600000 2023-05-27 14:02:26 INFO main - maxMessageSize=4194304 2023-05-27 14:02:26 INFO main - checkCRCOnRecover=true 2023-05-27 14:02:26 INFO main - flushCommitLogLeastPages=4 2023-05-27 14:02:26 INFO main - commitCommitLogLeastPages=4 2023-05-27 14:02:26 INFO main - flushLeastPagesWhenWarmMapedFile=4096 2023-05-27 14:02:26 INFO main - flushConsumeQueueLeastPages=2 2023-05-27 14:02:26 INFO main - flushCommitLogThoroughInterval=10000 2023-05-27 14:02:26 INFO main - commitCommitLogThoroughInterval=200 2023-05-27 14:02:26 INFO main - flushConsumeQueueThoroughInterval=60000 2023-05-27 14:02:26 INFO main - maxTransferBytesOnMessageInMemory=262144 2023-05-27 14:02:26 INFO main - maxTransferCountOnMessageInMemory=32 2023-05-27 14:02:26 INFO main - maxTransferBytesOnMessageInDisk=65536 2023-05-27 14:02:26 INFO main - maxTransferCountOnMessageInDisk=8 2023-05-27 14:02:26 INFO main - accessMessageInMemoryMaxRatio=40 2023-05-27 14:02:26 INFO main - messageIndexEnable=true 2023-05-27 14:02:26 INFO main - maxHashSlotNum=5000000 2023-05-27 14:02:26 INFO main - maxIndexNum=20000000 2023-05-27 14:02:26 INFO main - maxMsgsNumBatch=64 2023-05-27 14:02:26 INFO main - messageIndexSafe=false 2023-05-27 14:02:26 INFO main - haListenPort=10912 2023-05-27 14:02:26 INFO main - haSendHeartbeatInterval=5000 2023-05-27 14:02:26 INFO main - haHousekeepingInterval=20000 2023-05-27 14:02:26 INFO main - haTransferBatchSize=32768 2023-05-27 14:02:26 INFO main - haMasterAddress= 2023-05-27 14:02:26 INFO main - haSlaveFallbehindMax=268435456 2023-05-27 14:02:26 INFO main - brokerRole=ASYNC_MASTER 2023-05-27 14:02:26 INFO main - flushDiskType=ASYNC_FLUSH 2023-05-27 14:02:26 INFO main - syncFlushTimeout=5000 2023-05-27 14:02:26 INFO main - slaveTimeout=3000 2023-05-27 14:02:26 INFO main - messageDelayLevel=1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h 2023-05-27 14:02:26 INFO main - flushDelayOffsetInterval=10000 2023-05-27 14:02:26 INFO main - cleanFileForciblyEnable=true 2023-05-27 14:02:26 INFO main - warmMapedFileEnable=false 2023-05-27 14:02:26 INFO main - offsetCheckInSlave=false 2023-05-27 14:02:26 INFO main - debugLockEnable=false 2023-05-27 14:02:26 INFO main - duplicationEnable=false 2023-05-27 14:02:26 INFO main - diskFallRecorded=true 2023-05-27 14:02:26 INFO main - osPageCacheBusyTimeOutMills=1000 2023-05-27 14:02:26 INFO main - defaultQueryMaxNum=32 2023-05-27 14:02:26 INFO main - transientStorePoolEnable=false 2023-05-27 14:02:26 INFO main - transientStorePoolSize=5 2023-05-27 14:02:26 INFO main - fastFailIfNoBufferInStorePool=false 2023-05-27 14:02:26 INFO main - enableDLegerCommitLog=false 2023-05-27 14:02:26 INFO main - dLegerGroup= 2023-05-27 14:02:26 INFO main - dLegerPeers= 2023-05-27 14:02:26 INFO main - dLegerSelfId= 2023-05-27 14:02:26 INFO main - preferredLeaderId= 2023-05-27 14:02:26 INFO main - isEnableBatchPush=false 2023-05-27 14:02:26 INFO main - enableScheduleMessageStats=true 2023-05-27 14:02:26 INFO main - enableLmq=false 2023-05-27 14:02:26 INFO main - enableMultiDispatch=false 2023-05-27 14:02:26 INFO main - maxLmqConsumeQueueNum=20000 2023-05-27 14:02:26 INFO main - enableScheduleAsyncDeliver=false 2023-05-27 14:02:26 INFO main - scheduleAsyncDeliverMaxPendingLimit=2000 2023-05-27 14:02:26 INFO main - scheduleAsyncDeliverMaxResendNum2Blocked=3 2023-05-27 14:02:27 INFO main - load exist local topic, TopicConfig [topicName=SCHEDULE_TOPIC_XXXX, readQueueNums=18, writeQueueNums=18, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false] 2023-05-27 14:02:27 INFO main - load exist local topic, TopicConfig [topicName=RMQ_SYS_TRANS_HALF_TOPIC, readQueueNums=1, writeQueueNums=1, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false] 2023-05-27 14:02:27 INFO main - load exist local topic, TopicConfig [topicName=broker_REPLY_TOPIC, readQueueNums=1, writeQueueNums=1, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false] 2023-05-27 14:02:27 INFO main - load exist local topic, TopicConfig [topicName=BenchmarkTest, readQueueNums=1024, writeQueueNums=1024, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false] 2023-05-27 14:02:27 INFO main - load exist local topic, TopicConfig [topicName=OFFSET_MOVED_EVENT, readQueueNums=1, writeQueueNums=1, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false] 2023-05-27 14:02:27 INFO main - load exist local topic, TopicConfig [topicName=broker-0, readQueueNums=1, writeQueueNums=1, perm=RWX, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false] 2023-05-27 14:02:27 INFO main - load exist local topic, TopicConfig [topicName=TBW102, readQueueNums=8, writeQueueNums=8, perm=RWX, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false] 2023-05-27 14:02:27 INFO main - load exist local topic, TopicConfig [topicName=broker, readQueueNums=16, writeQueueNums=16, perm=RWX, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false] 2023-05-27 14:02:27 INFO main - load exist local topic, TopicConfig [topicName=SELF_TEST_TOPIC, readQueueNums=1, writeQueueNums=1, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false] 2023-05-27 14:02:27 INFO main - load /root/store/config/topics.json OK 2023-05-27 14:02:27 INFO main - load /root/store/config/consumerOffset.json OK 2023-05-27 14:02:27 INFO main - load /root/store/config/consumerFilter.json OK 2023-05-27 14:02:28 INFO main - Try to start service thread:AllocateMappedFileService started:false lastThread:null 2023-05-27 14:02:28 INFO main - load /root/store/config/delayOffset.json OK 2023-05-27 14:02:32 INFO main - Set user specified name server address: name-server-service.adk:9876 2023-05-27 14:02:32 WARN main - Load default transaction message hook service: TransactionalMessageServiceImpl 2023-05-27 14:02:32 WARN main - Load default discard message hook service: DefaultTransactionalMessageCheckListener 2023-05-27 14:02:32 INFO main - The broker dose not enable acl 2023-05-27 14:02:32 INFO main - Try to start service thread:ReputMessageService started:false lastThread:null 2023-05-27 14:02:32 INFO main - Try to start service thread:AcceptSocketService started:false lastThread:null 2023-05-27 14:02:32 INFO main - Try to start service thread:GroupTransferService started:false lastThread:null 2023-05-27 14:02:32 INFO main - Try to start service thread:HAClient started:false lastThread:null 2023-05-27 14:02:32 INFO main - load /root/store/config/delayOffset.json OK 2023-05-27 14:02:32 INFO main - Try to start service thread:FlushConsumeQueueService started:false lastThread:null 2023-05-27 14:02:32 INFO main - Try to start service thread:FlushRealTimeService started:false lastThread:null 2023-05-27 14:02:32 INFO main - Try to start service thread:FlushDiskWatcher started:false lastThread:null 2023-05-27 14:02:32 INFO main - Try to start service thread:StoreStatsService started:false lastThread:null 2023-05-27 14:02:32 INFO main - Try to start service thread:FileWatchService started:false lastThread:null 2023-05-27 14:02:32 INFO FileWatchService - FileWatchService service started 2023-05-27 14:02:32 INFO main - Try to start service thread:PullRequestHoldService started:false lastThread:null 2023-05-27 14:02:32 INFO PullRequestHoldService - PullRequestHoldService service started 2023-05-27 14:02:32 INFO main - Try to start service thread:TransactionalMessageCheckService started:false lastThread:null 2023-05-27 14:02:33 INFO brokerOutApi_thread_1 - register broker[0]to name server name-server-service.adk:9876 OK 2023-05-27 14:02:33 INFO main - The broker[broker-0, 10.241.115.63:10911] boot success. serializeType=JSON and name server is name-server-service.adk:9876 2023-05-27 14:02:42 INFO BrokerControllerScheduledThread1 - dispatch behind commit log 0 bytes 2023-05-27 14:02:42 INFO BrokerControllerScheduledThread1 - Slave fall behind master: 0 bytes

songchaohua commented 1 year ago

帮我看看日志哪里有问题?

jackyguhj commented 1 year ago

I resolved it by not setting the broker's nameserver and leaving it blank

caigy commented 1 year ago

@songchaohua Did you set namesrvAddr manually?

2023-05-27 14:02:26 INFO main - namesrvAddr=name-server-service.adk:9876