apache / rocketmq

Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.
https://rocketmq.apache.org/
Apache License 2.0
21.11k stars 11.63k forks source link

[Bug] slave node did not synchronize topicQueueTable #8539

Open yx9o opened 4 weeks ago

yx9o commented 4 weeks ago

Before Creating the Bug Report

Runtime platform environment

Linux

RocketMQ version

4.9.8

JDK Version

jdk 1.8

Describe the Bug

I want to implement master-slave switching through commands. After the master stops, I change the slave configuration to master. The offset produced is not continuous, which means that the data during the period of catching up with the consumption progress cannot be consumed.

image image image image

Steps to Reproduce

  1. Run a group of master-slave production and consumption data
  2. Stop the master and change the slave configuration to the master through commands
  3. The offset of the production message seen at this time is not continuous, while the offset of consumption is continuous

What Did You Expect to See?

After the slave node switches to the master, the offset produced is continuous.

What Did You See Instead?

The offset produced is not continuous.

Additional Context

No response

humkum commented 4 weeks ago

Slave node synchronization is a scheduled task. There is a time interval for synchronization. There may be some cases where there is no synchronization, but it should not affect

yx9o commented 3 weeks ago

Slave node synchronization is a scheduled task. There is a time interval for synchronization. There may be some cases where there is no synchronization, but it should not affect

Those two tables are not in the synchronization task.

humkum commented 2 weeks ago

offsetTable is in scheduled task, see syncAll() , lmqOffsetTable is not in scheduled task. lmqOffsetTable synchronization should fix in ISSUE #8509

yx9o commented 2 weeks ago

What has been synchronized is the consumption progress. The problem I am describing here is the offset returned when sending a message. These are two different things.

humkum commented 2 weeks ago

Sorry, it's my fault.In this case, it seems that the data from the master node has not been synchronized to slave successfully, otherwise the offset should be continuous.

yx9o commented 2 weeks ago

If you are interested, you can test it and you will understand.

David-on-Code commented 4 hours ago

Can you post the detailed master-slave switching process and logs?