dyrnq / docker-canal

docker-canal
https://github.com/alibaba/canal
0 stars 0 forks source link

!!!canal.instance.mysql.slaveId=0 mysql serverId , v1.0.26+ will autoGen #17

Open dyrnq opened 1 month ago

dyrnq commented 1 month ago
## mysql serverId , v1.0.26+ will autoGen
# canal.instance.mysql.slaveId=0
dyrnq commented 1 month ago

/otter/canal/destinations/test/1001/cursor

{
    "@type":"com.alibaba.otter.canal.protocol.position.LogPosition",
    "identity":{
        "slaveId":-1,
        "sourceAddress":{
            "address":"main-db5",
            "port":3306
        }
    },
    "postion":{
        "gtid":"",
        "included":false,
        "journalName":"mysql-bin.000003",
        "position":1790,
        "serverId":1,
        "timestamp":1729425993000
    }
}
dyrnq commented 1 month ago

slaveId=-1?

dyrnq commented 1 month ago
mysql> show slave hosts;
+-----------+---------------+-------+-----------+--------------------------------------+
| Server_id | Host          | Port  | Master_id | Slave_UUID                           |
+-----------+---------------+-------+-----------+--------------------------------------+
| 316670097 | 172.224.0.145 | 34648 |         1 | a092c717-8f41-11ef-96cb-0242ace00007 |
+-----------+---------------+-------+-----------+--------------------------------------+
1 row in set (0.00 sec)

mysql> SHOW VARIABLES LIKE 'slave%';
+------------------------------+-----------------------+
| Variable_name                | Value                 |
+------------------------------+-----------------------+
| slave_allow_batching         | OFF                   |
| slave_checkpoint_group       | 512                   |
| slave_checkpoint_period      | 300                   |
| slave_compressed_protocol    | OFF                   |
| slave_exec_mode              | STRICT                |
| slave_load_tmpdir            | /tmp                  |
| slave_max_allowed_packet     | 1073741824            |
| slave_net_timeout            | 60                    |
| slave_parallel_type          | DATABASE              |
| slave_parallel_workers       | 0                     |
| slave_pending_jobs_size_max  | 16777216              |
| slave_preserve_commit_order  | OFF                   |
| slave_rows_search_algorithms | TABLE_SCAN,INDEX_SCAN |
| slave_skip_errors            | OFF                   |
| slave_sql_verify_checksum    | ON                    |
| slave_transaction_retries    | 10                    |
| slave_type_conversions       |                       |
+------------------------------+-----------------------+
17 rows in set (0.00 sec)

mysql> 
mysql> SHOW BINARY LOGS;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000001 |       177 |
| mysql-bin.000002 |   2949916 |
| mysql-bin.000003 |       177 |
| mysql-bin.000004 |       177 |
| mysql-bin.000005 |      1865 |
| mysql-bin.000006 |       154 |
| mysql-bin.000007 |      5010 |
| mysql-bin.000008 |       177 |
| mysql-bin.000009 |       177 |
| mysql-bin.000010 |      3486 |
| mysql-bin.000011 |       154 |
| mysql-bin.000012 |       177 |
| mysql-bin.000013 |       154 |
| mysql-bin.000014 |      1973 |
| mysql-bin.000015 |       154 |
| mysql-bin.000016 |       177 |
| mysql-bin.000017 |       154 |
+------------------+-----------+
17 rows in set (0.00 sec)
dyrnq commented 1 month ago
canal.instance.mysql.slaveId=5555555
mysql> show slave hosts;
+-----------+---------------+-------+-----------+--------------------------------------+
| Server_id | Host          | Port  | Master_id | Slave_UUID                           |
+-----------+---------------+-------+-----------+--------------------------------------+
|   5555555 | 172.224.0.155 | 45884 |         1 | 040cebd2-8f45-11ef-96cb-0242ace00007 |
| 316670097 | 172.224.0.145 | 34648 |         1 | a092c717-8f41-11ef-96cb-0242ace00007 |
+-----------+---------------+-------+-----------+--------------------------------------+
2 rows in set (0.00 sec)

因此,SHOW SLAVE HOSTS 中的 Server_id 确实是指从服务器的 ID(slave ID),用于唯一标识每个从服务器,并在复制配置中起到关键作用。

dyrnq commented 1 month ago

似乎根据${instance}的名字+"journalName":"mysql-bin.000003"+ "position":1790就可以锁定状态,slaveId固定不固定已经意义不大了?

dyrnq commented 4 weeks ago
        // 随机生成slaveId
        if (this.slaveId <= 0) {
            this.slaveId = generateUniqueServerId();
        }
    private final long generateUniqueServerId() {
        try {
            // a=`echo $masterip|cut -d\. -f1`
            // b=`echo $masterip|cut -d\. -f2`
            // c=`echo $masterip|cut -d\. -f3`
            // d=`echo $masterip|cut -d\. -f4`
            // #server_id=`expr $a \* 256 \* 256 \* 256 + $b \* 256 \* 256 + $c
            // \* 256 + $d `
            // #server_id=$b$c$d
            // server_id=`expr $b \* 256 \* 256 + $c \* 256 + $d `
            InetAddress localHost = InetAddress.getLocalHost();
            byte[] addr = localHost.getAddress();
            int salt = (destination != null) ? destination.hashCode() : 0;
            return ((0x7f & salt) << 24) + ((0xff & (int) addr[1]) << 16) // NL
                   + ((0xff & (int) addr[2]) << 8) // NL
                   + (0xff & (int) addr[3]);
        } catch (UnknownHostException e) {
            throw new CanalParseException("Unknown host", e);
        }
    }
dyrnq commented 4 weeks ago

自动生成的slaveId似乎和ipv4地址相关的,这在k8s上pod ip不固定的情况下,是不大合适的,虽然zookeeper记录在不断的保存持久化了位点。