ogata-ryosuke commented 1 year ago

run meilisync start on Docker (with Laravel)

docker error logs (mysql:8-debian)

backend-meilisync-1     | 2023-10-26 00:23:04.096 | ERROR    | meilisync.source.mysql:__aiter__:121 - Binlog stream error: (1236, "A replica with the same server_uuid/server_id as this replica has connected to the source; the first event 'binlog.000028' at 157, the last event read from './binlog.000028' at 126, the last byte read from './binlog.000028' at 157."), restart...
backend-meilisync-1     | Traceback (most recent call last):
backend-meilisync-1     |
backend-meilisync-1     |   File "/usr/local/bin/meilisync", line 6, in <module>
backend-meilisync-1     |     sys.exit(app())
backend-meilisync-1     |     │   │    └ <typer.main.Typer object at 0x4001f89790>
backend-meilisync-1     |     │   └ <built-in function exit>
backend-meilisync-1     |     └ <module 'sys' (built-in)>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/typer/main.py", line 311, in __call__
backend-meilisync-1     |     return get_command(self)(*args, **kwargs)
backend-meilisync-1     |            │           │      │       └ {}
backend-meilisync-1     |            │           │      └ ()
backend-meilisync-1     |            │           └ <typer.main.Typer object at 0x4001f89790>
backend-meilisync-1     |            └ <function get_command at 0x400347e700>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
backend-meilisync-1     |     return self.main(*args, **kwargs)
backend-meilisync-1     |            │    │     │       └ {}
backend-meilisync-1     |            │    │     └ ()
backend-meilisync-1     |            │    └ <function TyperGroup.main at 0x4003455040>
backend-meilisync-1     |            └ <TyperGroup callback>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/typer/core.py", line 778, in main
backend-meilisync-1     |     return _main(
backend-meilisync-1     |            └ <function _main at 0x40033a9310>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/typer/core.py", line 216, in _main
backend-meilisync-1     |     rv = self.invoke(ctx)
backend-meilisync-1     |          │    │      └ <click.core.Context object at 0x4007d5e100>
backend-meilisync-1     |          │    └ <function MultiCommand.invoke at 0x4002c50550>
backend-meilisync-1     |          └ <TyperGroup callback>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
backend-meilisync-1     |     return _process_result(sub_ctx.command.invoke(sub_ctx))
backend-meilisync-1     |            │               │       │       │      └ <click.core.Context object at 0x40052ed250>
backend-meilisync-1     |            │               │       │       └ <function Command.invoke at 0x4002c50040>
backend-meilisync-1     |            │               │       └ <TyperCommand start>
backend-meilisync-1     |            │               └ <click.core.Context object at 0x40052ed250>
backend-meilisync-1     |            └ <function MultiCommand.invoke.<locals>._process_result at 0x40083c2820>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
backend-meilisync-1     |     return ctx.invoke(self.callback, **ctx.params)
backend-meilisync-1     |            │   │      │    │           │   └ {}
backend-meilisync-1     |            │   │      │    │           └ <click.core.Context object at 0x40052ed250>
backend-meilisync-1     |            │   │      │    └ <function start at 0x4008472a60>
backend-meilisync-1     |            │   │      └ <TyperCommand start>
backend-meilisync-1     |            │   └ <function Context.invoke at 0x4002c4adc0>
backend-meilisync-1     |            └ <click.core.Context object at 0x40052ed250>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 783, in invoke
backend-meilisync-1     |     return __callback(*args, **kwargs)
backend-meilisync-1     |                        │       └ {}
backend-meilisync-1     |                        └ ()
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/typer/main.py", line 683, in wrapper
backend-meilisync-1     |     return callback(**use_params)  # type: ignore
backend-meilisync-1     |            │          └ {'context': <click.core.Context object at 0x40052ed250>}
backend-meilisync-1     |            └ <function start at 0x4008472700>
backend-meilisync-1     |
backend-meilisync-1     |   File "/meilisync/meilisync/main.py", line 140, in start
backend-meilisync-1     |     asyncio.run(run())
backend-meilisync-1     |     │       │   └ <function start.<locals>.run at 0x40083c2940>
backend-meilisync-1     |     │       └ <function run at 0x40029d6e50>
backend-meilisync-1     |     └ <module 'asyncio' from '/usr/local/lib/python3.9/asyncio/__init__.py'>
backend-meilisync-1     |
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
backend-meilisync-1     |     return loop.run_until_complete(main)
backend-meilisync-1     |            │    │                  └ <coroutine object start.<locals>.run at 0x400845b6c0>
backend-meilisync-1     |            │    └ <function BaseEventLoop.run_until_complete at 0x40029ec8b0>
backend-meilisync-1     |            └ <_UnixSelectorEventLoop running=True closed=False debug=False>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 634, in run_until_complete
backend-meilisync-1     |     self.run_forever()
backend-meilisync-1     |     │    └ <function BaseEventLoop.run_forever at 0x40029ec820>
backend-meilisync-1     |     └ <_UnixSelectorEventLoop running=True closed=False debug=False>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
backend-meilisync-1     |     self._run_once()
backend-meilisync-1     |     │    └ <function BaseEventLoop._run_once at 0x40029f23a0>
backend-meilisync-1     |     └ <_UnixSelectorEventLoop running=True closed=False debug=False>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
backend-meilisync-1     |     handle._run()
backend-meilisync-1     |     │      └ <function Handle._run at 0x400240adc0>
backend-meilisync-1     |     └ <Handle <TaskWakeupMethWrapper object at 0x4007e71dc0>(<Future finished result=None>)>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
backend-meilisync-1     |     self._context.run(self._callback, *self._args)
backend-meilisync-1     |     │    │            │    │           │    └ <member '_args' of 'Handle' objects>
backend-meilisync-1     |     │    │            │    │           └ <Handle <TaskWakeupMethWrapper object at 0x4007e71dc0>(<Future finished result=None>)>
backend-meilisync-1     |     │    │            │    └ <member '_callback' of 'Handle' objects>
backend-meilisync-1     |     │    │            └ <Handle <TaskWakeupMethWrapper object at 0x4007e71dc0>(<Future finished result=None>)>
backend-meilisync-1     |     │    └ <member '_context' of 'Handle' objects>
backend-meilisync-1     |     └ <Handle <TaskWakeupMethWrapper object at 0x4007e71dc0>(<Future finished result=None>)>
backend-meilisync-1     |
backend-meilisync-1     |   File "/meilisync/meilisync/main.py", line 102, in _
backend-meilisync-1     |     async for event in source:
backend-meilisync-1     |               │        └ <meilisync.source.mysql.MySQL object at 0x4004fd0880>
backend-meilisync-1     |               └ ProgressEvent(progress={'master_log_file': 'binlog.000028', 'master_log_position': 157})
backend-meilisync-1     |
backend-meilisync-1     | > File "/meilisync/meilisync/source/mysql.py", line 100, in __aiter__
backend-meilisync-1     |     async for event in self.stream:
backend-meilisync-1     |                        │    └ <asyncmy.replication.binlogstream.BinLogStream object at 0x4007f52d60>
backend-meilisync-1     |                        └ <meilisync.source.mysql.MySQL object at 0x4004fd0880>
backend-meilisync-1     |
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/asyncmy/replication/binlogstream.py", line 373, in __anext__
backend-meilisync-1     |     ret = await self._read()
backend-meilisync-1     |                 │    └ <function BinLogStream._read at 0x400558eee0>
backend-meilisync-1     |                 └ <asyncmy.replication.binlogstream.BinLogStream object at 0x4007f52d60>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/asyncmy/replication/binlogstream.py", line 288, in _read
backend-meilisync-1     |     raise e
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/asyncmy/replication/binlogstream.py", line 282, in _read
backend-meilisync-1     |     pkt = await self._connection.read_packet()
backend-meilisync-1     |                 │    │           └ <cyfunction Connection.read_packet at 0x40054235f0>
backend-meilisync-1     |                 │    └ <asyncmy.connection.Connection object at 0x4007d932b0>
backend-meilisync-1     |                 └ <asyncmy.replication.binlogstream.BinLogStream object at 0x4007f52d60>
backend-meilisync-1     |   File "asyncmy/connection.pyx", line 644, in read_packet
backend-meilisync-1     |     packet.raise_for_error()
backend-meilisync-1     |   File "asyncmy/protocol.pyx", line 190, in asyncmy.protocol.MysqlPacket.raise_for_error
backend-meilisync-1     |     cpdef raise_for_error(self):
backend-meilisync-1     |   File "asyncmy/protocol.pyx", line 194, in asyncmy.protocol.MysqlPacket.raise_for_error
backend-meilisync-1     |     errors.raise_mysql_exception(self._data)
backend-meilisync-1     |     │      └ <cyfunction raise_mysql_exception at 0x40053102b0>
backend-meilisync-1     |     └ <module 'asyncmy.errors' from '/usr/local/lib/python3.9/site-packages/asyncmy/errors.cpython-39-x86_64-linux-gnu.so'>
backend-meilisync-1     |   File "asyncmy/errors.pyx", line 128, in asyncmy.errors.raise_mysql_exception
backend-meilisync-1     |     cpdef raise_mysql_exception(bytes data):
backend-meilisync-1     |           └ <cyfunction raise_mysql_exception at 0x40053102b0>
backend-meilisync-1     |   File "asyncmy/errors.pyx", line 137, in asyncmy.errors.raise_mysql_exception
backend-meilisync-1     |     raise error_class(errno, err_val)
backend-meilisync-1     |
backend-meilisync-1     | asyncmy.errors.OperationalError: (1236, "A replica with the same server_uuid/server_id as this replica has connected to the source; the first event 'binlog.000028' at 157, the last event read from './binlog.000028' at 126, the last byte read from './binlog.000028' at 157.")

(docker up log)

backend-meilisync-1     | 2023-10-26 01:14:26.378 | DEBUG    | meilisync.main:_:36 - plugins=[] progress=Progress(type=<ProgressType.file: 'file'>) debug=True source=Source(type=<SourceType.mysql: 'mysql'>, database='example_app', host='mysql', port=3306, user='mai', password='password') meilisearch=MeiliSearch(api_url='http://meilisearch:7700', api_key='cb35...', insert_size=100, insert_interval=10) sync=[Sync(plugins=[], table='animals', pk='id', full=False, index='catsearch_settings', fields={'id': None, 'travel_type': None, 'category_group': None, 'title': None, 'created_at': None, 'updated_at': None})] sentry=Sentry(dsn='', environment='production')
backend-meilisync-1     | 2023-10-26 01:14:27.155 | INFO     | meilisync.main:_:101 - Start increment sync data from "mysql" to MeiliSearch...
backend-meilisync-1     | 2023-10-26 01:14:27.413 | DEBUG    | meilisync.main:_:104 - progress={'master_log_file': 'binlog.000002', 'master_log_position': 157}

meilisync/config.yml

debug: true
progress:
  type: file
source:
  type: mysql
  host: mysql
  port: 3306
  user: dbuser
  password: password
  database: example_app
meilisearch:
  api_url: http://meilisearch:7700
  api_key: <MY_ADMIN_API_KEY>
  insert_size: 100
  insert_interval: 10
sync:
  - table: animals
    index: animals
    full: false
    fields:
      id:
      title:
      created_at:
      updated_at:

my.cnf

[mysqld]
character-set-server    = utf8mb4
collation-server        = utf8mb4_unicode_ci
explicit-defaults-for-timestamp = 1
secure-file-priv        = ""
max_allowed_packet      = 32M
max_connections         = 500
max_user_connections    = 10

[client]
default-character-set   = utf8mb4

binlog_format = ROW

mysql> SHOW GLOBAL VARIABLES LIKE 'binlog_format';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| binlog_format | ROW   |
+---------------+-------+
1 row in set (0.04 sec)

show grants;

GRANT APPLICATION_PASSWORD_ADMIN,AUDIT_ABORT_EXEMPT,AUDIT_ADMIN,AUTHENTICATION_POLICY_ADMIN,BACKUP_ADMIN,BINLOG_ADMIN,BINLOG_ENCRYPTION_ADMIN,CLONE_ADMIN,CONNECTION_ADMIN,ENCRYPTION_KEY_ADMIN,FIREWALL_EXEMPT,FLUSH_OPTIMIZER_COSTS,FLUSH_STATUS,FLUSH_TABLES,FLUSH_USER_RESOURCES,GROUP_REPLICATION_ADMIN,GROUP_REPLICATION_STREAM,INNODB_REDO_LOG_ARCHIVE,INNODB_REDO_LOG_ENABLE,PASSWORDLESS_USER_ADMIN,PERSIST_RO_VARIABLES_ADMIN,REPLICATION_APPLIER,REPLICATION_SLAVE_ADMIN,RESOURCE_GROUP_ADMIN,RESOURCE_GROUP_USER,ROLE_ADMIN,SENSITIVE_VARIABLES_OBSERVER,SERVICE_CONNECTION_ADMIN,SESSION_VARIABLES_ADMIN,SET_USER_ID,SHOW_ROUTINE,SYSTEM_USER,SYSTEM_VARIABLES_ADMIN,TABLE_ENCRYPTION_ADMIN,TELEMETRY_LOG_ADMIN,XA_RECOVER_ADMIN ON *.* TO `mai`@`localhost` WITH GRANT OPTION

long2ice commented 1 year ago

See here https://github.com/long2ice/meilisync#source

ogata-ryosuke commented 1 year ago

server_id is default 1.

mysql> select @@server_id;
+-------------+
| @@server_id |
+-------------+
|           1 |
+-------------+
1 row in set (0.00 sec)

Where should I reference for the other keys? asyncmy?

long2ice commented 1 year ago

Allow you set in config

ogata-ryosuke commented 1 year ago

same error...

This should be 'binlog.000007', isn't './binlog.000006'?

backend-meilisync-1     | asyncmy.errors.OperationalError: (1236, "A replica with the same server_uuid/server_id as this replica has connected to the source; the first event 'binlog.000006' at 157, the last event read from './binlog.000006' at 126, the last byte read from './binlog.000006' at 157.")

mysql> show master status;
+---------------+----------+--------------+------------------+-------------------+
| File          | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+---------------+----------+--------------+------------------+-------------------+
| binlog.000007 |      157 |              |                  |                   |
+---------------+----------+--------------+------------------+-------------------+
1 row in set (0.01 sec)

mysql> show slave status;
Empty set, 1 warning (0.01 sec)

root@9e6ed3f4349d:/meilisync# meilisync start
2023-10-26 03:13:48.473 | DEBUG    | meilisync.main:_:36 - plugins=[] progress=Progress(type=<ProgressType.file: 'file'>) debug=True source=Source(type=<SourceType.mysql: 'mysql'>, database='example_app', server_id=1, host='mysql', port=3306, user='root', password='password') meilisearch=MeiliSearch(api_url='http://meilisearch:7700', api_key='cb35a3ab4d7066df8ca617cfa89af8d1dcdc8bc6c13e4b468e0a49d9209a787c', insert_size=100, insert_interval=10) sync=[Sync(plugins=[], table='animals', pk='id', full=False, index='animals', fields={'id': None, 'name': None, 'created_at': None, 'updated_at': None})] sentry=Sentry(dsn='', environment='production')
2023-10-26 03:13:48.859 | INFO     | meilisync.main:_:101 - Start increment sync data from "mysql" to MeiliSearch...
2023-10-26 03:13:48.973 | DEBUG    | meilisync.main:_:104 - progress={'master_log_file': 'binlog.000007', 'master_log_position': 157}

# container startup log
backend-meilisync-1     | 2023-10-26 03:12:17.369 | DEBUG    | meilisync.main:_:36 - plugins=[] progress=Progress(type=<ProgressType.file: 'file'>) debug=True source=Source(type=<SourceType.mysql: 'mysql'>, database='example_app', server_id=1, host='mysql', port=3306, user='mai', password='password') meilisearch=MeiliSearch(api_url='http://meilisearch:7700', api_key='cb35a3ab4d7066df8ca617cfa89af8d1dcdc8bc6c13e4b468e0a49d9209a787c', insert_size=100, insert_interval=10) sync=[Sync(plugins=[], table='animals', pk='id', full=False, index='animals', fields={'id': None, 'name': None, 'created_at': None, 'updated_at': None})] sentry=Sentry(dsn='', environment='production')
backend-meilisync-1     | 2023-10-26 03:12:18.226 | INFO     | meilisync.main:_:101 - Start increment sync data from "mysql" to MeiliSearch...
backend-meilisync-1     | 2023-10-26 03:12:18.483 | DEBUG    | meilisync.main:_:104 - progress={'master_log_file': 'binlog.000007', 'master_log_position': 157}

# meilisync start
backend-meilisync-1     | 2023-10-26 03:06:18.891 | ERROR    | meilisync.source.mysql:__aiter__:121 - Binlog stream error: (1236, "A replica with the same server_uuid/server_id as this replica has connected to the source; the first event 'binlog.000006' at 157, the last event read from './binlog.000006' at 126, the last byte read from './binlog.000006' at 157."), restart...
backend-meilisync-1     | Traceback (most recent call last):
backend-meilisync-1     |
backend-meilisync-1     |   File "/usr/local/bin/meilisync", line 6, in <module>
backend-meilisync-1     |     sys.exit(app())
backend-meilisync-1     |     │   │    └ <typer.main.Typer object at 0x4001f89790>
backend-meilisync-1     |     │   └ <built-in function exit>
backend-meilisync-1     |     └ <module 'sys' (built-in)>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/typer/main.py", line 311, in __call__
backend-meilisync-1     |     return get_command(self)(*args, **kwargs)
backend-meilisync-1     |            │           │      │       └ {}
backend-meilisync-1     |            │           │      └ ()
backend-meilisync-1     |            │           └ <typer.main.Typer object at 0x4001f89790>
backend-meilisync-1     |            └ <function get_command at 0x400347e700>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
backend-meilisync-1     |     return self.main(*args, **kwargs)
backend-meilisync-1     |            │    │     │       └ {}
backend-meilisync-1     |            │    │     └ ()
backend-meilisync-1     |            │    └ <function TyperGroup.main at 0x4003455040>
backend-meilisync-1     |            └ <TyperGroup callback>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/typer/core.py", line 778, in main
backend-meilisync-1     |     return _main(
backend-meilisync-1     |            └ <function _main at 0x40033a9310>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/typer/core.py", line 216, in _main
backend-meilisync-1     |     rv = self.invoke(ctx)
backend-meilisync-1     |          │    │      └ <click.core.Context object at 0x40050a0f10>
backend-meilisync-1     |          │    └ <function MultiCommand.invoke at 0x4002c50550>
backend-meilisync-1     |          └ <TyperGroup callback>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
backend-meilisync-1     |     return _process_result(sub_ctx.command.invoke(sub_ctx))
backend-meilisync-1     |            │               │       │       │      └ <click.core.Context object at 0x4007d28dc0>
backend-meilisync-1     |            │               │       │       └ <function Command.invoke at 0x4002c50040>
backend-meilisync-1     |            │               │       └ <TyperCommand start>
backend-meilisync-1     |            │               └ <click.core.Context object at 0x4007d28dc0>
backend-meilisync-1     |            └ <function MultiCommand.invoke.<locals>._process_result at 0x40083c2820>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
backend-meilisync-1     |     return ctx.invoke(self.callback, **ctx.params)
backend-meilisync-1     |            │   │      │    │           │   └ {}
backend-meilisync-1     |            │   │      │    │           └ <click.core.Context object at 0x4007d28dc0>
backend-meilisync-1     |            │   │      │    └ <function start at 0x4008472a60>
backend-meilisync-1     |            │   │      └ <TyperCommand start>
backend-meilisync-1     |            │   └ <function Context.invoke at 0x4002c4adc0>
backend-meilisync-1     |            └ <click.core.Context object at 0x4007d28dc0>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 783, in invoke
backend-meilisync-1     |     return __callback(*args, **kwargs)
backend-meilisync-1     |                        │       └ {}
backend-meilisync-1     |                        └ ()
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/typer/main.py", line 683, in wrapper
backend-meilisync-1     |     return callback(**use_params)  # type: ignore
backend-meilisync-1     |            │          └ {'context': <click.core.Context object at 0x4007d28dc0>}
backend-meilisync-1     |            └ <function start at 0x4008472700>
backend-meilisync-1     |
backend-meilisync-1     |   File "/meilisync/meilisync/main.py", line 140, in start
backend-meilisync-1     |     asyncio.run(run())
backend-meilisync-1     |     │       │   └ <function start.<locals>.run at 0x40083c2940>
backend-meilisync-1     |     │       └ <function run at 0x40029d6e50>
backend-meilisync-1     |     └ <module 'asyncio' from '/usr/local/lib/python3.9/asyncio/__init__.py'>
backend-meilisync-1     |
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
backend-meilisync-1     |     return loop.run_until_complete(main)
backend-meilisync-1     |            │    │                  └ <coroutine object start.<locals>.run at 0x400845b6c0>
backend-meilisync-1     |            │    └ <function BaseEventLoop.run_until_complete at 0x40029eb8b0>
backend-meilisync-1     |            └ <_UnixSelectorEventLoop running=True closed=False debug=False>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 634, in run_until_complete
backend-meilisync-1     |     self.run_forever()
backend-meilisync-1     |     │    └ <function BaseEventLoop.run_forever at 0x40029eb820>
backend-meilisync-1     |     └ <_UnixSelectorEventLoop running=True closed=False debug=False>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
backend-meilisync-1     |     self._run_once()
backend-meilisync-1     |     │    └ <function BaseEventLoop._run_once at 0x40029f13a0>
backend-meilisync-1     |     └ <_UnixSelectorEventLoop running=True closed=False debug=False>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
backend-meilisync-1     |     handle._run()
backend-meilisync-1     |     │      └ <function Handle._run at 0x4002409dc0>
backend-meilisync-1     |     └ <Handle <TaskWakeupMethWrapper object at 0x40083f8af0>(<Future finished result=None>)>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
backend-meilisync-1     |     self._context.run(self._callback, *self._args)
backend-meilisync-1     |     │    │            │    │           │    └ <member '_args' of 'Handle' objects>
backend-meilisync-1     |     │    │            │    │           └ <Handle <TaskWakeupMethWrapper object at 0x40083f8af0>(<Future finished result=None>)>
backend-meilisync-1     |     │    │            │    └ <member '_callback' of 'Handle' objects>
backend-meilisync-1     |     │    │            └ <Handle <TaskWakeupMethWrapper object at 0x40083f8af0>(<Future finished result=None>)>
backend-meilisync-1     |     │    └ <member '_context' of 'Handle' objects>
backend-meilisync-1     |     └ <Handle <TaskWakeupMethWrapper object at 0x40083f8af0>(<Future finished result=None>)>
backend-meilisync-1     |
backend-meilisync-1     |   File "/meilisync/meilisync/main.py", line 102, in _
backend-meilisync-1     |     async for event in source:
backend-meilisync-1     |               │        └ <meilisync.source.mysql.MySQL object at 0x40050a0df0>
backend-meilisync-1     |               └ ProgressEvent(progress={'master_log_file': 'binlog.000006', 'master_log_position': 157})
backend-meilisync-1     |
backend-meilisync-1     | > File "/meilisync/meilisync/source/mysql.py", line 100, in __aiter__
backend-meilisync-1     |     async for event in self.stream:
backend-meilisync-1     |                        │    └ <asyncmy.replication.binlogstream.BinLogStream object at 0x400558d790>
backend-meilisync-1     |                        └ <meilisync.source.mysql.MySQL object at 0x40050a0df0>
backend-meilisync-1     |
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/asyncmy/replication/binlogstream.py", line 373, in __anext__
backend-meilisync-1     |     ret = await self._read()
backend-meilisync-1     |                 │    └ <function BinLogStream._read at 0x400558fee0>
backend-meilisync-1     |                 └ <asyncmy.replication.binlogstream.BinLogStream object at 0x400558d790>
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/asyncmy/replication/binlogstream.py", line 288, in _read
backend-meilisync-1     |     raise e
backend-meilisync-1     |   File "/usr/local/lib/python3.9/site-packages/asyncmy/replication/binlogstream.py", line 282, in _read
backend-meilisync-1     |     pkt = await self._connection.read_packet()
backend-meilisync-1     |                 │    │           └ <cyfunction Connection.read_packet at 0x40054235f0>
backend-meilisync-1     |                 │    └ <asyncmy.connection.Connection object at 0x4007f3b8e0>
backend-meilisync-1     |                 └ <asyncmy.replication.binlogstream.BinLogStream object at 0x400558d790>
backend-meilisync-1     |   File "asyncmy/connection.pyx", line 644, in read_packet
backend-meilisync-1     |     packet.raise_for_error()
backend-meilisync-1     |   File "asyncmy/protocol.pyx", line 190, in asyncmy.protocol.MysqlPacket.raise_for_error
backend-meilisync-1     |     cpdef raise_for_error(self):
backend-meilisync-1     |   File "asyncmy/protocol.pyx", line 194, in asyncmy.protocol.MysqlPacket.raise_for_error
backend-meilisync-1     |     errors.raise_mysql_exception(self._data)
backend-meilisync-1     |     │      └ <cyfunction raise_mysql_exception at 0x40053102b0>
backend-meilisync-1     |     └ <module 'asyncmy.errors' from '/usr/local/lib/python3.9/site-packages/asyncmy/errors.cpython-39-x86_64-linux-gnu.so'>
backend-meilisync-1     |   File "asyncmy/errors.pyx", line 128, in asyncmy.errors.raise_mysql_exception
backend-meilisync-1     |     cpdef raise_mysql_exception(bytes data):
backend-meilisync-1     |           └ <cyfunction raise_mysql_exception at 0x40053102b0>
backend-meilisync-1     |   File "asyncmy/errors.pyx", line 137, in asyncmy.errors.raise_mysql_exception
backend-meilisync-1     |     raise error_class(errno, err_val)
backend-meilisync-1     |
backend-meilisync-1     | asyncmy.errors.OperationalError: (1236, "A replica with the same server_uuid/server_id as this replica has connected to the source; the first event 'binlog.000006' at 157, the last event read from './binlog.000006' at 126, the last byte read from './binlog.000006' at 157.")
backend-meilisync-1     | 2023-10-26 03:06:18.937 | ERROR    | meilisync.source.mysql:__aiter__:121 - Binlog stream error: (2013, 'Lost connection to MySQL server during query'), restart...
backend-meilisync-1     | Traceback (most recent call last):
backend-meilisync-1     |
backend-meilisync-1     |   File "asyncmy/connection.pyx", line 656, in asyncmy.connection.Connection._read_bytes
backend-meilisync-1     |     data = await self._reader.readexactly(num_bytes)
backend-meilisync-1     |   File "/usr/local/lib/python3.9/asyncio/streams.py", line 721, in readexactly
backend-meilisync-1     |     raise exceptions.IncompleteReadError(incomplete, n)
backend-meilisync-1     |           │          │                   │           └ 4
backend-meilisync-1     |           │          │                   └ b''
backend-meilisync-1     |           │          └ <class 'asyncio.exceptions.IncompleteReadError'>
backend-meilisync-1     |           └ <module 'asyncio.exceptions' from '/usr/local/lib/python3.9/asyncio/exceptions.py'>
backend-meilisync-1     |
backend-meilisync-1     | asyncio.exceptions.IncompleteReadError: 0 bytes read on a total of 4 expected bytes
backend-meilisync-1     |
backend-meilisync-1     |
backend-meilisync-1     | The above exception was the direct cause of the following exception:
backend-meilisync-1     |
backend-meilisync-1     |

okxiaochen commented 2 months ago

I came across the same issue today. In my scenario, where multiple programs (such as multiple instances of meilisync) are reading the same MySQL binlog file, this can indeed cause the error A replica with the same server_uuid/server_id as this replica has connected to the source. This error occurs because MySQL’s replication mechanism requires each slave (including clients reading the binlog) connecting to the master to have a unique server_uuid and server_id. If multiple clients use the same identifier (i.e., the same server_uuid/server_id) to connect to the master, MySQL sees them as duplicate connections from the same slave, leading to this error.

Solutions

Assign a Unique server_id to Each meilisync Instance Ensure that each meilisync instance has a unique server_id. Typically, meilisync will use some form of server_id to identify itself when connecting to MySQL. You need to check and modify the configuration of each meilisync instance to use different server_id values.

For example, in the MySQL connection configuration, you could set it up as follows:

# meilisync configuration file example
mysql:
  host: "your_mysql_host"
  port: 3306
  user: "your_username"
  password: "your_password"
  server_id: 1001  # Use a unique server_id for each instance

Ensure that each meilisync instance uses a different server_id, such as 1001, 1002, 1003, etc.

long2ice / meilisync

asyncmy.errors.OperationalError: (1236, "A replica with the same server_uuid/server_id as this replica has connected to the source #27

Solutions