ilogus
commented
2 months ago Hello, @George-alaabsi, I had the same problem using the mariadb image, but when I switched to the mysql 8 image, I had no more problems, hope this help :)
Open George-alaabsi opened 3 months ago
ilogus
commented
2 months ago Hello, @George-alaabsi, I had the same problem using the mariadb image, but when I switched to the mysql 8 image, I had no more problems, hope this help :)
morgang5522
commented
1 month ago I've got a similar issue, however we are unable to upgrade to the MySQL v8 image at this time. The data seems to be indexed in meilisearch, but meilisync just falls over afterwards and then gets stuck in a restart loop.
Running in docker:
services:
meilisearch:
container_name: meilisearch
platform: linux/amd64
image: getmeili/meilisearch:v1.8
volumes:
- ./meili-data:/meili_data
ports:
- 7700:7700
environment:
MEILI_MASTER_KEY: <key>
meilisync:
container_name: meilisync
platform: linux/amd64
image: long2ice/meilisync:dev
volumes:
- ./config.yml:/meilisync/config.ymlTail end of the error log
meilisync | │ ╭──────────────────────────── locals ────────────────────────────╮ │
meilisync | │ │ self = <meilisync.source.mysql.MySQL object at 0x7fe6ed5e0d70> │ │
meilisync | │ ╰────────────────────────────────────────────────────────────────╯ │
meilisync | │ │
meilisync | │ /meilisync/meilisync/source/mysql.py:77 in get_current_progress │
meilisync | │ │
meilisync | │ 74 │ │ │ │ │ await cur.execute("SHOW MASTER STATUS") │
meilisync | │ 75 │ │ │ │ ret = await cur.fetchone() │
meilisync | │ 76 │ │ │ │ return { │
meilisync | │ ❱ 77 │ │ │ │ │ "master_log_file": ret["File"], │
meilisync | │ 78 │ │ │ │ │ "master_log_position": ret["Position"], │
meilisync | │ 79 │ │ │ │ } │
meilisync | │ 80 │
meilisync | │ │
meilisync | │ ╭────────────────────────────── locals ──────────────────────────────╮ │
meilisync | │ │ conn = <asyncmy.connection.Connection object at 0x7fe6e771dd30> │ │
meilisync | │ │ cur = <asyncmy.cursors.DictCursor object at 0x7fe6ecd82210> │ │
meilisync | │ │ ret = None │ │
meilisync | │ │ self = <meilisync.source.mysql.MySQL object at 0x7fe6ed5e0d70> │ │
meilisync | │ │ version = '5.7.44' │ │
meilisync | │ ╰────────────────────────────────────────────────────────────────────╯ │
meilisync | ╰──────────────────────────────────────────────────────────────────────────────╯
meilisync | TypeError: 'NoneType' object is not subscriptable
meilisync exited with code 1Okay so I actually worked this out in the end. This happens when your MySQL server does not have replication enabled. meilisync is looking for a record to be found when you run SHOW MASTER STATUS; and if it doesn't exist it will fail.
The solution is to enable replication, so that when you run the command you get an output, then meilisync works with 5.7.
dinihou
commented
1 month ago Okay so I actually worked this out in the end. This happens when your MySQL server does not have replication enabled.
meilisyncis looking for a record to be found when you runSHOW MASTER STATUS;and if it doesn't exist it will fail.The solution is to enable replication, so that when you run the command you get an output, then
meilisyncworks with 5.7.
Hi, same here , how to enable replication ?
morgang5522
commented
1 month ago Okay so I actually worked this out in the end. This happens when your MySQL server does not have replication enabled.
meilisyncis looking for a record to be found when you runSHOW MASTER STATUS;and if it doesn't exist it will fail. The solution is to enable replication, so that when you run the command you get an output, thenmeilisyncworks with 5.7.Hi, same here , how to enable replication ?
This article helped me: https://support.simplebackups.com/en/articles/6226925-how-to-enable-binary-log-for-mysql-or-mariadb
2024-03-27 06:21:05.534 | INFO | meilisync.main::101 - Start increment sync data from "SourceType.mysql" to MeiliSearch... ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /meilisync/meilisync/main.py:140 in start │ │ │ │ 137 │ │ lock = asyncio.Lock() │ │ 138 │ │ await asyncio.gather((), interval()) │ │ 139 │ │ │ ❱ 140 │ asyncio.run(run()) │ │ 141 │ │ 142 │ │ 143 @app.command(help="Refresh all data by swap index") │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ = <function start.. at 0x7fe7ee59c2c0> │ │
│ │ collection = <meilisync.event.EventCollection object at │ │
│ │ 0x7fe7ef2150d0> │ │
│ │ context = <click.core.Context object at 0x7fe7ef566000> │ │
│ │ current_progress = None │ │
│ │ interval = <function start..interval at 0x7fe7ee59c180> │ │
│ │ lock = <asyncio.locks.Lock object at 0x7fe7ee97b170 │ │
│ │ [unlocked]> │ │
│ │ meili = <meilisync.meili.Meili object at 0x7fe7ef802b10> │ │
│ │ meili_settings = MeiliSearch( │ │
│ │ │ api_url='http://host.docker.internal:7700', │ │
│ │ │ │ │
│ │ api_key='2e4b99ff802ebfcb96b9134ad74cbcdd5297af4547a… │ │
│ │ │ insert_size=1000, │ │
│ │ │ insert_interval=10 │ │
│ │ ) │ │
│ │ progress = <meilisync.progress.file.File object at │ │
│ │ 0x7fe7ef1c9d60> │ │
│ │ run = <function start..run at 0x7fe7ee59d620> │ │
│ │ settings = Settings( │ │
│ │ │ plugins=[], │ │
│ │ │ progress=Progress( │ │
│ │ │ │ type=<ProgressType.file: 'file'> │ │
│ │ │ ), │ │
│ │ │ debug=False, │ │
│ │ │ source=Source( │ │
│ │ │ │ type=<SourceType.mysql: 'mysql'>, │ │
│ │ │ │ database='aars', │ │
│ │ │ │ host='host.docker.internal', │ │
│ │ │ │ port=3306, │ │
│ │ │ │ user='usr', │ │
│ │ │ │ password='12345' │ │
│ │ │ ), │ │
│ │ │ meilisearch=MeiliSearch( │ │
│ │ │ │ api_url='http://host.docker.internal:7700', │ │
│ │ │ │ │ │
│ │ api_key='2e4b99ff802ebfcb96b9134ad74cbcdd5297af4547a… │ │
│ │ │ │ insert_size=1000, │ │
│ │ │ │ insert_interval=10 │ │
│ │ │ ), │ │
│ │ │ sync=[ │ │
│ │ │ │ Sync( │ │
│ │ │ │ │ plugins=[], │ │
│ │ │ │ │ table='bb', │ │
│ │ │ │ │ pk='id', │ │
│ │ │ │ │ full=False, │ │
│ │ │ │ │ index='products', │ │
│ │ │ │ │ fields=None │ │
│ │ │ │ ) │ │
│ │ │ ], │ │
│ │ │ sentry=None │ │
│ │ ) │ │
│ │ source = <meilisync.source.mysql.MySQL object at │ │
│ │ 0x7fe7ef777e30> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /usr/local/lib/python3.12/asyncio/runners.py:194 in run │
│ │
│ 191 │ │ │ "asyncio.run() cannot be called from a running event loop" │
│ 192 │ │
│ 193 │ with Runner(debug=debug, loop_factory=loop_factory) as runner: │
│ ❱ 194 │ │ return runner.run(main) │
│ 195 │
│ 196 │
│ 197 def _cancel_all_tasks(loop): │
│ │
│ ╭──────────────────────────────── locals ────────────────────────────────╮ │
│ │ debug = None │ │
│ │ loop_factory = None │ │
│ │ main = <coroutine object start..run at 0x7fe7ee5acba0> │ │
│ │ runner = <asyncio.runners.Runner object at 0x7fe7ef2146b0> │ │
│ ╰────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /usr/local/lib/python3.12/asyncio/runners.py:118 in run │
│ │
│ 115 │ │ │
│ 116 │ │ self._interrupt_count = 0 │
│ 117 │ │ try: │
│ ❱ 118 │ │ │ return self._loop.run_until_complete(task) │
│ 119 │ │ except exceptions.CancelledError: │
│ 120 │ │ │ if self._interrupt_count > 0: │
│ 121 │ │ │ │ uncancel = getattr(task, "uncancel", None) │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ context = <_contextvars.Context object at 0x7fe7f17c76c0> │ │
│ │ coro = <coroutine object start..run at 0x7fe7ee5acba0> │ │
│ │ self = <asyncio.runners.Runner object at 0x7fe7ef2146b0> │ │
│ │ sigint_handler = functools.partial(<bound method Runner._on_sigint of │ │
│ │ <asyncio.runners.Runner object at 0x7fe7ef2146b0>>, │ │
│ │ main_task=<Task finished name='Task-4' │ │
│ │ coro=<start..run() done, defined at │ │
│ │ /meilisync/meilisync/main.py:135> │ │
│ │ exception=TypeError("'NoneType' object is not │ │
│ │ subscriptable")>) │ │
│ │ task = <Task finished name='Task-4' coro=<start..run() │ │
│ │ done, defined at /meilisync/meilisync/main.py:135> │ │
│ │ exception=TypeError("'NoneType' object is not │ │
│ │ subscriptable")> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /usr/local/lib/python3.12/asyncio/base_events.py:664 in run_until_complete │
│ │
│ 661 │ │ if not future.done(): │
│ 662 │ │ │ raise RuntimeError('Event loop stopped before Future comp │
│ 663 │ │ │
│ ❱ 664 │ │ return future.result() │
│ 665 │ │
│ 666 │ def stop(self): │
│ 667 │ │ """Stop running the event loop. │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ future = <Task finished name='Task-4' coro=<start..run() done, │ │
│ │ defined at /meilisync/meilisync/main.py:135> │ │
│ │ ### exception=TypeError("'NoneType' object is not │ │
│ │ subscriptable")> │ │
│ │ new_task = False │ │
│ │ self = <UnixSelectorEventLoop running=False closed=True │ │
│ │ debug=False> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /meilisync/meilisync/main.py:138 in run │
│ │
│ 135 │ async def run(): │
│ 136 │ │ nonlocal lock │
│ 137 │ │ lock = asyncio.Lock() │
│ ❱ 138 │ │ await asyncio.gather((), interval()) │
│ 139 │ │
│ 140 │ asyncio.run(run()) │
│ 141 │
│ │
│ ╭────────────────────────────── locals ───────────────────────────────╮ │
│ │ = <function start.. at 0x7fe7ee59c2c0> │ │
│ │ interval = <function start..interval at 0x7fe7ee59c180> │ │
│ │ lock = <asyncio.locks.Lock object at 0x7fe7ee97b170 [unlocked]> │ │
│ ╰─────────────────────────────────────────────────────────────────────╯ │
│ │
│ /meilisync/meilisync/main.py:102 in _ │
│ │
│ 99 │ │ │ │ │ │ f'No data found for table "{settings.source.da │
│ 100 │ │ │ │ │ ) │
│ 101 │ │ logger.info(f'Start increment sync data from "{settings.source │
│ ❱ 102 │ │ async for event in source: │
│ 103 │ │ │ if settings.debug: │
│ 104 │ │ │ │ logger.debug(event) │
│ 105 │ │ │ current_progress = event.progress │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ collection = <meilisync.event.EventCollection object at │ │
│ │ 0x7fe7ef2150d0> │ │
│ │ current_progress = None │ │
│ │ lock = <asyncio.locks.Lock object at 0x7fe7ee97b170 │ │
│ │ [unlocked]> │ │
│ │ meili = <meilisync.meili.Meili object at 0x7fe7ef802b10> │ │
│ │ meili_settings = MeiliSearch( │ │
│ │ │ api_url='http://host.docker.internal:7700', │ │
│ │ │ │ │
│ │ api_key='2e4b99ff802ebfcb96b9134ad74cbcdd5297af4547a… │ │
│ │ │ insert_size=1000, │ │
│ │ │ insert_interval=10 │ │
│ │ ) │ │
│ │ progress = <meilisync.progress.file.File object at │ │
│ │ 0x7fe7ef1c9d60> │ │
│ │ settings = Settings( │ │
│ │ │ plugins=[], │ │
│ │ │ progress=Progress( │ │
│ │ │ │ type=<ProgressType.file: 'file'> │ │
│ │ │ ), │ │
│ │ │ debug=False, │ │
│ │ │ source=Source( │ │
│ │ │ │ type=<SourceType.mysql: 'mysql'>, │ │
│ │ │ │ database='aars', │ │
│ │ │ │ host='host.docker.internal', │ │
│ │ │ │ port=3306, │ │
│ │ │ │ user='usr', │ │
│ │ │ │ password='12345' │ │
│ │ │ ), │ │
│ │ │ meilisearch=MeiliSearch( │ │
│ │ │ │ api_url='http://host.docker.internal:7700', │ │
│ │ │ │ │ │
│ │ api_key='2e4b99ff802ebfcb96b9134ad74cbcdd5297af4547a… │ │
│ │ │ │ insert_size=1000, │ │
│ │ │ │ insert_interval=10 │ │
│ │ │ ), │ │
│ │ │ sync=[ │ │
│ │ │ │ Sync( │ │
│ │ │ │ │ plugins=[], │ │
│ │ │ │ │ table='bb', │ │
│ │ │ │ │ pk='id', │ │
│ │ │ │ │ full=False, │ │
│ │ │ │ │ index='products', │ │
│ │ │ │ │ fields=None │ │
│ │ │ │ ) │ │
│ │ │ ], │ │
│ │ │ sentry=None │ │
│ │ ) │ │
│ │ source = <meilisync.source.mysql.MySQL object at │ │
│ │ 0x7fe7ef777e30> │ │
│ │ sync = Sync( │ │
│ │ │ plugins=[], │ │
│ │ │ table='bb', │ │
│ │ │ pk='id', │ │
│ │ │ full=False, │ │
│ │ │ index='products', │ │
│ │ │ fields=None │ │
│ │ ) │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /meilisync/meilisync/source/mysql.py:93 in aiter │
│ │
│ 90 │ │ self.conn = await asyncmy.connect(self.kwargs) │
│ 91 │ │ self.ctl_conn = await asyncmy.connect(self.kwargs) │
│ 92 │ │ if not self.progress: │
│ ❱ 93 │ │ │ self.progress = await self.get_current_progress() │
│ 94 │ │ yield ProgressEvent( │
│ 95 │ │ │ progress=self.progress, │
│ 96 │ │ ) │
│ │
│ ╭──────────────────────────── locals ────────────────────────────╮ │
│ │ self = <meilisync.source.mysql.MySQL object at 0x7fe7ef777e30> │ │
│ ╰────────────────────────────────────────────────────────────────╯ │
│ │
│ /meilisync/meilisync/source/mysql.py:70 in get_current_progress │
│ │
│ 67 │ │ │ │ await cur.execute("SHOW MASTER STATUS") │
│ 68 │ │ │ │ ret = await cur.fetchone() │
│ 69 │ │ │ │ return { │
│ ❱ 70 │ │ │ │ │ "master_log_file": ret["File"], │
│ 71 │ │ │ │ │ "master_log_position": ret["Position"], │
│ 72 │ │ │ │ } │
│ 73 │
│ │
│ ╭──────────────────────────── locals ─────────────────────────────╮ │
│ │ conn = <asyncmy.connection.Connection object at 0x7fe7ee965e50> │ │
│ │ cur = <asyncmy.cursors.DictCursor object at 0x7fe7ee13e8d0> │ │
│ │ ret = None │ │
│ │ self = <meilisync.source.mysql.MySQL object at 0x7fe7ef777e30> │ │
│ ╰─────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
TypeError: 'NoneType' object is not subscriptable
Exception ignored in: <function StreamWriter.del at 0x7fe7f19faf20>
Traceback (most recent call last):
File "/usr/local/lib/python3.12/asyncio/streams.py", line 397, in del
File "/usr/local/lib/python3.12/asyncio/streams.py", line 343, in close
File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 1206, in close
File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 871, in close
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 772, in call_soon
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 519, in _check_closed
RuntimeError: Event loop is closed
Exception ignored in: <function StreamWriter.del at 0x7fe7f19faf20>
Traceback (most recent call last):
File "/usr/local/lib/python3.12/asyncio/streams.py", line 397, in del
File "/usr/local/lib/python3.12/asyncio/streams.py", line 343, in close
File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 1206, in close
File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 871, in close
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 772, in call_soon
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 519, in _check_closed
RuntimeError: Event loop is closed