tortoise / aerich

A database migrations tool for TortoiseORM, ready to production.
https://github.com/tortoise/aerich
Apache License 2.0
804 stars 90 forks source link

Double semicolon on CREATE TABLE statement causing table creation upgrades to fail #280

Closed CortexPE closed 1 year ago

CortexPE commented 1 year ago

This took me a while to find where the issue occurs because the queries worked everywhere else EXCEPT aerich upgrade, I'm also new to aerich and tortoise-orm.

Despite being wrapped in an in_transaction context manager, it didn't rollback... it still created the channels table (cuz it's a DDL statement)... Perhaps a way to backup and restore everything in the target database?? Otherwise I just can't afford to let errors like this happen on a production environment without a definite way to scrap everything and restore to what is essentially a snapshot (without mysqldump).

The deepest my knowledge goes about this issue is that the code generated at this point already has a semicolon (printed below):

CREATE TABLE IF NOT EXISTS `channels` (
    `id` VARCHAR(32) NOT NULL  PRIMARY KEY,
    `last_annoyed` DATETIME(6)
) CHARACTER SET utf8mb4;

Possibly from the template here

And this code is joining all of them with yet another semicolon, while everything else like the ALTER TABLE generator doesn't include one:

ALTER TABLE `users` ADD `first_command_run` DATETIME(6)
ALTER TABLE `users` ADD `do_not_annoy_until` DATETIME(6)

Hope this helps solve the issue quick!

Version Information:

Python: 3.10.7 MySQL: 8.0.30 aerich: 0.7.1

Produced file:

from tortoise import BaseDBAsyncClient

async def upgrade(db: BaseDBAsyncClient) -> str:
    return """
        CREATE TABLE IF NOT EXISTS `channels` (
    `id` VARCHAR(32) NOT NULL  PRIMARY KEY,
    `last_annoyed` DATETIME(6)
) CHARACTER SET utf8mb4;;
        ALTER TABLE `users` ADD `do_not_annoy_until` DATETIME(6);
        ALTER TABLE `users` ADD `first_command_run` DATETIME(6);"""

async def downgrade(db: BaseDBAsyncClient) -> str:
    return """
        ALTER TABLE `users` DROP COLUMN `do_not_annoy_until`;
        ALTER TABLE `users` DROP COLUMN `first_command_run`;
        DROP TABLE IF EXISTS `channels`;"""

Error:

Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\tortoise\backends\mysql\client.py", line 44, in translate_exceptions_
    return await func(self, *args)
  File "C:\Python310\lib\site-packages\tortoise\backends\mysql\client.py", line 213, in execute_script
    async with connection.cursor() as cursor:
  File "C:\Python310\lib\site-packages\aiomysql\utils.py", line 78, in __aexit__
    await self._obj.close()
  File "C:\Python310\lib\site-packages\aiomysql\cursors.py", line 158, in close
    while (await self.nextset()):
  File "C:\Python310\lib\site-packages\aiomysql\cursors.py", line 191, in nextset
    await conn.next_result()
  File "C:\Python310\lib\site-packages\aiomysql\connection.py", line 473, in next_result
    await self._read_query_result()
  File "C:\Python310\lib\site-packages\aiomysql\connection.py", line 672, in _read_query_result
    await result.read()
  File "C:\Python310\lib\site-packages\aiomysql\connection.py", line 1153, in read
    first_packet = await self.connection._read_packet()
  File "C:\Python310\lib\site-packages\aiomysql\connection.py", line 641, in _read_packet
    packet.raise_for_error()
  File "C:\Python310\lib\site-packages\pymysql\protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "C:\Python310\lib\site-packages\pymysql\err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ';\n        ALTER TABLE `users` ADD `do_not_annoy_until` DATETIME(6);\n        ALTE' at line 1")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Python310\Scripts\aerich.exe\__main__.py", line 7, in <module>
  File "C:\Python310\lib\site-packages\aerich\cli.py", line 258, in main
    cli()
  File "C:\Python310\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\Python310\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Python310\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Python310\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Python310\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Python310\lib\site-packages\click\decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "C:\Python310\lib\site-packages\aerich\cli.py", line 31, in wrapper
    loop.run_until_complete(f(*args, **kwargs))
  File "C:\Python310\lib\asyncio\base_events.py", line 646, in run_until_complete
    return future.result()
  File "C:\Python310\lib\site-packages\aerich\cli.py", line 97, in upgrade
    migrated = await command.upgrade()
  File "C:\Python310\lib\site-packages\aerich\__init__.py", line 53, in upgrade
    await conn.execute_script(await upgrade(conn))
  File "C:\Python310\lib\site-packages\tortoise\backends\mysql\client.py", line 52, in translate_exceptions_
    raise OperationalError(exc)
tortoise.exceptions.OperationalError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ';\n        ALTER TABLE `users` ADD `do_not_annoy_until` DATETIME(6);\n        ALTE' at line 1")
CortexPE commented 1 year ago

Figured that the least invasive change would be to add rstrip(";") to the end of create_table, since... Upon skimming through the codebase, it doesn't seem like anything else has this issue (because they're templated without semicolons) but I'm also not 100% sure

Will open a PR with the change, please review it.