qizhiyi / sqlalchemy-migrate

Automatically exported from code.google.com/p/sqlalchemy-migrate
MIT License
0 stars 0 forks source link

Multiple DROP TABLE statements in a migration causes mysql error 2014 'commands out of sync' #99

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
What steps will reproduce the problem?
1. This migration causes an error when applied:
DROP TABLE IF EXISTS `market_pool_trunks_map`, 
DROP TABLE IF EXISTS `market_npa_nxx`;

2. Rewriting the migration fixes the error:
DROP TABLE IF EXISTS `market_pool_trunks_map`, `market_npa_nxx`;

Why?  What is wrong with multiple DROP TABLE commands in a migration?

What is the expected output? What do you see instead?

The error:
  File "/apps/python/lib/db-migrate.py", line 174, in <module>
    DbMigrate().main(sys.argv[1:])
  File "/apps/python/lib/db-migrate.py", line 28, in main
    self.migrateMain()
  File "/apps/python/lib/db-migrate.py", line 115, in migrateMain
    do_migrate(self.args,**kwargs)
  File ".../python/lib/python2.6/site-packages/sqlalchemy_migrate-0.6-py2.6.egg/migrate/versioning/shell.py", line 203, in main
    ret = command_func(**kwargs)
  File ".../python/lib/python2.6/site-packages/sqlalchemy_migrate-0.6-py2.6.egg/migrate/versioning/api.py", line 185, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
  File "<string>", line 2, in _migrate
  File ".../python/lib/python2.6/site-packages/sqlalchemy_migrate-0.6-py2.6.egg/migrate/versioning/util/__init__.py", line 160, in with_engine
    return f(*a, **kw)
  File ".../python/lib/python2.6/site-packages/sqlalchemy_migrate-0.6-py2.6.egg/migrate/versioning/api.py", line 364, in _migrate
    schema.runchange(ver, change, changeset.step)
  File ".../python/lib/python2.6/site-packages/sqlalchemy_migrate-0.6-py2.6.egg/migrate/versioning/schema.py", line 83, in runchange
    change.run(self.engine, step)
  File ".../python/lib/python2.6/site-packages/sqlalchemy_migrate-0.6-py2.6.egg/migrate/versioning/script/sql.py", line 43, in run
    conn.execute(text)
  File ".../python/lib/python2.6/site-packages/SQLAlchemy-0.6.2-py2.6.egg/sqlalchemy/engine/base.py", line 1157, in execute
    params)
  File ".../python/lib/python2.6/site-packages/SQLAlchemy-0.6.2-py2.6.egg/sqlalchemy/engine/base.py", line 1253, in _execute_text
    return self.__execute_context(context)
  File ".../python/lib/python2.6/site-packages/SQLAlchemy-0.6.2-py2.6.egg/sqlalchemy/engine/base.py", line 1268, in __execute_context
    context.parameters[0], context=context)
  File ".../python/lib/python2.6/site-packages/SQLAlchemy-0.6.2-py2.6.egg/sqlalchemy/engine/base.py", line 1367, in _cursor_execute
    context)
  File ".../python/lib/python2.6/site-packages/SQLAlchemy-0.6.2-py2.6.egg/sqlalchemy/engine/base.py", line 1360, in _cursor_execute
    context)
  File ".../python/lib/python2.6/site-packages/SQLAlchemy-0.6.2-py2.6.egg/sqlalchemy/engine/default.py", line 288, in do_execute
    cursor.execute(statement, parameters)
  File "build/bdist.linux-x86_64/egg/MySQLdb/cursors.py", line 176, in execute
  File "build/bdist.linux-x86_64/egg/MySQLdb/cursors.py", line 85, in _warning_check
  File "build/bdist.linux-x86_64/egg/MySQLdb/connections.py", line 317, in show_warnings
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (2014, "Commands out of 
sync; you can't run this command now") 'DROP TABLE IF EXISTS 
`market_pool_trunks_map`;\nDROP TABLE IF EXISTS `market_npa_nxx`;\n\n' ()

What version of the product are you using? On what operating system?

sqlalchemy-0.6.2, migrate-0.6, python 2.6

Original issue reported on code.google.com by jeremy.g...@gmail.com on 4 Nov 2010 at 3:50

GoogleCodeExporter commented 8 years ago
Hello? Anybody home? I'm hitting this same error as well. Is anybody responding 
to issues in sqlalchemy-migrate any more?

Original comment by jaypi...@gmail.com on 20 Mar 2011 at 6:30

GoogleCodeExporter commented 8 years ago
On investigation into the sa-migrate code, I uncovered this, from 
/migrate/versioning/script/sql.py:

    def run(self, engine, step=None, executemany=True):
        """Runs SQL script through raw dbapi execute call"""
        text = self.source()
        # Don't rely on SA's autocommit here
        # (SA uses .startswith to check if a commit is needed. What if script
        # starts with a comment?)
        conn = engine.connect()
        try:
            trans = conn.begin()
            try:
                # HACK: SQLite doesn't allow multiple statements through
                # its execute() method, but it provides executescript() instead
                dbapi = conn.engine.raw_connection()
                if executemany and getattr(dbapi, 'executescript', None):
                    dbapi.executescript(text)
                else:
                    conn.execute(text)
                trans.commit()
            except:
                trans.rollback()
                raise
        finally:
            conn.close()

The above is the method that runs when a SQL file is used for the 
upgrade/downgrade. The reason that MySQLdb is barfing the Warning about 
Commands out of sync is because the code above is supplying the connection with 
a single string, instead of conn.execute()'ing each of the SQL statements in 
the file.

I believe the warning can be safely ignored... it doesn't seem to be indicative 
that an error actually occurred.

Alternately, the code above should probably be modified to work better with 
MySQLdb, which specifically has issues executing multiple SQL queries in a 
single string.

One option might be to use the python-sqlparse library and do something like 
this:

import sqlparse
...

statements = sqlparse.split(text)
for statement in statements:
    conn.execute(statement)

Original comment by jaypi...@gmail.com on 20 Mar 2011 at 7:44

GoogleCodeExporter commented 8 years ago
Hmm, seems that there actually *is* a problem that cannot be safely ignored 
with the warning about Commands out of sync.

Still investigating...

Original comment by jaypi...@gmail.com on 20 Mar 2011 at 7:51

GoogleCodeExporter commented 8 years ago
Any progress in fixing this issue?

Original comment by eliqui...@gmail.com on 6 Mar 2012 at 8:28

GoogleCodeExporter commented 8 years ago
For real, I'm getting this error too, with multiple update statements.

Original comment by cc.emera...@gmail.com on 7 Mar 2013 at 1:13

GoogleCodeExporter commented 8 years ago
This appears to have caused "silent failures" in our recent migrations. The 
migration logged the failure but did not abort and prevent further migrations 
from running. This left our database in an undefined state and required manual 
intervention to correct. We now think that migrations must be run only one at a 
time and that close examination of the logs will tell us whether we have to 
manually run the contents of the migration if SA has failed but reported 
success.

To trigger this failure, we had several update statements. We worked around 
this by using only a single statement for the migration.

81 -> 80...
Error closing cursor: (2014, "Commands out of sync; you can't run this command 
now")
done
80 -> 79...
done
79 -> 78...
done
78 -> 77...
done

Original comment by jj...@yapta.com on 9 Mar 2013 at 7:57