bookieio / Bookie

Python based delicious.com replacement
GNU Affero General Public License v3.0
633 stars 139 forks source link

Installing a new MySQL database fails #452

Open jcharaoui opened 10 years ago

jcharaoui commented 10 years ago

On Debian, using MySQL 5.5.35+dfsg-0+wheezy1 and MySQLdb 1.2.5, installing into an empty database fails with the following error log:

bin/python bin/alembic upgrade head
INFO  [alembic.migration] Context impl MySQLImpl.
INFO  [alembic.migration] Will assume non-transactional DDL.
INFO  [alembic.migration] Running upgrade None -> 5920b225d05d
INFO  [alembic.migration] Running upgrade 5920b225d05d -> 9f274a38d84
INFO  [alembic.migration] Running upgrade 9f274a38d84 -> 44dccb7b8b82
Traceback (most recent call last):
  File "bin/alembic", line 9, in <module>
    load_entry_point('alembic==0.3.4', 'console_scripts', 'alembic')()
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/config.py", line 229, in main
    **dict((k, getattr(options, k)) for k in kwarg)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/command.py", line 120, in upgrade
    script.run_env()
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/script.py", line 188, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/util.py", line 185, in load_python_file
    module = imp.load_source(module_id, path, open(path, 'rb'))
  File "dbversions/env.py", line 93, in <module>
    run_migrations_online()
  File "dbversions/env.py", line 86, in run_migrations_online
    context.run_migrations()
  File "<string>", line 7, in run_migrations
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/environment.py", line 450, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/migration.py", line 211, in run_migrations
    change(**kw)
  File "dbversions/versions/44dccb7b8b82_update_username_to_l.py", line 25, in upgrade
    op.drop_constraint("bmarks_username_fkey", "bmarks")
  File "<string>", line 7, in drop_constraint
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/operations.py", line 557, in drop_constraint
    self.impl.drop_constraint(const)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 134, in drop_constraint
    self._exec(schema.DropConstraint(const))
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 75, in _exec
    conn.execute(construct, *multiparams, **params)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 662, in execute
    params)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 714, in _execute_ddl
    compiled = ddl.compile(dialect=dialect)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/sqlalchemy/sql/expression.py", line 1912, in compile
    return self._compiler(dialect, bind=bind, **kw)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/sqlalchemy/schema.py", line 2899, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/sqlalchemy/engine/interfaces.py", line 787, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/sqlalchemy/engine/interfaces.py", line 806, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/sqlalchemy/ext/compiler.py", line 404, in <lambda>
    lambda *arg, **kw: existing(*arg, **kw))
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/sqlalchemy/ext/compiler.py", line 442, in __call__
    return fn(element, compiler, **kw)
  File "/usr/local/share/Bookie/local/lib/python2.7/site-packages/alembic/ddl/mysql.py", line 105, in _mysql_drop_constraint
    "No generic 'DROP CONSTRAINT' in MySQL - "
NotImplementedError: No generic 'DROP CONSTRAINT' in MySQL - please specify constraint type
mitechie commented 10 years ago

Thanks for the report. I'll work on getting mysql into CI and will have to investigate what tweaks to the db migration needs to be done for mysql.

http://ci.bookie.io:8080/

mitechie commented 10 years ago

I've got this replicated in CI and will work on getting that to pass.

http://ci.bookie.io:8080/job/bookie-mysql/6/console

mitechie commented 10 years ago

So in chasing this down the migration needs to be adjusted to drop the right constraint in the right way. Note that mysql required the "type" to be specified per http://alembic.readthedocs.org/en/latest/ops.html#alembic.operations.Operations.drop_constraint:

I've dumped the constraints out in order to figure out what they are. Will have to build a detection that this migration is running in mysql and use the tweaked names/etc to drop the right constraint so the migration can take place.

So lots of data pulled, but more work to be done.

+--------------------+-------------------+-----------------------+----------------+---------------------------+-----------------+
| CONSTRAINT_CATALOG | CONSTRAINT_SCHEMA | CONSTRAINT_NAME       | TABLE_SCHEMA   | TABLE_NAME                | CONSTRAINT_TYPE |
+--------------------+-------------------+-----------------------+----------------+---------------------------+-----------------+
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | activations               | PRIMARY KEY     |
| def                | jenkins_bookie    | activations_ibfk_1    | jenkins_bookie | activations               | FOREIGN KEY     |
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | bmark_readable            | PRIMARY KEY     |
| def                | jenkins_bookie    | bmark_readable_ibfk_1 | jenkins_bookie | bmark_readable            | FOREIGN KEY     |
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | bmark_tags                | PRIMARY KEY     |
| def                | jenkins_bookie    | bmark_tags_ibfk_1     | jenkins_bookie | bmark_tags                | FOREIGN KEY     |
| def                | jenkins_bookie    | bmark_tags_ibfk_2     | jenkins_bookie | bmark_tags                | FOREIGN KEY     |
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | bmarks                    | PRIMARY KEY     |
| def                | jenkins_bookie    | username              | jenkins_bookie | bmarks                    | UNIQUE          |
| def                | jenkins_bookie    | bmarks_ibfk_1         | jenkins_bookie | bmarks                    | FOREIGN KEY     |
| def                | jenkins_bookie    | bmarks_ibfk_2         | jenkins_bookie | bmarks                    | FOREIGN KEY     |
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | import_queue              | PRIMARY KEY     |
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | logging                   | PRIMARY KEY     |
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | stats_bookmarks           | PRIMARY KEY     |
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | tags                      | PRIMARY KEY     |
| def                | jenkins_bookie    | name                  | jenkins_bookie | tags                      | UNIQUE          |
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | url_hash                  | PRIMARY KEY     |
| def                | jenkins_bookie    | PRIMARY               | jenkins_bookie | users                     | PRIMARY KEY     |
| def                | jenkins_bookie    | email                 | jenkins_bookie | users                     | UNIQUE          |
| def                | jenkins_bookie    | username              | jenkins_bookie | users                     | UNIQUE          |
| def                | mysql             | PRIMARY               | mysql          | columns_priv              | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | db                        | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | event                     | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | func                      | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | help_category             | PRIMARY KEY     |
| def                | mysql             | name                  | mysql          | help_category             | UNIQUE          |
| def                | mysql             | PRIMARY               | mysql          | help_keyword              | PRIMARY KEY     |
| def                | mysql             | name                  | mysql          | help_keyword              | UNIQUE          |
| def                | mysql             | PRIMARY               | mysql          | help_relation             | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | help_topic                | PRIMARY KEY     |
| def                | mysql             | name                  | mysql          | help_topic                | UNIQUE          |
| def                | mysql             | PRIMARY               | mysql          | host                      | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | ndb_binlog_index          | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | plugin                    | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | proc                      | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | procs_priv                | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | proxies_priv              | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | servers                   | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | tables_priv               | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | time_zone                 | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | time_zone_leap_second     | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | time_zone_name            | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | time_zone_transition      | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | time_zone_transition_type | PRIMARY KEY     |
| def                | mysql             | PRIMARY               | mysql          | user                      | PRIMARY KEY     |
+--------------------+-------------------+-----------------------+----------------+---------------------------+-----------------+
jcharaoui commented 10 years ago

It's working for me now.

mitechie commented 10 years ago

That's awesome, the test suite still doesn't pass on mysql but glad things are working for you. We've got more to do, but getting the new GSoC features out the door on a new server has taken priority over getting mysql passing.