search

varapp / varapp-backend-py

Django backend to varapp
GNU General Public License v3.0
20 stars 9 forks source link

Manage migrate does not work on empty database #1

Closed sndrtj closed 8 years ago

sndrtj commented 8 years ago

Hi there,

I've been trying to set up an instance of varapp, but I'm stuck at the database migrations step. Per the documentation it should be as simple as running as python manage migrate after setting up the database, but this step fails with the following stacktrace:

./manage.py migrate

-----------  << RESTART >> -----------

Settings file: settings.py
--------------------------------------
Traceback (most recent call last):
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/backends/mysql/base.py", line 112, in execute
    return self.cursor.execute(query, args)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/cursors.py", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorvalue
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/connections.py", line 280, in query
    _mysql.connection.query(self, query)
_mysql_exceptions.ProgrammingError: (1146, "Table 'users_db.variants_db' doesn't exist")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/core/management/__init__.py", line 327, in execute
    django.setup()
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/apps/registry.py", line 115, in populate
    app_config.ready()
  File "/home/user/backend/django-varapp-1.0/varapp/apps.py", line 20, in ready
    self.make_connections()
  File "/home/user/backend/django-varapp-1.0/varapp/apps.py", line 47, in make_connections
    for vdb in vdbs:
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/models/query.py", line 258, in __iter__
    self._fetch_all()
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/models/query.py", line 1074, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/models/query.py", line 52, in __iter__
    results = compiler.execute_sql()
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/models/sql/compiler.py", line 848, in execute_sql
    cursor.execute(sql, params)
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/user/varapp_env/lib/python3.4/site-packages/Django-1.9.6-py3.4.egg/django/db/backends/mysql/base.py", line 112, in execute
    return self.cursor.execute(query, args)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/cursors.py", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorvalue
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/home/user/varapp_env/lib/python3.4/site-packages/MySQLdb/connections.py", line 280, in query
    _mysql.connection.query(self, query)
django.db.utils.ProgrammingError: (1146, "Table 'users_db.variants_db' doesn't exist")

The error seems to be at line 47 in varapps/app.py. The VarappConfig object attempt to load all VariantsDb objects before the table has been created.

This not only causes migrate to fail, but pretty much any command of manage.py (even -h does not work)

I've tried the approach outlined here, but - although I do get manage -h to work again - uncommenting varapp from the INSTALLED_APPS list causes another exception (RuntimeError: Model class varapp.models.gemini.GeneDetailed doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS. as might be reasonably expected).

jdelafon commented 8 years ago

Hello sndrtj , Thanks a lot for reporting. It looks like the docs are missing a critical step. I will try again to reinstall the whole app from scratch to see what's missing, and will fix it asap. I have an idea of what happens here. Stay in touch !

jdelafon commented 8 years ago

I published a fix for that bug, along with important changes from our master branch. The (mysql) users db is empty at this point but the app wanted to do something with it. I also fixed the docs following that point, as the Roles entries have to be filled before the Users entries. I could successfully run the backend server from a fresh git clone of the Github version. Tell me if you could not.

sndrtj commented 8 years ago

Thanks for the quick reply! It seems to work now :-)

One small issue I had is with the last loaddata command. Loading the data_dbaccess.json will fail because the mydb.db that is created is is obviously empty. You might want to add a sentence about first having to create a Gemini database under that name before running said command.

jdelafon commented 8 years ago

Very true, thanks. Actually what I want is to ship a small test database so that it works right away. But I must be very careful with what data I share publicly...

My main concern now is that because I have to hard-code models in Django, it will complain if one of the expected annotations is not found in the Gemini db. There is a new chapter in the docs about what commands we run with both VEP and Gemini after calling with GATK. Please tell me if you run into such an issue. As a quick fix I may deactivate some annotation on this branch.