Database create fails on association

[Studnicky]

Hey there. Working on a slack-bot that requires his own pokemon database instance, was hoping I could just snag from your app. Ran into the following when I ran your build.py:

>>> execfile('data/v2/build.py')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "data/v2/build.py", line 50, in <module>
    clearTable(Language)
  File "data/v2/build.py", line 34, in clearTable
    model.objects.all().delete()
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 556, in delete
    collector.collect(del_query)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/deletion.py", line 168, in collect
    reverse_dependency=reverse_dependency)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/deletion.py", line 79, in add
    if not objs:
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 145, in __nonzero__
    self._fetch_all()
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 966, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 265, in iterator
    for row in compiler.results_iter():
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 700, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
ProgrammingError: relation "pokemon_v2_language" does not exist
LINE 1: ...."iso3166", "pokemon_v2_language"."official" FROM "pokemon_v...
                                                             ^

I unfortunately do not know django well enough yet to fix this.

[Studnicky]

Successfully built database using sqlite3.

The problem lies in the postgresql build. Sorry I can't be more helpful than that right now.

[kengorab]

I'm having the same problem as you. What did you do to get this working?

[Studnicky]

Haven't had time yet. The issue is the foreign key constraints being built in the wrong order, thus making postgres fail when it attempts an integrity lookup.

I'm pretty sure it will build properly if the data relations are removed, but I'd prefer to keep them. I also know postgres has a DEFERRABLE transaction flag that will prevent it from freaking out over key constraints, but I'm unsure of the implementation yet.

Ideally, the constraints need to be set to deferred, then run the build as a transaction.

If you get it working before I do, please let me know.

[phalt]

@Studnicky Why aren't you just using the API for your slackbot?

[Studnicky]

For convenience and speed, I want to alter the data structure a bit. It simply doesn't make sense to make four of five calls and work with all those promises to get all the necessary data from the API when I could just be retrieving it from a local database with a single transaction instead.

The issue obviously isn't with the data or API itself, it's the postgres build script.

[phalt]

If you just want data, then you should use https://github.com/veekun/pokedex instead.

[phalt]

Sorry I was confused, I assumed you wanted the data and I shared a link :)

[zaneadix]

So I just dug into this issue. Started a clean db and ran the build script. The error you're getting should only be happening if you haven't run migrations. I really dont think theres an issue with the execution order of Foreign Keys constraints. There was, however, a small issue keeping migrations from executing that should be taken car of in pull request https://github.com/phalt/pokeapi/pull/55 that I just submitted.

If you pull this branch of mine down then run

python manage.py migrate

you should be good to go with running the build script. Please report back so we can close this issue if building works.

[Studnicky]

That was it. I'd not run the migrations, assumed they were part of the build script.

[Studnicky]

Actually no, that wasn't it. Migrations are failing now.

python manage.py migrate System check identified some issues:

WARNINGS: ?: (1_6.W001) Some project unittests may not execute as expected. HINT: Django 1.6 introduced a new default test runner. It looks like this project was generated using Django 1.5 or earlier. You should ensure your tests are all running & behaving as expected. See https://docs.djangoproject.com/en/dev/releases/1.6/#new-test-runner for more information. Operations to perform: Synchronize unmigrated apps: corsheaders Apply all migrations: hits, sessions, admin, pokemon_v2, sites, auth, tastypie, contenttypes Synchronizing apps without migrations: Creating tables... Creating table corsheaders_corsmodel Installing custom SQL... Installing indexes... Running migrations: Applying contenttypes.0001_initial... OK Applying auth.0001_initial... OK Applying admin.0001_initial... OK Applying hits.0001_initial... OK Applying pokemon_v2.0001_squashed_0013_auto_20150420_0114...Traceback (most recent call last): File "manage.py", line 10, in execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/dist-packages/django/core/management/init.py", line 385, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/dist-packages/django/core/management/init.py", line 377, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 288, in run_from_argv self.execute(_args, _options.dict) File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 338, in execute output = self.handle(_args, _options) File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 160, in handle executor.migrate(targets, plan, fake=options.get("fake", False)) File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 63, in migrate self.apply_migration(migration, fake=fake) File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 97, in apply_migration migration.apply(project_state, schema_editor) File "/usr/local/lib/python2.7/dist-packages/django/db/backends/schema.py", line 83, in exit self.execute(sql) File "/usr/local/lib/python2.7/dist-packages/django/db/backends/schema.py", line 99, in execute cursor.execute(sql, params) File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in exit six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) django.db.utils.ProgrammingError: column "introduces_in_version_group_id" does not exist

[Studnicky]

Solved with reference to: https://github.com/phalt/pokeapi/issues/52#issuecomment-139842562

Assuming I can now build the database, I'll make a pull including this fix and postgres docs.