Django 1.7 compatibilty

[jieter]

51 was my naive attempt to use django-pyodbc with 1.7b1, it seems to be more complicated...

This is what I've found so far:

1. A ProgrammingError: Attempt to use a closed cursor is raised on select queries.commit comment If I replace cursor.close() with pass on django/db/models/sql/compiler.sql:820, selecting stuff in the shell works again. But it obviously needs to be fixed at django-pyodbcs side.
2. The use of TEST_* prefixes in the DATABASES setting is deprecated, but django-pyodbc assumes the keys exist. This makes test database creation fail.

3. Schema editors. Django 1.7 ships with migrations, so the BaseDatabaseWrapper defines a method schema_editor which raises NotImplementedError.

   ...
   File "/home/jieter/.virtualenvs/pyodbc-test/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 546, in schema_editor
   raise NotImplementedError('subclasses of BaseDatabaseWrapper may require a schema_editor() method')
   NotImplementedError: subclasses of BaseDatabaseWrapper may require a schema_editor() method

   If I add a method returning a simple SchemaEditor like this:

   def schema_editor(self, *args, **kwargs):
      "Returns a new instance of this backend's SchemaEditor"
      return DatabaseSchemaEditor(self, *args, **kwargs)

   (together with a DatabaseSchemaEditor extending BaseDatabaseSchemaEditor), I get a TransactionManagementError when ./manage.py runserver:

   ...
   File "/home/jieter/.virtualenvs/pyodbc-test/local/lib/python2.7/site-packages/django/db/migrations/loader.py", line 148, in build_graph
    self.applied_migrations = recorder.applied_migrations()
   File "/home/jieter/.virtualenvs/pyodbc-test/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 48, in applied_migrations
    self.ensure_schema()
   File "/home/jieter/.virtualenvs/pyodbc-test/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 41, in ensure_schema
    with self.connection.schema_editor() as editor:
   File "/home/jieter/.virtualenvs/pyodbc-test/local/lib/python2.7/site-packages/django/db/backends/schema.py", line 74, in __enter__
    atomic(self.connection.alias, self.connection.features.can_rollback_ddl).__enter__()
   File "/home/jieter/.virtualenvs/pyodbc-test/local/lib/python2.7/site-packages/django/db/transaction.py", line 274, in __enter__
    "The outermost 'atomic' block cannot use "
   django.db.transaction.TransactionManagementError: The outermost 'atomic' block cannot use savepoint = False when autocommit is off.

idea's?

[jordanorc]

Issue 2 can be solved with this patch: https://gist.github.com/jordanorc/eadb5b1fd4bcd7fa232f. It need more improvements, mainly to prevent repeating TEST_* in many places like is done now.

[dlo]

@jordanorc awesome! could you submit a PR?

[jordanorc]

Yes, I will do that. Just let me see if I can help with the other issues.

[jordanorc]

PR created: https://github.com/lionheart/django-pyodbc/pull/55.

Django 1.7 Standalone scripts should configure Django with django.setup(). An exemple can be found here: https://github.com/django/django/blob/master/tests/runtests.py (line 128). I will create another PR.

I'm having a strange error with tests. When it pass firstly in _get_connection_string (base.py), settings_dict["NAME"] has value. After that, the value of it is None, raising exception:

Traceback (most recent call last):
  File "/Users/jordanorc/development/projects/django-pyodbc/tests/runtests.py", line 352, in <module>
    options.failfast, args)
  File "/Users/jordanorc/development/projects/django-pyodbc/tests/runtests.py", line 193, in django_tests
    test_labels or get_installed(), extra_tests=extra_tests)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/test/runner.py", line 147, in run_tests
    old_config = self.setup_databases()
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/test/runner.py", line 109, in setup_databases
    return setup_databases(self.verbosity, self.interactive, **kwargs)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/test/runner.py", line 297, in setup_databases
    verbosity, autoclobber=not interactive)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/backends/creation.py", line 354, in create_test_db
    self._create_test_db(verbosity, autoclobber)
  File "/Users/jordanorc/development/projects/django-pyodbc/django_pyodbc/creation.py", line 104, in _create_test_db
    return super(DatabaseCreation, self)._create_test_db(verbosity, autoclobber)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/backends/creation.py", line 409, in _create_test_db
    with self._nodb_connection.cursor() as cursor:
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/backends/__init__.py", line 159, in cursor
    cursor = utils.CursorWrapper(self._cursor(), self)
  File "/Users/jordanorc/development/projects/django-pyodbc/django_pyodbc/base.py", line 273, in _cursor
    connstr = self._get_connection_string()#';'.join(cstr_parts)
  File "/Users/jordanorc/development/projects/django-pyodbc/django_pyodbc/base.py", line 208, in _get_connection_string
    raise ImproperlyConfigured('You need to specify NAME in your Django settings file.')
django.core.exceptions.ImproperlyConfigured: You need to specify NAME in your Django settings file.

idea's?

[dlo]

Hmm, not sure, @tax?

[jieter]

Yes, due to the method of making a connection with no db selected: https://github.com/django/django/blob/stable/1.7.x/django/db/backends/creation.py#L36

django commit django ticket

[jordanorc]

I will look that. Thanks @jieter!

[jordanorc]

You were absolutely right @jieter. It's working now. Going to schema_editor. Someone is working on migrations?

[jordanorc]

I copied all tests from Django 1.7 (including migrations) and everything seems to be working after follow the tip from @jieter about issue 1 (ProgrammingError: Attempt to use a closed cursor).

I'm running tests now (no problems yet). I create a basic Schema Editor with help of this link: https://bitbucket.org/andrewgodwin/south/src/0e17add9b5e0f30f7cf5acf47961eea6a7f4032c/south/db/sql_server/pyodbc.py?at=default. It need more improvements, but is working.

I wait until we find a solution for issue 1 or create a PR with what I alread have?

[dlo]

We definitely shouldn't be closing a cursor for a connection that's already closed, so I think it'd be best if we fix 1 as well.

What line was throwing that error on django-pyodbc's side?

[jordanorc]

Traceback shows:

Traceback (most recent call last):
  File "tests/runtests.py", line 357, in <module>
    options.failfast, args)
  File "tests/runtests.py", line 198, in django_tests
    test_labels or get_installed(), extra_tests=extra_tests)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/test/runner.py", line 147, in run_tests
    old_config = self.setup_databases()
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/test/runner.py", line 109, in setup_databases
    return setup_databases(self.verbosity, self.interactive, **kwargs)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/test/runner.py", line 297, in setup_databases
    verbosity, autoclobber=not interactive)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/backends/creation.py", line 368, in create_test_db
    test_database=True)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/core/management/__init__.py", line 167, in call_command
    return klass.execute(*args, **defaults)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/core/management/base.py", line 337, in execute
    output = self.handle(*args, **options)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 62, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/migrations/executor.py", line 14, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/migrations/loader.py", line 48, in __init__
    self.build_graph()
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/migrations/loader.py", line 148, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 49, in applied_migrations
    return set(tuple(x) for x in self.Migration.objects.values_list("app", "name"))
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/query.py", line 141, in __iter__
    self._fetch_all()
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/query.py", line 961, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/query.py", line 1200, in iterator
    for row in self.query.get_compiler(self.db).results_iter():
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 694, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 816, in execute_sql
    cursor.close()
pyodbc.ProgrammingError: Attempt to use a closed cursor.

[jordanorc]

One more thing I discovered: I updated development version using pytds instead of pyodbc and I have no more the error pointed by @jieter (ProgrammingError: Attempt to use closed cursor), which leads me to believe it is a related problem with pyobc.

Any suggestions?

[dlo]

In the traceback above, I see django-pytds, not django-pyodbc. Was it another package that had this error?

[jordanorc]

It's just the name of my virtualenv.

[dlo]

This traceback seems right...what is an example of code you're running that causes this error?

[jordanorc]

It is difficult to know because I can not see which tests the error comes.

I print the last executed sql:

Importing application aggregation
Importing application aggregation_regress
Importing application backends
Importing application base
Importing application bulk_create
Importing application cache
Importing application commands_sql
Importing application custom_columns
Importing application custom_columns_regress
Importing application custom_lookups
Importing application custom_managers
Importing application custom_managers_regress
Importing application custom_methods
Importing application custom_pk
Importing application datatypes
Importing application dates
Importing application datetimes
Importing application db_backends
Importing application db_typecasts
Importing application defer
Importing application defer_regress
Importing application delete
Importing application delete_regress
Importing application distinct_on_fields
Importing application expressions
Importing application force_insert_update
Importing application foreign_object
Importing application generic_relations
Importing application generic_relations_regress
Importing application get_earliest_or_latest
Importing application get_object_or_404
Importing application get_or_create
Importing application indexes
Importing application initial_sql_regress
Importing application inspectdb
Importing application introspection
Importing application known_related_objects
Importing application lookup
Importing application m2m_and_m2o
Importing application m2m_intermediary
Importing application m2m_multiple
Importing application m2m_recursive
Importing application m2m_regress
Importing application m2m_signals
Importing application m2m_through
Importing application m2m_through_regress
Importing application m2o_recursive
Importing application many_to_many
Importing application many_to_one
Importing application many_to_one_null
Importing application many_to_one_regress
Importing application max_lengths
Importing application migrate_signals
Importing application migrations
Importing application model_inheritance
Importing application model_inheritance_regress
Importing application multiple_database
Importing application mutually_referential
Importing application nested_foreign_keys
Importing application null_fk
Importing application null_fk_ordering
Importing application null_queries
Importing application one_to_one
Importing application one_to_one_regress
Importing application or_lookups
Importing application order_with_respect_to
Importing application ordering
Importing application pagination
Importing application prefetch_related
Importing application queries
Importing application raw_query
Importing application reserved_names
Importing application reverse_lookup
Importing application reverse_single_related
Importing application select_for_update
Importing application select_related
Importing application select_related_onetoone
Importing application select_related_regress
Importing application string_lookup
Importing application tablespaces
Importing application transactions
Importing application update
Importing application update_only_fields
Importing application admin
Importing application admindocs
Importing application auth
Importing application comments
Importing application contenttypes
Importing application flatpages
Importing application formtools
Importing application gis
Importing application humanize
Importing application messages
Importing application redirects
Importing application sessions
Importing application sitemaps
Importing application sites
Importing application staticfiles
Importing application syndication
Importing application webdesign
Creating test database for alias 'default' ('test_defaultdb')...
SELECT CAST(SERVERPROPERTY('EngineEdition') as integer)
CREATE DATABASE [test_defaultdb] 
Type 'yes' if you would like to try deleting the test database 'test_defaultdb', or 'no' to cancel: Got an error creating the test database: ('42000', "[42000] [FreeTDS][SQL Server]Database 'test_defaultdb' already exists. Choose a different database name. (1801) (SQLExecDirectW)")
yes
Destroying old test database 'default'...
DROP DATABASE [test_defaultdb]
CREATE DATABASE [test_defaultdb] 
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE'
CREATE TABLE [django_migrations] ([id] int IDENTITY (1, 1) NOT NULL PRIMARY KEY, [app] nvarchar(255) NOT NULL, [name] nvarchar(255) NOT NULL, [applied] datetime NOT NULL)
SELECT [django_migrations].[app], [django_migrations].[name] FROM [django_migrations]
Traceback (most recent call last):
  File "/Users/jordanorc/development/projects/django-pyodbc/tests/runtests.py", line 357, in <module>
    options.failfast, args)
  File "/Users/jordanorc/development/projects/django-pyodbc/tests/runtests.py", line 198, in django_tests
    test_labels or get_installed(), extra_tests=extra_tests)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/test/runner.py", line 147, in run_tests
    old_config = self.setup_databases()
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/test/runner.py", line 109, in setup_databases
    return setup_databases(self.verbosity, self.interactive, **kwargs)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/test/runner.py", line 297, in setup_databases
    verbosity, autoclobber=not interactive)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/backends/creation.py", line 368, in create_test_db
    test_database=True)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/core/management/__init__.py", line 167, in call_command
    return klass.execute(*args, **defaults)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/core/management/base.py", line 337, in execute
    output = self.handle(*args, **options)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 62, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/migrations/executor.py", line 14, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/migrations/loader.py", line 48, in __init__
    self.build_graph()
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/migrations/loader.py", line 148, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 49, in applied_migrations
    return set(tuple(x) for x in self.Migration.objects.values_list("app", "name"))
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/query.py", line 141, in __iter__
    self._fetch_all()
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/query.py", line 961, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/query.py", line 1200, in iterator
    for row in self.query.get_compiler(self.db).results_iter():
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 694, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/Users/jordanorc/development/env/django-pytds/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 816, in execute_sql
    cursor.close()
pyodbc.ProgrammingError: Attempt to use a closed cursor.

The codes seems to be related to django migrations. Some SELECT queries are successfully executed.

Any ideas?

[jieter]

If I try with a test app with just a single model, every <model>.objects.get(pk=1) raises the ProgrammingError: Attempt to use a closed cursor.

[jordanorc]

Base.py doesn't implement the methods of the newest versions of BaseDatabaseWrapper class (django 1.6 +).

We are overriding the _cursor instead of implementing the methods get_connection_params, get_new_connection, init_connection_state and create_cursor method. It's probably due for backwards compatibility.

What I did was update the class in a way that works with the newer and older versions. Thus, it is more organized. It's interesting to do this update?

[jieter]

@dlo

I'd like to support the effort to make this compatible to django 1.7, but it seems to stall. I propose merging the 1.7-related pulls in a feature branch up to a working level and then back into the master, so we can speed up the development...

[dlo]

@jieter Good idea--I just created a django-1.7 branch. Any interest in taking lead on this?

[jieter]

@dlo, I do not have much in-depth knowledge of odbc nor djagno, but I'm willing to take the lead...

[GordonFreemem]

Hi, any updates on this issue? I would like to start a new project on Django 1.7, but alway get the 3rd error "NotImplementedError: subclasses of BaseDatabaseWrapper may require a schema_editor() method" on Windows and Linux systems. If I downgrade django to 1.6 and syncdb everything works like expected.

My environments: Python 2.7.8 (Windows 8.1) / Python 2.7.5 (CentOS7) django 1.7 pyodbc 3.0.7 django_pyodbc 0.2.5

[olivergeorge]

@jordanorc would you mind posting your patch. I'd love to have tests working. Thanks.

[olivergeorge]

Ah, I think I've found it in your repo under django17 branch

[tsobeng]

Hi all,

I have the same error,

I use django1.7 and pyobdc on windows 7

when I try to connected to an MS Access DB, I have this error

raise NotImplementedError('subclasses of BaseDatabaseWrapper may require a schema_editor() method') NotImplementedError: subclasses of BaseDatabaseWrapper may require a schema_editor() method

Thanks

[deumo]

I just got the same error on Windows 7. Which django version can correct this or which patch should I apply whith Django 1.7 ? I use pyodbc 3.0.7.

Thanks.

[tax]

Django 1.7 is not yet supported so you might try using django 1.6

[GordonFreemem]

I switched to django-pyodbc-azure. Works perfect with Django 1.7 and MS SQL-Server.

[gereltod-g]

same problem here. django 1.7

[deumo]

thank you for your reply.

2015-02-10 17:54 GMT+01:00 Gordon Schultz notifications@github.com:

I switched to django-pyodbc-azure. Works perfect with Django 1.7 and MS SQL-Server.

— Reply to this email directly or view it on GitHub https://github.com/lionheart/django-pyodbc/issues/52#issuecomment-73736447 .

[RossRogers]

For "Issue 1" - A ProgrammingError: Attempt to use a closed cursor, I believe this is just a behavior difference between pyodbc and other database backends. If you look at the Django trunk for file django/db/models/sql/compiler.py, it has these two snippets:

849 result = cursor_iter(
850     cursor, self.connection.features.empty_fetchmany_value,
851     self.col_count
852 )
853 if not self.connection.features.can_use_chunked_reads:
854     try:
855         # If we are using non-chunked reads, we return the same data
856         # structure as normally, but ensure it is all read into memory
857         # before going any further.
858         return list(result)
859     finally:
860         # done with the cursor
861         cursor.close()

and the cursor_iter() definition :

1129 def cursor_iter(cursor, sentinel, col_count):
        [...]
1134     try:
1135         for rows in iter((lambda: cursor.fetchmany(GET_ITERATOR_CHUNK_SIZE)),
1136                          sentinel):
1137             yield [r[0:col_count] for r in rows]
1138     finally:
1139         cursor.close()

849 creates an iterator with a built-in close() function. 858 invokes the iterator to completion, calling that close() function, and 861 invokes the close() function a 2nd time.

So I think the solution might be to wrapper the cursor's close() function to silence the ProgrammingError exceptions resulting from closing a pyodbc cursor more than once:

diff --git a/django_pyodbc/base.py b/django_pyodbc/base.py
index 747f36e..47099ae 100644
--- a/django_pyodbc/base.py
+++ b/django_pyodbc/base.py
@@ -367,6 +367,12 @@ class CursorWrapper(object):
         self.last_params = ()
         self.encoding = encoding

+    def close(self):
+        try:
+            self.cursor.close()
+        except Database.ProgrammingError:
+            pass
+
     def format_sql(self, sql, n_params=None):
         if not self.driver_supports_utf8 and isinstance(sql, text_type):
             # Older FreeTDS (and other ODBC drivers?) don't support Unicode yet, so

Let me know if this is the correct path and whether I should create a pull request for it.

[dlo]

@RossRogers this looks like a solid path to pursue. :+1: on a PR.

[steve17767]

I don't understand if this problem was solved or not.

I'm upgrading from Djano 1.6 to 1.7. Python 2.7.6. pyodbc 3.0.7 django_pyodbc 0.2.5 Ubuntu 12.04

Everything went as planned except for accessing remote database tables (MySQL and Ms-SQL Server). All of the tables are configured "manage=false". When I try to access the table I get:

RuntimeError: Conflicting 'bugview_deferred_description_disposition_divisioaf59cb28117de5eb6ddfea0476dab601' models in application 'nvbugsdw': <class 'bug_metrics.models_bugs.BugView_Deferred_Description_Disposition_Divisioaf59cb28117de5eb6ddfea0476dab601'> and <class 'bug_metrics.models_bugs.BugView_Deferred_Description_Disposition_Divisioaf59cb28117de5eb6ddfea0476dab601'>

To fix that I was instructed to makemigrations on the remote (not managed by Django) DB (see http://stackoverflow.com/questions/34620588/conflicting-models-in-application-with-external-database-tables). So I tried and got:

NotImplementedError: subclasses of BaseDatabaseWrapper may require a schema_editor() method

Any suggestions? I've been stuck for a couple of days.

Thanks in advance for any help.

[RossRogers]

It looks like this class is somehow getting included twice:

bug_metrics.models_bugs.BugView_Deferred_Description_Disposition_Divisioaf59cb28117de5eb6ddfea0476dab601

Do you have the bug_metrics "app" installed twice?

[steve17767]

No. Problem appeared when I upgrade from 1.6 to 1.7. It happens only on tables that are defined on remote MS-SQL Servers. The model.py file containing the definition is incldued in multiple other files (views, commands, etc.) but the application is only loaded once.

I have 11 apps. 2 access remote DBs the other 9 access local MySQL DB tables. Only the two apps that access the remote DB display this problem.

[RossRogers]

So, the table BugView_Deferred_Description_Disposition_Divisioaf59cb28117de5eb6ddfea0476dab601 has Meta field managed=False like?

class Meta:
    managed = False

Did it transition from managed to un-managed?

I've seen the Django migration system get screwed up before. If you don't think you'll need to rollback any migrations ( you can always create a forward migration at a later date that undoes the edits), then I would try:

• Nuking all files in the bug_metrics/migrations directory
• Nuke all rows in table django_migrations

• Recreate an initial migration with:

  python manage.py makemigrations bug_metrics

• Then fake out the migration, as it is already in a known state:

  python manage.py migrate --fake bug_metrics

(Hopefully done in a copy of the real databases so you're not changing the live DB)

[RossRogers]

One more thought: If you want to root cause this, I'd launch a debugger like winpdb:

Then put this line:

import rpdb2; rpdb2.start_embedded_debugger('foo')

in file django/apps/registry.py right before the RuntimeError is thrown:

raise RuntimeError(
                "Conflicting '%s' models in application '%s': %s and %s." %
                (model_name, app_label, app_models[model_name], model))

Then:

• run your command that throws the error
• wait for it to hang ( right before that error)
• launch winpdb GUI
• File->Attach, password 'foo'
• Click up the stack and see how it got to that error.

[steve17767]

Don't know if this makes any difference or not:

The application bug_metrics has 3 models*.py files. Two of them define local database tables. The third one defines classes for managed = False tables that were created by hand (may have used inspectdb tool). So the bug_metrics applicaiton has a mix of tables.

I migrated bug_metrics (cleared out the migrations directory of everything but init.py, ran makemiragtion and migrate) that created the FAKE migrations.

The "bugview" table ("bug_metrics.modelsbugs.BugView...") has always been manged = False.

class BugView(models.Model):
    BugId            = models.BigIntegerField(primary_key=True)
    Module           = models.CharField(max_length=128, null=False)
    Severity         = models.CharField(max_length=50, null=False)
    Disposition      = models.CharField(max_length=64, null=False)
    Division         = models.CharField(max_length=64, null=False)
. . .
class Meta:
    app_label = 'nvbugsdw'
    managed   = False
    db_table  = 'bugview'

[steve17767]

Ok. And I'll try the debugger approach too.

[RossRogers]

Is that app_label= 'nvbugsdw' necessary? If you nuke that line, does this issue disappear?

[steve17767]

Same issue with or without it.

[RossRogers]

Interesting. I'd definitely go the debugger route then. If you haven't done that before, it is extremely powerful when root-causing issues.

[steve17767]

I did notice one other oddity. When I run

./manage.py runserver --nothreading 0.0.0.0:8081

With print statements in manage.py, manage.py seems to be run twice. Is that expected?

[RossRogers]

manage.py will reload everything anytime a file in any loaded apps is changed.

[steve17767]

Thanks for the debug tip. Found this:

In my view I define: context_object_name = 'open_bugs' model = BugView

I override get_queryset() to retrieve the list of record I want: bv = BugView() sql = bv.get_open_bugs_for_user_sql(self.filter_username) bugs = BugView.objects.raw(sql)

I then iterate over the list to cont certain states, etc. for bug in bugs: if bug.State == ...

I pass the "bugs" results to context:

get_context_data(self, **kwargs):
    context = super(UserBugsView, self).get_context_data(**kwargs)

Then in the template I iterate over the passed in "open_bugs" (which I have already iterated over once):

{% for bug in open_bugs %}
<tr>
<td><a href="http://nvbugs/{{ bug.BugId }}" target="_blank">{{ bug.BugId }}</a> &nbsp;&nbsp;</td>
...

Looking at dist-packages/django/apps/registry.py, line 210-ish

def register_model(self, app_label, model):
    # Since this method is called when models are imported, it cannot
    # perform imports because of the risk of import loops. It mustn't
    # call get_app_config().
    model_name = model._meta.model_name
    app_models = self.all_models[app_label]
    print "-----> app_label  = "+app_label
    print "-- --> model_name = "+model_name
    print "- - -> app_models = "+str(app_models)
    if model_name in app_models:
        raise RuntimeError(
            "Conflicting '%s' models in application '%s': %s and %s." %
            (model_name, app_label, app_models[model_name], model))
    app_models[model_name] = model
    self.clear_cache()

(I added the print lines)

When the views code counts the bugs I see in the trace:

-----> app_label  = nvbugsdw
-- --> model_name = bugview_deferred_description_disposition_divisioaf59cb28117de5eb6ddfea0476dab601
- - -> app_models = OrderedDict([('bugview', <class 'bug_metrics.models_bugs.BugView'>)])

and then when the open_bugs gets to the template I see in the trace:

-----> app_label  = nvbugsdw
-- --> model_name = bugview_deferred_description_disposition_divisioaf59cb28117de5eb6ddfea0476dab601
- - -> app_models = OrderedDict([('bugview', <class 'bug_metrics.models_bugs.BugView'>), ('bugview_deferred_description_disposition_divisioaf59cb28117de5eb6ddfea0476dab601', <class 'bug_metrics.models_bugs.BugView_Deferred_Description_Disposition_Divisioaf59cb28117de5eb6ddfea0476dab601'>)])

which raises the error.

This is the same code I ran under 1.5 and 1.6 Django.

Thoughts?

[RossRogers]

That code you just pasted is not the latest 1.7 minor release number. sudo pip install django 1.7.11. If you do pip install 1.7 it literally installs 1.7, not the highest minor number. To find the versions, do :

pip install yolk # might need sudo pip ...
yolk -V django

The latest 1.7 minor/bug release number is 1.7.11.

So, let's start with the Django 1.7 that has the most bug fixes already.

[steve17767]

UNBELIEVABLE!!!! I'm such an idiot.

Thank you so much for noticing that. It works now.

[RossRogers]

That has bit me before too. I think pip install django==1.7 should pick the highest bug release as well. Oh well.

Yay team.

[steve17767]

Yes, I just assumed it would. Didn't some general say something about assumption is the mother of all ..."

Again, thanks.

[RossRogers]

I believe we have generic 1.7 support on the trunk as I am using it (shame on me). For specific 1.7 bugs, please file a new ticket.