Django 3 incompatibilities

[jayvdb]

Quite a few deprecated imports were dropped, and the code now needs to import from the new location.

Also django.utils.six was dropped, and needs to be replaced with direct import of six.

The model unicode helper was dropped, and can possibly be discarded as Django 3 doesnt run on Python 2 iirc.

[jayvdb]

https://github.com/barsgroup/m3-django-compat/tree/master/src/m3_django_compat has two of them.

[pakal]

Thanks for the inputs, I'll look at updating DCP to Django3 (unless you can provide some PR in the meantime)

[jayvdb]

I have been preparing to do the optparse to argparse issue; less motivation atm a decent Django 3 update, but I am moderately interested in it, so happy to help.

[pakal]

I'm currently on restoring the deleted py2 compatibility aliases, it should be ok

[jayvdb]

Some random failures on Django 3. I'll help sorting them out later.

https://pypi.org/project/django-preflight/ - https://launchpad.net/django-preflight

  File "/usr/lib/python3.8/site-packages/preflight/models.py", line 15, in <module>
    from django.core.cache import get_cache
ImportError: cannot import name 'get_cache' from 'django.core.cache' (/usr/lib/python3.8/site-packages/django/core/cache/__init__.py)

  File "/usr/lib/python3.8/site-packages/test_utils/management/commands/testmaker.py", line 11, in <module>
    from test_utils.testmaker import Testmaker
  File "/usr/lib/python3.8/site-packages/test_utils/testmaker/__init__.py", line 7, in <module>
    from test_utils.management.commands.relational_dumpdata import _relational_dumpdata
  File "/usr/lib/python3.8/site-packages/test_utils/management/commands/relational_dumpdata.py", line 7, in <module>
    from django.db.models import get_app, get_apps, get_models
ImportError: cannot import name 'get_app' from 'django.db.models' (/usr/lib/python3.8/site-packages/django/db/models/__init__.py)

(django-evolution also does the same as django-test-utils)

  File "/usr/lib/python3.8/site-packages/djcelery/admin.py", line 11, in <module>
    from django.shortcuts import render_to_response
ImportError: cannot import name 'render_to_response' from 'django.shortcuts' (/usr/lib/python3.8/site-packages/django/shortcuts.py)

https://pypi.org/project/django-urls-map/ (the only thing missing from django-extensions show_urls is indentation, so not a high priority)

Traceback (most recent call last):
  File "./manage.py", line 51, in <module>
    main()
  File "./manage.py", line 47, in main
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python3.8/site-packages/django_compat_patcher/fixers/django1_10.py", line 416, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3.8/site-packages/django_urls_map/management/commands/urlmap.py", line 42, in handle
    self._walk_urlresolver("/", rootresolver)
  File "/usr/lib/python3.8/site-packages/django_urls_map/management/commands/urlmap.py", line 47, in _walk_urlresolver
    current_path = parent + self._clear_regexpath(item.regex.pattern)
AttributeError: 'URLResolver' object has no attribute 'regex'

[jayvdb]

  File "/home/abuild/rpmbuild/BUILD/tddspry-0.3.1/tddspry/django/cases.py", line 6, in <module>
    from django.core.servers.basehttp import AdminMediaHandler
ImportError: cannot import name 'AdminMediaHandler' from 'django.core.servers.basehttp' (/usr/lib/python3.8/site-packages/django/core/servers/basehttp.py)

  File "/usr/lib/python3.8/site-packages/curation/models.py", line 5, in <module>
    from django.utils.encoding import smart_unicode
ImportError: cannot import name 'smart_unicode' from 'django.utils.encoding' (/usr/lib/python3.8/site-packages/django/utils/encoding.py)

  File "/usr/lib/python3.8/site-packages/fast_fixtures/management/commands/dumpdata.py", line 21, in handle
    return super(Command, self).handle(*app_labels, **options)
  File "/usr/lib/python3.8/site-packages/django/core/management/commands/dumpdata.py", line 70, in handle
    using = options['database']
KeyError: 'database'

[jayvdb]

  File "/usr/lib/python3.8/site-packages/graceful_session_cleanup/management/commands/graceful_session_cleanup.py", line 45, in handle
    actual_delete_count = cursor.execute("DELETE FROM django_session WHERE expire_date<now() LIMIT %s;", [delete_count])
  File "/usr/lib/python3.8/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/usr/lib/python3.8/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/lib/python3.8/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/lib/python3.8/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/lib/python3.8/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py", line 396, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such function: now

[jayvdb]

    module = __import__(module_name)
  File "/home/abuild/rpmbuild/BUILD/django-autotest-command-1.5.5/autotest/management/commands/autotest.py", line 23, in <module>
    ORIG_HAS_CHANGED = autoreload.code_changed
AttributeError: module 'django.utils.autoreload' has no attribute 'code_changed'

[jayvdb]

  File "/usr/lib/python3.8/site-packages/workon/__init__.py", line 6, in <module>
    from workon.utils import *
  File "/usr/lib/python3.8/site-packages/workon/utils/__init__.py", line 22, in <module>
    from workon.utils.string import *
  File "/usr/lib/python3.8/site-packages/workon/utils/string.py", line 3, in <module>
    from django.utils.text import Truncator, slugify as django_slugify, mark_safe
ImportError: cannot import name 'mark_safe' from 'django.utils.text' (/usr/lib/python3.8/site-packages/django/utils/text.py)

https://stackoverflow.com/questions/58319239/django-where-is-mark-safe-located

[jayvdb]

  File "/usr/lib/python3.8/site-packages/django_test/__init__.py", line 69, in <module>
    from django.test.testcases import disable_transaction_methods, restore_transaction_methods
ImportError: cannot import name 'disable_transaction_methods' from 'django.test.testcases' (/usr/lib/python3.8/site-packages/django/test/testcases.py)

[pakal]

Hm most of these errors are supposed to be solved with, DCP properly enabled, and a Django version recent enough (forward-compatibility fixers are rare, it's mainly backwards-compatibility shims). By default DCP logs the fixers being actually applied (see setting DCP_LOGGING_LEVEL), so it'll be the first thing to check.

If some fixers don't work as expected (especially the easy ones, restoring a utility that Django had removed), I'll need a ticket with the whole "pip freeze" and git commit hash of the tested package, to be able to reproduce the problem.

[jayvdb]

I've done code greps on the repo for names in the above, and there are no fixers appearing in my search results. Maybe there are unreleased fixers?

[pakal]

Sorry I must have been confused, idneed these are new regressions.

I've readded render_to_response() and get_cache() into master branch, I'll have a look at others later.

[pakal]

I've investigated a bit some of these errors :

from django.db.models import get_app, get_apps, get_models seems to reference very old APIs (deprecated <2013), considered private, which have moved a lot since then (cache.py has been turned into registry.py, and methods of these APIs have been renamed here and there). It'd be a quite big effort to provide shims for these private APIs (but PRs welcome).

URLResolver' object has no attribute 'regex seems to be a tricky one, although fetching URLResolver.pattern.regex via a property might do the trick. I might see later.

AdminMediaHandler has been removed in 2012, i'm not sure it's doable to restore all its needed scaffolding (settings have changed since then) in a proportionate time.

"smart_unicode" was a python2-only helper, it seems django-curation now doesn't have it in its depot anymore.

[pakal]

Could you please list the still-present errors, with the exact versions of Python and Django used?

[pakal]

If such failures keep happening with the latest DCP, please keep me in touch ^^