Changelog
### 2.2.4
```
==========================
*August 1, 2019*
Django 2.2.4 fixes security issues and several bugs in 2.2.3.
CVE-2019-14232: Denial-of-service possibility in ``django.utils.text.Truncator``
================================================================================
If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods
were passed the ``html=True`` argument, they were extremely slow to evaluate
certain inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``chars()`` and ``words()`` methods are used to implement the
:tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template
filters, which were thus vulnerable.
The regular expressions used by ``Truncator`` have been simplified in order to
avoid potential backtracking issues. As a consequence, trailing punctuation may
now at times be included in the truncated output.
CVE-2019-14233: Denial-of-service possibility in ``strip_tags()``
=================================================================
Due to the behavior of the underlying ``HTMLParser``,
:func:`django.utils.html.strip_tags` would be extremely slow to evaluate
certain inputs containing large sequences of nested incomplete HTML entities.
The ``strip_tags()`` method is used to implement the corresponding
:tfilter:`striptags` template filter, which was thus also vulnerable.
``strip_tags()`` now avoids recursive calls to ``HTMLParser`` when progress
removing tags, but necessarily incomplete HTML entities, stops being made.
Remember that absolutely NO guarantee is provided about the results of
``strip_tags()`` being HTML safe. So NEVER mark safe the result of a
``strip_tags()`` call without escaping it first, for example with
:func:`django.utils.html.escape`.
CVE-2019-14234: SQL injection possibility in key and index lookups for ``JSONField``/``HStoreField``
====================================================================================================
:lookup:`Key and index lookups <jsonfield.key>` for
:class:`~django.contrib.postgres.fields.JSONField` and :lookup:`key lookups
<hstorefield.key>` for :class:`~django.contrib.postgres.fields.HStoreField`
were subject to SQL injection, using a suitably crafted dictionary, with
dictionary expansion, as the ``**kwargs`` passed to ``QuerySet.filter()``.
CVE-2019-14235: Potential memory exhaustion in ``django.utils.encoding.uri_to_iri()``
=====================================================================================
If passed certain inputs, :func:`django.utils.encoding.uri_to_iri` could lead
to significant memory usage due to excessive recursion when re-percent-encoding
invalid UTF-8 octet sequences.
``uri_to_iri()`` now avoids recursion when re-percent-encoding invalid UTF-8
octet sequences.
Bugfixes
========
* Fixed a regression in Django 2.2 when ordering a ``QuerySet.union()``,
``intersection()``, or ``difference()`` by a field type present more than
once results in the wrong ordering being used (:ticket:`30628`).
* Fixed a migration crash on PostgreSQL when adding a check constraint
with a ``contains`` lookup on
:class:`~django.contrib.postgres.fields.DateRangeField` or
:class:`~django.contrib.postgres.fields.DateTimeRangeField`, if the right
hand side of an expression is the same type (:ticket:`30621`).
* Fixed a regression in Django 2.2 where auto-reloader crashes if a file path
contains nulls characters (``'\x00'``) (:ticket:`30506`).
* Fixed a regression in Django 2.2 where auto-reloader crashes if a translation
directory cannot be resolved (:ticket:`30647`).
==========================
```
### 2.2.3
```
==========================
*July 1, 2019*
Django 2.2.3 fixes a security issue and several bugs in 2.2.2. Also, the latest
string translations from Transifex are incorporated.
CVE-2019-12781: Incorrect HTTP detection with reverse-proxy connecting via HTTPS
--------------------------------------------------------------------------------
When deployed behind a reverse-proxy connecting to Django via HTTPS,
:attr:`django.http.HttpRequest.scheme` would incorrectly detect client
requests made via HTTP as using HTTPS. This entails incorrect results for
:meth:`~django.http.HttpRequest.is_secure`, and
:meth:`~django.http.HttpRequest.build_absolute_uri`, and that HTTP
requests would not be redirected to HTTPS in accordance with
:setting:`SECURE_SSL_REDIRECT`.
``HttpRequest.scheme`` now respects :setting:`SECURE_PROXY_SSL_HEADER`, if it is
configured, and the appropriate header is set on the request, for both HTTP and
HTTPS requests.
If you deploy Django behind a reverse-proxy that forwards HTTP requests, and
that connects to Django via HTTPS, be sure to verify that your application
correctly handles code paths relying on ``scheme``, ``is_secure()``,
``build_absolute_uri()``, and ``SECURE_SSL_REDIRECT``.
Bugfixes
========
* Fixed a regression in Django 2.2 where :class:`~django.db.models.Avg`,
:class:`~django.db.models.StdDev`, and :class:`~django.db.models.Variance`
crash with ``filter`` argument (:ticket:`30542`).
* Fixed a regression in Django 2.2.2 where auto-reloader crashes with
``AttributeError``, e.g. when using ``ipdb`` (:ticket:`30588`).
==========================
```
### 2.2.2
```
==========================
*June 3, 2019*
Django 2.2.2 fixes security issues and several bugs in 2.2.1.
CVE-2019-12308: AdminURLFieldWidget XSS
---------------------------------------
The clickable "Current URL" link generated by ``AdminURLFieldWidget`` displayed
the provided value without validating it as a safe URL. Thus, an unvalidated
value stored in the database, or a value provided as a URL query parameter
payload, could result in an clickable JavaScript link.
``AdminURLFieldWidget`` now validates the provided value using
:class:`~django.core.validators.URLValidator` before displaying the clickable
link. You may customize the validator by passing a ``validator_class`` kwarg to
``AdminURLFieldWidget.__init__()``, e.g. when using
:attr:`~django.contrib.admin.ModelAdmin.formfield_overrides`.
Patched bundled jQuery for CVE-2019-11358: Prototype pollution
--------------------------------------------------------------
jQuery before 3.4.0, mishandles ``jQuery.extend(true, {}, ...)`` because of
``Object.prototype`` pollution. If an unsanitized source object contained an
enumerable ``__proto__`` property, it could extend the native
``Object.prototype``.
The bundled version of jQuery used by the Django admin has been patched to
allow for the ``select2`` library's use of ``jQuery.extend()``.
Bugfixes
========
* Fixed a regression in Django 2.2 that stopped Show/Hide toggles working on
dynamically added admin inlines (:ticket:`30459`).
* Fixed a regression in Django 2.2 where deprecation message crashes if
``Meta.ordering`` contains an expression (:ticket:`30463`).
* Fixed a regression in Django 2.2.1 where
:class:`~django.contrib.postgres.search.SearchVector` generates SQL with a
redundant ``Coalesce`` call (:ticket:`30488`).
* Fixed a regression in Django 2.2 where auto-reloader doesn't detect changes
in ``manage.py`` file when using ``StatReloader`` (:ticket:`30479`).
* Fixed crash of :class:`~django.contrib.postgres.aggregates.ArrayAgg` and
:class:`~django.contrib.postgres.aggregates.StringAgg` with ``ordering``
argument when used in a ``Subquery`` (:ticket:`30315`).
* Fixed a regression in Django 2.2 that caused a crash of auto-reloader when
an exception with custom signature is raised (:ticket:`30516`).
* Fixed a regression in Django 2.2.1 where auto-reloader unnecessarily reloads
translation files multiple times when using ``StatReloader``
(:ticket:`30523`).
==========================
```
### 2.2.1
```
==========================
*May 1, 2019*
Django 2.2.1 fixes several bugs in 2.2.
Bugfixes
========
* Fixed a regression in Django 2.1 that caused the incorrect quoting of
database user password when using :djadmin:`dbshell` on Oracle
(:ticket:`30307`).
* Added compatibility for ``psycopg2`` 2.8 (:ticket:`30331`).
* Fixed a regression in Django 2.2 that caused a crash when loading the
template for the technical 500 debug page (:ticket:`30324`).
* Fixed crash of ``ordering`` argument in
:class:`~django.contrib.postgres.aggregates.ArrayAgg` and
:class:`~django.contrib.postgres.aggregates.StringAgg` when it contains an
expression with params (:ticket:`30332`).
* Fixed a regression in Django 2.2 that caused a single instance fast-delete
to not set the primary key to ``None`` (:ticket:`30330`).
* Prevented :djadmin:`makemigrations` from generating infinite migrations for
check constraints and partial indexes when ``condition`` contains
a :class:`~python:range` object (:ticket:`30350`).
* Reverted an optimization in Django 2.2 (:ticket:`29725`) that caused the
inconsistent behavior of ``count()`` and ``exists()`` on a reverse
many-to-many relationship with a custom manager (:ticket:`30325`).
* Fixed a regression in Django 2.2 where
:class:`~django.core.paginator.Paginator` crashes if ``object_list`` is
a queryset ordered or aggregated over a nested ``JSONField`` key transform
(:ticket:`30335`).
* Fixed a regression in Django 2.2 where ``IntegerField`` validation of
database limits crashes if ``limit_value`` attribute in a custom validator is
callable (:ticket:`30328`).
* Fixed a regression in Django 2.2 where
:class:`~django.contrib.postgres.search.SearchVector` generates SQL that is
not indexable (:ticket:`30385`).
* Fixed a regression in Django 2.2 that caused an exception to be raised when
a custom error handler could not be imported (:ticket:`30318`).
* Relaxed the system check added in Django 2.2 for the admin app's dependencies
to reallow use of
:class:`~django.contrib.sessions.middleware.SessionMiddleware` subclasses,
rather than requiring :mod:`django.contrib.sessions` to be in
:setting:`INSTALLED_APPS` (:ticket:`30312`).
* Increased the default timeout when using ``Watchman`` to 5 seconds to prevent
falling back to ``StatReloader`` on larger projects and made it customizable
via the ``DJANGO_WATCHMAN_TIMEOUT`` environment variable (:ticket:`30361`).
* Fixed a regression in Django 2.2 that caused a crash when migrating
permissions for proxy models if the target permissions already existed. For
example, when a permission had been created manually or a model had been
migrated from concrete to proxy (:ticket:`30351`).
* Fixed a regression in Django 2.2 that caused a crash of :djadmin:`runserver`
when URLConf modules raised exceptions (:ticket:`30323`).
* Fixed a regression in Django 2.2 where changes were not reliably detected by
auto-reloader when using ``StatReloader`` (:ticket:`30323`).
* Fixed a migration crash on Oracle and PostgreSQL when adding a check
constraint with a ``contains``, ``startswith``, or ``endswith`` lookup (or
their case-insensitive variant) (:ticket:`30408`).
* Fixed a migration crash on Oracle and SQLite when adding a check constraint
with ``condition`` contains ``|`` (``OR``) operator (:ticket:`30412`).
========================
```
### 2.2
```
========================
*April 1, 2019*
Welcome to Django 2.2!
These release notes cover the :ref:`new features <whats-new-2.2>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-2.2>` you'll
want to be aware of when upgrading from Django 2.1 or earlier. We've
:ref:`begun the deprecation process for some features
<deprecated-features-2.2>`.
See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.
Django 2.2 is designated as a :term:`long-term support release`. It will
receive security updates for at least three years after its release. Support
for the previous LTS, Django 1.11, will end in April 2020.
Python compatibility
====================
Django 2.2 supports Python 3.5, 3.6, and 3.7. We **highly recommend** and only
officially support the latest release of each series.
.. _whats-new-2.2:
What's new in Django 2.2
========================
Constraints
-----------
The new :class:`~django.db.models.CheckConstraint` and
:class:`~django.db.models.UniqueConstraint` classes enable adding custom
database constraints. Constraints are added to models using the
:attr:`Meta.constraints <django.db.models.Options.constraints>` option.
Minor features
--------------
:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Added a CSS class to the column headers of
:class:`~django.contrib.admin.TabularInline`.
:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~
* The ``HttpRequest`` is now passed as the first positional argument to
:meth:`.RemoteUserBackend.configure_user`, if it accepts it.
:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~
* Added Oracle support for the
:class:`~django.contrib.gis.db.models.functions.Envelope` function.
* Added SpatiaLite support for the :lookup:`coveredby` and :lookup:`covers`
lookups.
:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* The new ``ordering`` argument for
:class:`~django.contrib.postgres.aggregates.ArrayAgg` and
:class:`~django.contrib.postgres.aggregates.StringAgg` determines the
ordering of the aggregated elements.
* The new :class:`~django.contrib.postgres.indexes.BTreeIndex`,
:class:`~django.contrib.postgres.indexes.HashIndex` and
:class:`~django.contrib.postgres.indexes.SpGistIndex` classes allow
creating ``B-Tree``, ``hash``, and ``SP-GiST`` indexes in the database.
* :class:`~django.contrib.postgres.indexes.BrinIndex` now has the
``autosummarize`` parameter.
* The new ``search_type`` parameter of
:class:`~django.contrib.postgres.search.SearchQuery` allows searching for
a phrase or raw expression.
:mod:`django.contrib.staticfiles`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Added path matching to the :option:`collectstatic --ignore` option so that
patterns like ``/vendor/*.js`` can be used.
Database backends
~~~~~~~~~~~~~~~~~
* Added result streaming for :meth:`.QuerySet.iterator` on SQLite.
Generic Views
~~~~~~~~~~~~~
* The new :meth:`View.setup <django.views.generic.base.View.setup>` hook
initializes view attributes before calling
:meth:`~django.views.generic.base.View.dispatch`. It allows mixins to setup
instance attributes for reuse in child classes.
Internationalization
~~~~~~~~~~~~~~~~~~~~
* Added support and translations for the Armenian language.
Management Commands
~~~~~~~~~~~~~~~~~~~
* The new :option:`--force-color` option forces colorization of the command
output.
* :djadmin:`inspectdb` now creates models for foreign tables on PostgreSQL.
* :option:`inspectdb --include-views` now creates models for materialized views
on Oracle and PostgreSQL.
* The new :option:`inspectdb --include-partitions` option allows creating
models for partition tables on PostgreSQL. In older versions, models are
created child tables instead the parent.
* :djadmin:`inspectdb` now introspects :class:`~django.db.models.DurationField`
for Oracle and PostgreSQL, and :class:`~django.db.models.AutoField` for
SQLite.
* On Oracle, :djadmin:`dbshell` is wrapped with ``rlwrap``, if available.
``rlwrap`` provides a command history and editing of keyboard input.
* The new :option:`makemigrations --no-header` option avoids writing header
comments in generated migration file(s). This option is also available for
:djadmin:`squashmigrations`.
* :djadmin:`runserver` can now use `Watchman
<https://facebook.github.io/watchman/>`_ to improve the performance of
watching a large number of files for changes.
Migrations
~~~~~~~~~~
* The new :option:`migrate --plan` option prints the list of migration
operations that will be performed.
* ``NoneType`` can now be serialized in migrations.
* You can now :ref:`register custom serializers <custom-migration-serializers>`
for migrations.
Models
~~~~~~
* Added support for PostgreSQL operator classes (:attr:`.Index.opclasses`).
* Added support for partial indexes (:attr:`.Index.condition`).
* Added the :class:`~django.db.models.functions.NullIf` and
:class:`~django.db.models.functions.Reverse` database functions, as well as
many :ref:`math database functions <math-functions>`.
* Setting the new ``ignore_conflicts`` parameter of
:meth:`.QuerySet.bulk_create` to ``True`` tells the database to ignore
failure to insert rows that fail uniqueness constraints or other checks.
* The new :class:`~django.db.models.functions.ExtractIsoYear` function extracts
ISO-8601 week-numbering years from :class:`~django.db.models.DateField` and
:class:`~django.db.models.DateTimeField`, and the new :lookup:`iso_year`
lookup allows querying by an ISO-8601 week-numbering year.
* The new :meth:`.QuerySet.bulk_update` method allows efficiently updating
specific fields on multiple model instances.
* Django no longer always starts a transaction when a single query is being
performed, such as ``Model.save()``, ``QuerySet.update()``, and
``Model.delete()``. This improves the performance of autocommit by reducing
the number of database round trips.
* Added SQLite support for the :class:`~django.db.models.StdDev` and
:class:`~django.db.models.Variance` functions.
* The handling of ``DISTINCT`` aggregation is added to the
:class:`~django.db.models.Aggregate` class. Adding :attr:`allow_distinct =
True <django.db.models.Aggregate.allow_distinct>` as a class attribute on
``Aggregate`` subclasses allows a ``distinct`` keyword argument to be
specified on initialization to ensure that the aggregate function is only
called for each distinct value of ``expressions``.
* The :meth:`.RelatedManager.add`, :meth:`~.RelatedManager.create`,
:meth:`~.RelatedManager.remove`, :meth:`~.RelatedManager.set`,
``get_or_create()``, and ``update_or_create()`` methods are now allowed on
many-to-many relationships with intermediate models. The new
``through_defaults`` argument is used to specify values for new intermediate
model instance(s).
Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~
* Added :attr:`.HttpRequest.headers` to allow simple access to a request's
headers.
Serialization
~~~~~~~~~~~~~
* You can now deserialize data using natural keys containing :ref:`forward
references <natural-keys-and-forward-references>` by passing
``handle_forward_references=True`` to ``serializers.deserialize()``.
Additionally, :djadmin:`loaddata` handles forward references automatically.
Tests
~~~~~
* The new :meth:`.SimpleTestCase.assertURLEqual` assertion checks for a given
URL, ignoring the ordering of the query string.
:meth:`~.SimpleTestCase.assertRedirects` uses the new assertion.
* The test :class:`~.django.test.Client` now supports automatic JSON
serialization of list and tuple ``data`` when
``content_type='application/json'``.
* The new :setting:`ORACLE_MANAGED_FILES <TEST_ORACLE_MANAGED_FILES>` test
database setting allows using Oracle Managed Files (OMF) tablespaces.
* Deferrable database constraints are now checked at the end of each
:class:`~django.test.TestCase` test on SQLite 3.20+, just like on other
backends that support deferrable constraints. These checks aren't implemented
for older versions of SQLite because they would require expensive table
introspection there.
* :class:`~django.test.runner.DiscoverRunner` now skips the setup of databases
not :ref:`referenced by tests<testing-multi-db>`.
URLs
~~~~
* The new :attr:`.ResolverMatch.route` attribute stores the route of the
matching URL pattern.
Validators
~~~~~~~~~~
* :class:`.MaxValueValidator`, :class:`.MinValueValidator`,
:class:`.MinLengthValidator`, and :class:`.MaxLengthValidator` now accept
a callable ``limit_value``.
.. _backwards-incompatible-2.2:
Backwards incompatible changes in 2.2
=====================================
Database backend API
--------------------
This section describes changes that may be needed in third-party database
backends.
* Third-party database backends must implement support for table check
constraints or set ``DatabaseFeatures.supports_table_check_constraints`` to
``False``.
* Third party database backends must implement support for ignoring
constraints or uniqueness errors while inserting or set
``DatabaseFeatures.supports_ignore_conflicts`` to ``False``.
* Third party database backends must implement introspection for
``DurationField`` or set ``DatabaseFeatures.can_introspect_duration_field``
to ``False``.
* ``DatabaseFeatures.uses_savepoints`` now defaults to ``True``.
* Third party database backends must implement support for partial indexes or
set ``DatabaseFeatures.supports_partial_indexes`` to ``False``.
* ``DatabaseIntrospection.table_name_converter()`` and
``column_name_converter()`` are removed. Third party database backends may
need to instead implement ``DatabaseIntrospection.identifier_converter()``.
In that case, the constraint names that
``DatabaseIntrospection.get_constraints()`` returns must be normalized by
``identifier_converter()``.
* SQL generation for indexes is moved from :class:`~django.db.models.Index` to
``SchemaEditor`` and these ``SchemaEditor`` methods are added:
* ``_create_primary_key_sql()`` and ``_delete_primary_key_sql()``
* ``_delete_index_sql()`` (to pair with ``_create_index_sql()``)
* ``_delete_unique_sql`` (to pair with ``_create_unique_sql()``)
* ``_delete_fk_sql()`` (to pair with ``_create_fk_sql()``)
* ``_create_check_sql()`` and ``_delete_check_sql()``
* The third argument of ``DatabaseWrapper.__init__()``,
``allow_thread_sharing``, is removed.
Admin actions are no longer collected from base ``ModelAdmin`` classes
----------------------------------------------------------------------
For example, in older versions of Django::
from django.contrib import admin
class BaseAdmin(admin.ModelAdmin):
actions = ['a']
class SubAdmin(BaseAdmin):
actions = ['b']
``SubAdmin`` would have actions ``'a'`` and ``'b'``.
Now ``actions`` follows standard Python inheritance. To get the same result as
before::
class SubAdmin(BaseAdmin):
actions = BaseAdmin.actions + ['b']
:mod:`django.contrib.gis`
-------------------------
* Support for GDAL 1.9 and 1.10 is dropped.
``TransactionTestCase`` serialized data loading
-----------------------------------------------
Initial data migrations are now loaded in
:class:`~django.test.TransactionTestCase` at the end of the test, after the
database flush. In older versions, this data was loaded at the beginning of the
test, but this prevents the :option:`test --keepdb` option from working
properly (the database was empty at the end of the whole test suite). This
change shouldn't have an impact on your tests unless you've customized
:class:`~django.test.TransactionTestCase`'s internals.
``sqlparse`` is required dependency
-----------------------------------
To simplify a few parts of Django's database handling, `sqlparse
<https://pypi.org/project/sqlparse/>`_ is now a required dependency. It's
automatically installed along with Django.
``cached_property`` aliases
---------------------------
In usage like::
from django.utils.functional import cached_property
class A:
cached_property
def base(self):
return ...
alias = base
``alias`` is not cached. Where the problem can be detected (Python 3.6 and
later), such usage now raises ``TypeError: Cannot assign the same
cached_property to two different names ('base' and 'alias').``
Use this instead::
import operator
class A:
...
alias = property(operator.attrgetter('base'))
Permissions for proxy models
----------------------------
:ref:`Permissions for proxy models <proxy-models-permissions-topic>` are now
created using the content type of the proxy model rather than the content type
of the concrete model. A migration will update existing permissions when you
run :djadmin:`migrate`.
In the admin, the change is transparent for proxy models having the same
``app_label`` as their concrete model. However, in older versions, users with
permissions for a proxy model with a *different* ``app_label`` than its
concrete model couldn't access the model in the admin. That's now fixed, but
you might want to audit the permissions assignments for such proxy models
(``[add|view|change|delete]_myproxy``) prior to upgrading to ensure the new
access is appropriate.
Finally, proxy model permission strings must be updated to use their own
``app_label``. For example, for ``app.MyProxyModel`` inheriting from
``other_app.ConcreteModel``, update
``user.has_perm('other_app.add_myproxymodel')`` to
``user.has_perm('app.add_myproxymodel')``.
Merging of form ``Media`` assets
--------------------------------
Form ``Media`` assets are now merged using a topological sort algorithm, as the
old pairwise merging algorithm is insufficient for some cases. CSS and
JavaScript files which don't include their dependencies may now be sorted
incorrectly (where the old algorithm produced results correctly by
coincidence).
Audit all ``Media`` classes for any missing dependencies. For example,
widgets depending on ``django.jQuery`` must specify
``js=['admin/js/jquery.init.js', ...]`` when :ref:`declaring form media assets
<assets-as-a-static-definition>`.
Miscellaneous
-------------
* To improve readability, the ``UUIDField`` form field now displays values with
dashes, e.g. ``550e8400-e29b-41d4-a716-446655440000`` instead of
``550e8400e29b41d4a716446655440000``.
* On SQLite, ``PositiveIntegerField`` and ``PositiveSmallIntegerField`` now
include a check constraint to prevent negative values in the database. If you
have existing invalid data and run a migration that recreates a table, you'll
see ``CHECK constraint failed``.
* For consistency with WSGI servers, the test client now sets the
``Content-Length`` header to a string rather than an integer.
* The return value of :func:`django.utils.text.slugify` is no longer marked as
HTML safe.
* The default truncation character used by the :tfilter:`urlizetrunc`,
:tfilter:`truncatechars`, :tfilter:`truncatechars_html`,
:tfilter:`truncatewords`, and :tfilter:`truncatewords_html` template filters
is now the real ellipsis character (``…``) instead of 3 dots. You may have to
adapt some test output comparisons.
* Support for bytestring paths in the template filesystem loader is removed.
* :func:`django.utils.http.urlsafe_base64_encode` now returns a string instead
of a bytestring, and :func:`django.utils.http.urlsafe_base64_decode` may no
longer be passed a bytestring.
* Support for ``cx_Oracle`` < 6.0 is removed.
* The minimum supported version of ``mysqlclient`` is increased from 1.3.7 to
1.3.13.
* The minimum supported version of SQLite is increased from 3.7.15 to 3.8.3.
* In an attempt to provide more semantic query data, ``NullBooleanSelect`` now
renders ``<option>`` values of ``unknown``, ``true``, and ``false`` instead
of ``1``, ``2``, and ``3``. For backwards compatibility, the old values are
still accepted as data.
* :attr:`Group.name <django.contrib.auth.models.Group.name>` ``max_length``
is increased from 80 to 150 characters.
* Tests that violate deferrable database constraints now error when run on
SQLite 3.20+, just like on other backends that support such constraints.
* To catch usage mistakes, the test :class:`~django.test.Client` and
:func:`django.utils.http.urlencode` now raise ``TypeError`` if ``None`` is
passed as a value to encode because ``None`` can't be encoded in GET and POST
data. Either pass an empty string or omit the value.
* The :djadmin:`ping_google` management command now defaults to ``https``
instead of ``http`` for the sitemap's URL. If your site uses http, use the
new :option:`ping_google --sitemap-uses-http` option. If you use the
:func:`~django.contrib.sitemaps.ping_google` function, set the new
``sitemap_uses_https`` argument to ``False``.
* :djadmin:`runserver` no longer supports `pyinotify` (replaced by Watchman).
* The :class:`~django.db.models.Avg`, :class:`~django.db.models.StdDev`, and
:class:`~django.db.models.Variance` aggregate functions now return a
``Decimal`` instead of a ``float`` when the input is ``Decimal``.
* Tests will fail on SQLite if apps without migrations have relations to apps
with migrations. This has been a documented restriction since migrations were
added in Django 1.7, but it fails more reliably now. You'll see tests failing
with errors like ``no such table: <app_label>_<model>``. This was observed
with several third-party apps that had models in tests without migrations.
You must add migrations for such models.
.. _deprecated-features-2.2:
Features deprecated in 2.2
==========================
Model ``Meta.ordering`` will no longer affect ``GROUP BY`` queries
------------------------------------------------------------------
A model's ``Meta.ordering`` affecting ``GROUP BY`` queries (such as
``.annotate().values()``) is a common source of confusion. Such queries now
issue a deprecation warning with the advice to add an ``order_by()`` to retain
the current query. ``Meta.ordering`` will be ignored in such queries starting
in Django 3.1.
Miscellaneous
-------------
* ``django.utils.timezone.FixedOffset`` is deprecated in favor of
:class:`datetime.timezone`.
* The undocumented ``QuerySetPaginator`` alias of
``django.core.paginator.Paginator`` is deprecated.
* The ``FloatRangeField`` model and form fields in ``django.contrib.postgres``
are deprecated in favor of a new name, ``DecimalRangeField``, to match the
underlying ``numrange`` data type used in the database.
* The ``FILE_CHARSET`` setting is deprecated. Starting with Django 3.1, files
read from disk must be UTF-8 encoded.
* ``django.contrib.staticfiles.storage.CachedStaticFilesStorage`` is
deprecated due to the intractable problems that it has. Use
:class:`.ManifestStaticFilesStorage` or a third-party cloud storage instead.
* :meth:`.RemoteUserBackend.configure_user` is now passed ``request`` as the
first positional argument, if it accepts it. Support for overrides that don't
accept it will be removed in Django 3.1.
* The :attr:`.SimpleTestCase.allow_database_queries`,
:attr:`.TransactionTestCase.multi_db`, and :attr:`.TestCase.multi_db`
attributes are deprecated in favor of :attr:`.SimpleTestCase.databases`,
:attr:`.TransactionTestCase.databases`, and :attr:`.TestCase.databases`.
These new attributes allow databases dependencies to be declared in order to
prevent unexpected queries against non-default databases to leak state
between tests. The previous behavior of ``allow_database_queries=True`` and
``multi_db=True`` can be achieved by setting ``databases='__all__'``.
===========================
```
### 2.1.11
```
===========================
*August 1, 2019*
Django 2.1.11 fixes security issues in 2.1.10.
CVE-2019-14232: Denial-of-service possibility in ``django.utils.text.Truncator``
================================================================================
If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods
were passed the ``html=True`` argument, they were extremely slow to evaluate
certain inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``chars()`` and ``words()`` methods are used to implement the
:tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template
filters, which were thus vulnerable.
The regular expressions used by ``Truncator`` have been simplified in order to
avoid potential backtracking issues. As a consequence, trailing punctuation may
now at times be included in the truncated output.
CVE-2019-14233: Denial-of-service possibility in ``strip_tags()``
=================================================================
Due to the behavior of the underlying ``HTMLParser``,
:func:`django.utils.html.strip_tags` would be extremely slow to evaluate
certain inputs containing large sequences of nested incomplete HTML entities.
The ``strip_tags()`` method is used to implement the corresponding
:tfilter:`striptags` template filter, which was thus also vulnerable.
``strip_tags()`` now avoids recursive calls to ``HTMLParser`` when progress
removing tags, but necessarily incomplete HTML entities, stops being made.
Remember that absolutely NO guarantee is provided about the results of
``strip_tags()`` being HTML safe. So NEVER mark safe the result of a
``strip_tags()`` call without escaping it first, for example with
:func:`django.utils.html.escape`.
CVE-2019-14234: SQL injection possibility in key and index lookups for ``JSONField``/``HStoreField``
====================================================================================================
:lookup:`Key and index lookups <jsonfield.key>` for
:class:`~django.contrib.postgres.fields.JSONField` and :lookup:`key lookups
<hstorefield.key>` for :class:`~django.contrib.postgres.fields.HStoreField`
were subject to SQL injection, using a suitably crafted dictionary, with
dictionary expansion, as the ``**kwargs`` passed to ``QuerySet.filter()``.
CVE-2019-14235: Potential memory exhaustion in ``django.utils.encoding.uri_to_iri()``
=====================================================================================
If passed certain inputs, :func:`django.utils.encoding.uri_to_iri` could lead
to significant memory usage due to excessive recursion when re-percent-encoding
invalid UTF-8 octet sequences.
``uri_to_iri()`` now avoids recursion when re-percent-encoding invalid UTF-8
octet sequences.
===========================
```
### 2.1.10
```
===========================
*July 1, 2019*
Django 2.1.10 fixes a security issue in 2.1.9.
CVE-2019-12781: Incorrect HTTP detection with reverse-proxy connecting via HTTPS
--------------------------------------------------------------------------------
When deployed behind a reverse-proxy connecting to Django via HTTPS,
:attr:`django.http.HttpRequest.scheme` would incorrectly detect client
requests made via HTTP as using HTTPS. This entails incorrect results for
:meth:`~django.http.HttpRequest.is_secure`, and
:meth:`~django.http.HttpRequest.build_absolute_uri`, and that HTTP
requests would not be redirected to HTTPS in accordance with
:setting:`SECURE_SSL_REDIRECT`.
``HttpRequest.scheme`` now respects :setting:`SECURE_PROXY_SSL_HEADER`, if it
is configured, and the appropriate header is set on the request, for both HTTP
and HTTPS requests.
If you deploy Django behind a reverse-proxy that forwards HTTP requests, and
that connects to Django via HTTPS, be sure to verify that your application
correctly handles code paths relying on ``scheme``, ``is_secure()``,
``build_absolute_uri()``, and ``SECURE_SSL_REDIRECT``.
============================
```
### 2.1.9
```
============================
*June 3, 2019*
Django 2.1.9 fixes security issues in 2.1.8.
CVE-2019-12308: AdminURLFieldWidget XSS
---------------------------------------
The clickable "Current URL" link generated by ``AdminURLFieldWidget`` displayed
the provided value without validating it as a safe URL. Thus, an unvalidated
value stored in the database, or a value provided as a URL query parameter
payload, could result in an clickable JavaScript link.
``AdminURLFieldWidget`` now validates the provided value using
:class:`~django.core.validators.URLValidator` before displaying the clickable
link. You may customize the validator by passing a ``validator_class`` kwarg to
``AdminURLFieldWidget.__init__()``, e.g. when using
:attr:`~django.contrib.admin.ModelAdmin.formfield_overrides`.
Patched bundled jQuery for CVE-2019-11358: Prototype pollution
--------------------------------------------------------------
jQuery before 3.4.0, mishandles ``jQuery.extend(true, {}, ...)`` because of
``Object.prototype`` pollution. If an unsanitized source object contained an
enumerable ``__proto__`` property, it could extend the native
``Object.prototype``.
The bundled version of jQuery used by the Django admin has been patched to
allow for the ``select2`` library's use of ``jQuery.extend()``.
==========================
```
### 2.1.8
```
==========================
*April 1, 2019*
Django 2.1.8 fixes a bug in 2.1.7.
Bugfixes
========
* Prevented admin inlines for a ``ManyToManyField``\'s implicit through model
from being editable if the user only has the view permission
(:ticket:`30289`).
==========================
```
Links
- PyPI: https://pypi.org/project/django
- Changelog: https://pyup.io/changelogs/django/
- Homepage: https://www.djangoproject.com/
Fastjur
commented
5 years ago
This PR updates Django from 2.1.7 to 2.2.4.
Changelog
### 2.2.4 ``` ========================== *August 1, 2019* Django 2.2.4 fixes security issues and several bugs in 2.2.3. CVE-2019-14232: Denial-of-service possibility in ``django.utils.text.Truncator`` ================================================================================ If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods were passed the ``html=True`` argument, they were extremely slow to evaluate certain inputs due to a catastrophic backtracking vulnerability in a regular expression. The ``chars()`` and ``words()`` methods are used to implement the :tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template filters, which were thus vulnerable. The regular expressions used by ``Truncator`` have been simplified in order to avoid potential backtracking issues. As a consequence, trailing punctuation may now at times be included in the truncated output. CVE-2019-14233: Denial-of-service possibility in ``strip_tags()`` ================================================================= Due to the behavior of the underlying ``HTMLParser``, :func:`django.utils.html.strip_tags` would be extremely slow to evaluate certain inputs containing large sequences of nested incomplete HTML entities. The ``strip_tags()`` method is used to implement the corresponding :tfilter:`striptags` template filter, which was thus also vulnerable. ``strip_tags()`` now avoids recursive calls to ``HTMLParser`` when progress removing tags, but necessarily incomplete HTML entities, stops being made. Remember that absolutely NO guarantee is provided about the results of ``strip_tags()`` being HTML safe. So NEVER mark safe the result of a ``strip_tags()`` call without escaping it first, for example with :func:`django.utils.html.escape`. CVE-2019-14234: SQL injection possibility in key and index lookups for ``JSONField``/``HStoreField`` ==================================================================================================== :lookup:`Key and index lookups <jsonfield.key>` for :class:`~django.contrib.postgres.fields.JSONField` and :lookup:`key lookups <hstorefield.key>` for :class:`~django.contrib.postgres.fields.HStoreField` were subject to SQL injection, using a suitably crafted dictionary, with dictionary expansion, as the ``**kwargs`` passed to ``QuerySet.filter()``. CVE-2019-14235: Potential memory exhaustion in ``django.utils.encoding.uri_to_iri()`` ===================================================================================== If passed certain inputs, :func:`django.utils.encoding.uri_to_iri` could lead to significant memory usage due to excessive recursion when re-percent-encoding invalid UTF-8 octet sequences. ``uri_to_iri()`` now avoids recursion when re-percent-encoding invalid UTF-8 octet sequences. Bugfixes ======== * Fixed a regression in Django 2.2 when ordering a ``QuerySet.union()``, ``intersection()``, or ``difference()`` by a field type present more than once results in the wrong ordering being used (:ticket:`30628`). * Fixed a migration crash on PostgreSQL when adding a check constraint with a ``contains`` lookup on :class:`~django.contrib.postgres.fields.DateRangeField` or :class:`~django.contrib.postgres.fields.DateTimeRangeField`, if the right hand side of an expression is the same type (:ticket:`30621`). * Fixed a regression in Django 2.2 where auto-reloader crashes if a file path contains nulls characters (``'\x00'``) (:ticket:`30506`). * Fixed a regression in Django 2.2 where auto-reloader crashes if a translation directory cannot be resolved (:ticket:`30647`). ========================== ``` ### 2.2.3 ``` ========================== *July 1, 2019* Django 2.2.3 fixes a security issue and several bugs in 2.2.2. Also, the latest string translations from Transifex are incorporated. CVE-2019-12781: Incorrect HTTP detection with reverse-proxy connecting via HTTPS -------------------------------------------------------------------------------- When deployed behind a reverse-proxy connecting to Django via HTTPS, :attr:`django.http.HttpRequest.scheme` would incorrectly detect client requests made via HTTP as using HTTPS. This entails incorrect results for :meth:`~django.http.HttpRequest.is_secure`, and :meth:`~django.http.HttpRequest.build_absolute_uri`, and that HTTP requests would not be redirected to HTTPS in accordance with :setting:`SECURE_SSL_REDIRECT`. ``HttpRequest.scheme`` now respects :setting:`SECURE_PROXY_SSL_HEADER`, if it is configured, and the appropriate header is set on the request, for both HTTP and HTTPS requests. If you deploy Django behind a reverse-proxy that forwards HTTP requests, and that connects to Django via HTTPS, be sure to verify that your application correctly handles code paths relying on ``scheme``, ``is_secure()``, ``build_absolute_uri()``, and ``SECURE_SSL_REDIRECT``. Bugfixes ======== * Fixed a regression in Django 2.2 where :class:`~django.db.models.Avg`, :class:`~django.db.models.StdDev`, and :class:`~django.db.models.Variance` crash with ``filter`` argument (:ticket:`30542`). * Fixed a regression in Django 2.2.2 where auto-reloader crashes with ``AttributeError``, e.g. when using ``ipdb`` (:ticket:`30588`). ========================== ``` ### 2.2.2 ``` ========================== *June 3, 2019* Django 2.2.2 fixes security issues and several bugs in 2.2.1. CVE-2019-12308: AdminURLFieldWidget XSS --------------------------------------- The clickable "Current URL" link generated by ``AdminURLFieldWidget`` displayed the provided value without validating it as a safe URL. Thus, an unvalidated value stored in the database, or a value provided as a URL query parameter payload, could result in an clickable JavaScript link. ``AdminURLFieldWidget`` now validates the provided value using :class:`~django.core.validators.URLValidator` before displaying the clickable link. You may customize the validator by passing a ``validator_class`` kwarg to ``AdminURLFieldWidget.__init__()``, e.g. when using :attr:`~django.contrib.admin.ModelAdmin.formfield_overrides`. Patched bundled jQuery for CVE-2019-11358: Prototype pollution -------------------------------------------------------------- jQuery before 3.4.0, mishandles ``jQuery.extend(true, {}, ...)`` because of ``Object.prototype`` pollution. If an unsanitized source object contained an enumerable ``__proto__`` property, it could extend the native ``Object.prototype``. The bundled version of jQuery used by the Django admin has been patched to allow for the ``select2`` library's use of ``jQuery.extend()``. Bugfixes ======== * Fixed a regression in Django 2.2 that stopped Show/Hide toggles working on dynamically added admin inlines (:ticket:`30459`). * Fixed a regression in Django 2.2 where deprecation message crashes if ``Meta.ordering`` contains an expression (:ticket:`30463`). * Fixed a regression in Django 2.2.1 where :class:`~django.contrib.postgres.search.SearchVector` generates SQL with a redundant ``Coalesce`` call (:ticket:`30488`). * Fixed a regression in Django 2.2 where auto-reloader doesn't detect changes in ``manage.py`` file when using ``StatReloader`` (:ticket:`30479`). * Fixed crash of :class:`~django.contrib.postgres.aggregates.ArrayAgg` and :class:`~django.contrib.postgres.aggregates.StringAgg` with ``ordering`` argument when used in a ``Subquery`` (:ticket:`30315`). * Fixed a regression in Django 2.2 that caused a crash of auto-reloader when an exception with custom signature is raised (:ticket:`30516`). * Fixed a regression in Django 2.2.1 where auto-reloader unnecessarily reloads translation files multiple times when using ``StatReloader`` (:ticket:`30523`). ========================== ``` ### 2.2.1 ``` ========================== *May 1, 2019* Django 2.2.1 fixes several bugs in 2.2. Bugfixes ======== * Fixed a regression in Django 2.1 that caused the incorrect quoting of database user password when using :djadmin:`dbshell` on Oracle (:ticket:`30307`). * Added compatibility for ``psycopg2`` 2.8 (:ticket:`30331`). * Fixed a regression in Django 2.2 that caused a crash when loading the template for the technical 500 debug page (:ticket:`30324`). * Fixed crash of ``ordering`` argument in :class:`~django.contrib.postgres.aggregates.ArrayAgg` and :class:`~django.contrib.postgres.aggregates.StringAgg` when it contains an expression with params (:ticket:`30332`). * Fixed a regression in Django 2.2 that caused a single instance fast-delete to not set the primary key to ``None`` (:ticket:`30330`). * Prevented :djadmin:`makemigrations` from generating infinite migrations for check constraints and partial indexes when ``condition`` contains a :class:`~python:range` object (:ticket:`30350`). * Reverted an optimization in Django 2.2 (:ticket:`29725`) that caused the inconsistent behavior of ``count()`` and ``exists()`` on a reverse many-to-many relationship with a custom manager (:ticket:`30325`). * Fixed a regression in Django 2.2 where :class:`~django.core.paginator.Paginator` crashes if ``object_list`` is a queryset ordered or aggregated over a nested ``JSONField`` key transform (:ticket:`30335`). * Fixed a regression in Django 2.2 where ``IntegerField`` validation of database limits crashes if ``limit_value`` attribute in a custom validator is callable (:ticket:`30328`). * Fixed a regression in Django 2.2 where :class:`~django.contrib.postgres.search.SearchVector` generates SQL that is not indexable (:ticket:`30385`). * Fixed a regression in Django 2.2 that caused an exception to be raised when a custom error handler could not be imported (:ticket:`30318`). * Relaxed the system check added in Django 2.2 for the admin app's dependencies to reallow use of :class:`~django.contrib.sessions.middleware.SessionMiddleware` subclasses, rather than requiring :mod:`django.contrib.sessions` to be in :setting:`INSTALLED_APPS` (:ticket:`30312`). * Increased the default timeout when using ``Watchman`` to 5 seconds to prevent falling back to ``StatReloader`` on larger projects and made it customizable via the ``DJANGO_WATCHMAN_TIMEOUT`` environment variable (:ticket:`30361`). * Fixed a regression in Django 2.2 that caused a crash when migrating permissions for proxy models if the target permissions already existed. For example, when a permission had been created manually or a model had been migrated from concrete to proxy (:ticket:`30351`). * Fixed a regression in Django 2.2 that caused a crash of :djadmin:`runserver` when URLConf modules raised exceptions (:ticket:`30323`). * Fixed a regression in Django 2.2 where changes were not reliably detected by auto-reloader when using ``StatReloader`` (:ticket:`30323`). * Fixed a migration crash on Oracle and PostgreSQL when adding a check constraint with a ``contains``, ``startswith``, or ``endswith`` lookup (or their case-insensitive variant) (:ticket:`30408`). * Fixed a migration crash on Oracle and SQLite when adding a check constraint with ``condition`` contains ``|`` (``OR``) operator (:ticket:`30412`). ======================== ``` ### 2.2 ``` ======================== *April 1, 2019* Welcome to Django 2.2! These release notes cover the :ref:`new features <whats-new-2.2>`, as well as some :ref:`backwards incompatible changes <backwards-incompatible-2.2>` you'll want to be aware of when upgrading from Django 2.1 or earlier. We've :ref:`begun the deprecation process for some features <deprecated-features-2.2>`. See the :doc:`/howto/upgrade-version` guide if you're updating an existing project. Django 2.2 is designated as a :term:`long-term support release`. It will receive security updates for at least three years after its release. Support for the previous LTS, Django 1.11, will end in April 2020. Python compatibility ==================== Django 2.2 supports Python 3.5, 3.6, and 3.7. We **highly recommend** and only officially support the latest release of each series. .. _whats-new-2.2: What's new in Django 2.2 ======================== Constraints ----------- The new :class:`~django.db.models.CheckConstraint` and :class:`~django.db.models.UniqueConstraint` classes enable adding custom database constraints. Constraints are added to models using the :attr:`Meta.constraints <django.db.models.Options.constraints>` option. Minor features -------------- :mod:`django.contrib.admin` ~~~~~~~~~~~~~~~~~~~~~~~~~~~ * Added a CSS class to the column headers of :class:`~django.contrib.admin.TabularInline`. :mod:`django.contrib.auth` ~~~~~~~~~~~~~~~~~~~~~~~~~~ * The ``HttpRequest`` is now passed as the first positional argument to :meth:`.RemoteUserBackend.configure_user`, if it accepts it. :mod:`django.contrib.gis` ~~~~~~~~~~~~~~~~~~~~~~~~~ * Added Oracle support for the :class:`~django.contrib.gis.db.models.functions.Envelope` function. * Added SpatiaLite support for the :lookup:`coveredby` and :lookup:`covers` lookups. :mod:`django.contrib.postgres` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * The new ``ordering`` argument for :class:`~django.contrib.postgres.aggregates.ArrayAgg` and :class:`~django.contrib.postgres.aggregates.StringAgg` determines the ordering of the aggregated elements. * The new :class:`~django.contrib.postgres.indexes.BTreeIndex`, :class:`~django.contrib.postgres.indexes.HashIndex` and :class:`~django.contrib.postgres.indexes.SpGistIndex` classes allow creating ``B-Tree``, ``hash``, and ``SP-GiST`` indexes in the database. * :class:`~django.contrib.postgres.indexes.BrinIndex` now has the ``autosummarize`` parameter. * The new ``search_type`` parameter of :class:`~django.contrib.postgres.search.SearchQuery` allows searching for a phrase or raw expression. :mod:`django.contrib.staticfiles` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * Added path matching to the :option:`collectstatic --ignore` option so that patterns like ``/vendor/*.js`` can be used. Database backends ~~~~~~~~~~~~~~~~~ * Added result streaming for :meth:`.QuerySet.iterator` on SQLite. Generic Views ~~~~~~~~~~~~~ * The new :meth:`View.setup <django.views.generic.base.View.setup>` hook initializes view attributes before calling :meth:`~django.views.generic.base.View.dispatch`. It allows mixins to setup instance attributes for reuse in child classes. Internationalization ~~~~~~~~~~~~~~~~~~~~ * Added support and translations for the Armenian language. Management Commands ~~~~~~~~~~~~~~~~~~~ * The new :option:`--force-color` option forces colorization of the command output. * :djadmin:`inspectdb` now creates models for foreign tables on PostgreSQL. * :option:`inspectdb --include-views` now creates models for materialized views on Oracle and PostgreSQL. * The new :option:`inspectdb --include-partitions` option allows creating models for partition tables on PostgreSQL. In older versions, models are created child tables instead the parent. * :djadmin:`inspectdb` now introspects :class:`~django.db.models.DurationField` for Oracle and PostgreSQL, and :class:`~django.db.models.AutoField` for SQLite. * On Oracle, :djadmin:`dbshell` is wrapped with ``rlwrap``, if available. ``rlwrap`` provides a command history and editing of keyboard input. * The new :option:`makemigrations --no-header` option avoids writing header comments in generated migration file(s). This option is also available for :djadmin:`squashmigrations`. * :djadmin:`runserver` can now use `Watchman <https://facebook.github.io/watchman/>`_ to improve the performance of watching a large number of files for changes. Migrations ~~~~~~~~~~ * The new :option:`migrate --plan` option prints the list of migration operations that will be performed. * ``NoneType`` can now be serialized in migrations. * You can now :ref:`register custom serializers <custom-migration-serializers>` for migrations. Models ~~~~~~ * Added support for PostgreSQL operator classes (:attr:`.Index.opclasses`). * Added support for partial indexes (:attr:`.Index.condition`). * Added the :class:`~django.db.models.functions.NullIf` and :class:`~django.db.models.functions.Reverse` database functions, as well as many :ref:`math database functions <math-functions>`. * Setting the new ``ignore_conflicts`` parameter of :meth:`.QuerySet.bulk_create` to ``True`` tells the database to ignore failure to insert rows that fail uniqueness constraints or other checks. * The new :class:`~django.db.models.functions.ExtractIsoYear` function extracts ISO-8601 week-numbering years from :class:`~django.db.models.DateField` and :class:`~django.db.models.DateTimeField`, and the new :lookup:`iso_year` lookup allows querying by an ISO-8601 week-numbering year. * The new :meth:`.QuerySet.bulk_update` method allows efficiently updating specific fields on multiple model instances. * Django no longer always starts a transaction when a single query is being performed, such as ``Model.save()``, ``QuerySet.update()``, and ``Model.delete()``. This improves the performance of autocommit by reducing the number of database round trips. * Added SQLite support for the :class:`~django.db.models.StdDev` and :class:`~django.db.models.Variance` functions. * The handling of ``DISTINCT`` aggregation is added to the :class:`~django.db.models.Aggregate` class. Adding :attr:`allow_distinct = True <django.db.models.Aggregate.allow_distinct>` as a class attribute on ``Aggregate`` subclasses allows a ``distinct`` keyword argument to be specified on initialization to ensure that the aggregate function is only called for each distinct value of ``expressions``. * The :meth:`.RelatedManager.add`, :meth:`~.RelatedManager.create`, :meth:`~.RelatedManager.remove`, :meth:`~.RelatedManager.set`, ``get_or_create()``, and ``update_or_create()`` methods are now allowed on many-to-many relationships with intermediate models. The new ``through_defaults`` argument is used to specify values for new intermediate model instance(s). Requests and Responses ~~~~~~~~~~~~~~~~~~~~~~ * Added :attr:`.HttpRequest.headers` to allow simple access to a request's headers. Serialization ~~~~~~~~~~~~~ * You can now deserialize data using natural keys containing :ref:`forward references <natural-keys-and-forward-references>` by passing ``handle_forward_references=True`` to ``serializers.deserialize()``. Additionally, :djadmin:`loaddata` handles forward references automatically. Tests ~~~~~ * The new :meth:`.SimpleTestCase.assertURLEqual` assertion checks for a given URL, ignoring the ordering of the query string. :meth:`~.SimpleTestCase.assertRedirects` uses the new assertion. * The test :class:`~.django.test.Client` now supports automatic JSON serialization of list and tuple ``data`` when ``content_type='application/json'``. * The new :setting:`ORACLE_MANAGED_FILES <TEST_ORACLE_MANAGED_FILES>` test database setting allows using Oracle Managed Files (OMF) tablespaces. * Deferrable database constraints are now checked at the end of each :class:`~django.test.TestCase` test on SQLite 3.20+, just like on other backends that support deferrable constraints. These checks aren't implemented for older versions of SQLite because they would require expensive table introspection there. * :class:`~django.test.runner.DiscoverRunner` now skips the setup of databases not :ref:`referenced by tests<testing-multi-db>`. URLs ~~~~ * The new :attr:`.ResolverMatch.route` attribute stores the route of the matching URL pattern. Validators ~~~~~~~~~~ * :class:`.MaxValueValidator`, :class:`.MinValueValidator`, :class:`.MinLengthValidator`, and :class:`.MaxLengthValidator` now accept a callable ``limit_value``. .. _backwards-incompatible-2.2: Backwards incompatible changes in 2.2 ===================================== Database backend API -------------------- This section describes changes that may be needed in third-party database backends. * Third-party database backends must implement support for table check constraints or set ``DatabaseFeatures.supports_table_check_constraints`` to ``False``. * Third party database backends must implement support for ignoring constraints or uniqueness errors while inserting or set ``DatabaseFeatures.supports_ignore_conflicts`` to ``False``. * Third party database backends must implement introspection for ``DurationField`` or set ``DatabaseFeatures.can_introspect_duration_field`` to ``False``. * ``DatabaseFeatures.uses_savepoints`` now defaults to ``True``. * Third party database backends must implement support for partial indexes or set ``DatabaseFeatures.supports_partial_indexes`` to ``False``. * ``DatabaseIntrospection.table_name_converter()`` and ``column_name_converter()`` are removed. Third party database backends may need to instead implement ``DatabaseIntrospection.identifier_converter()``. In that case, the constraint names that ``DatabaseIntrospection.get_constraints()`` returns must be normalized by ``identifier_converter()``. * SQL generation for indexes is moved from :class:`~django.db.models.Index` to ``SchemaEditor`` and these ``SchemaEditor`` methods are added: * ``_create_primary_key_sql()`` and ``_delete_primary_key_sql()`` * ``_delete_index_sql()`` (to pair with ``_create_index_sql()``) * ``_delete_unique_sql`` (to pair with ``_create_unique_sql()``) * ``_delete_fk_sql()`` (to pair with ``_create_fk_sql()``) * ``_create_check_sql()`` and ``_delete_check_sql()`` * The third argument of ``DatabaseWrapper.__init__()``, ``allow_thread_sharing``, is removed. Admin actions are no longer collected from base ``ModelAdmin`` classes ---------------------------------------------------------------------- For example, in older versions of Django:: from django.contrib import admin class BaseAdmin(admin.ModelAdmin): actions = ['a'] class SubAdmin(BaseAdmin): actions = ['b'] ``SubAdmin`` would have actions ``'a'`` and ``'b'``. Now ``actions`` follows standard Python inheritance. To get the same result as before:: class SubAdmin(BaseAdmin): actions = BaseAdmin.actions + ['b'] :mod:`django.contrib.gis` ------------------------- * Support for GDAL 1.9 and 1.10 is dropped. ``TransactionTestCase`` serialized data loading ----------------------------------------------- Initial data migrations are now loaded in :class:`~django.test.TransactionTestCase` at the end of the test, after the database flush. In older versions, this data was loaded at the beginning of the test, but this prevents the :option:`test --keepdb` option from working properly (the database was empty at the end of the whole test suite). This change shouldn't have an impact on your tests unless you've customized :class:`~django.test.TransactionTestCase`'s internals. ``sqlparse`` is required dependency ----------------------------------- To simplify a few parts of Django's database handling, `sqlparse <https://pypi.org/project/sqlparse/>`_ is now a required dependency. It's automatically installed along with Django. ``cached_property`` aliases --------------------------- In usage like:: from django.utils.functional import cached_property class A: cached_property def base(self): return ... alias = base ``alias`` is not cached. Where the problem can be detected (Python 3.6 and later), such usage now raises ``TypeError: Cannot assign the same cached_property to two different names ('base' and 'alias').`` Use this instead:: import operator class A: ... alias = property(operator.attrgetter('base')) Permissions for proxy models ---------------------------- :ref:`Permissions for proxy models <proxy-models-permissions-topic>` are now created using the content type of the proxy model rather than the content type of the concrete model. A migration will update existing permissions when you run :djadmin:`migrate`. In the admin, the change is transparent for proxy models having the same ``app_label`` as their concrete model. However, in older versions, users with permissions for a proxy model with a *different* ``app_label`` than its concrete model couldn't access the model in the admin. That's now fixed, but you might want to audit the permissions assignments for such proxy models (``[add|view|change|delete]_myproxy``) prior to upgrading to ensure the new access is appropriate. Finally, proxy model permission strings must be updated to use their own ``app_label``. For example, for ``app.MyProxyModel`` inheriting from ``other_app.ConcreteModel``, update ``user.has_perm('other_app.add_myproxymodel')`` to ``user.has_perm('app.add_myproxymodel')``. Merging of form ``Media`` assets -------------------------------- Form ``Media`` assets are now merged using a topological sort algorithm, as the old pairwise merging algorithm is insufficient for some cases. CSS and JavaScript files which don't include their dependencies may now be sorted incorrectly (where the old algorithm produced results correctly by coincidence). Audit all ``Media`` classes for any missing dependencies. For example, widgets depending on ``django.jQuery`` must specify ``js=['admin/js/jquery.init.js', ...]`` when :ref:`declaring form media assets <assets-as-a-static-definition>`. Miscellaneous ------------- * To improve readability, the ``UUIDField`` form field now displays values with dashes, e.g. ``550e8400-e29b-41d4-a716-446655440000`` instead of ``550e8400e29b41d4a716446655440000``. * On SQLite, ``PositiveIntegerField`` and ``PositiveSmallIntegerField`` now include a check constraint to prevent negative values in the database. If you have existing invalid data and run a migration that recreates a table, you'll see ``CHECK constraint failed``. * For consistency with WSGI servers, the test client now sets the ``Content-Length`` header to a string rather than an integer. * The return value of :func:`django.utils.text.slugify` is no longer marked as HTML safe. * The default truncation character used by the :tfilter:`urlizetrunc`, :tfilter:`truncatechars`, :tfilter:`truncatechars_html`, :tfilter:`truncatewords`, and :tfilter:`truncatewords_html` template filters is now the real ellipsis character (``…``) instead of 3 dots. You may have to adapt some test output comparisons. * Support for bytestring paths in the template filesystem loader is removed. * :func:`django.utils.http.urlsafe_base64_encode` now returns a string instead of a bytestring, and :func:`django.utils.http.urlsafe_base64_decode` may no longer be passed a bytestring. * Support for ``cx_Oracle`` < 6.0 is removed. * The minimum supported version of ``mysqlclient`` is increased from 1.3.7 to 1.3.13. * The minimum supported version of SQLite is increased from 3.7.15 to 3.8.3. * In an attempt to provide more semantic query data, ``NullBooleanSelect`` now renders ``<option>`` values of ``unknown``, ``true``, and ``false`` instead of ``1``, ``2``, and ``3``. For backwards compatibility, the old values are still accepted as data. * :attr:`Group.name <django.contrib.auth.models.Group.name>` ``max_length`` is increased from 80 to 150 characters. * Tests that violate deferrable database constraints now error when run on SQLite 3.20+, just like on other backends that support such constraints. * To catch usage mistakes, the test :class:`~django.test.Client` and :func:`django.utils.http.urlencode` now raise ``TypeError`` if ``None`` is passed as a value to encode because ``None`` can't be encoded in GET and POST data. Either pass an empty string or omit the value. * The :djadmin:`ping_google` management command now defaults to ``https`` instead of ``http`` for the sitemap's URL. If your site uses http, use the new :option:`ping_google --sitemap-uses-http` option. If you use the :func:`~django.contrib.sitemaps.ping_google` function, set the new ``sitemap_uses_https`` argument to ``False``. * :djadmin:`runserver` no longer supports `pyinotify` (replaced by Watchman). * The :class:`~django.db.models.Avg`, :class:`~django.db.models.StdDev`, and :class:`~django.db.models.Variance` aggregate functions now return a ``Decimal`` instead of a ``float`` when the input is ``Decimal``. * Tests will fail on SQLite if apps without migrations have relations to apps with migrations. This has been a documented restriction since migrations were added in Django 1.7, but it fails more reliably now. You'll see tests failing with errors like ``no such table: <app_label>_<model>``. This was observed with several third-party apps that had models in tests without migrations. You must add migrations for such models. .. _deprecated-features-2.2: Features deprecated in 2.2 ========================== Model ``Meta.ordering`` will no longer affect ``GROUP BY`` queries ------------------------------------------------------------------ A model's ``Meta.ordering`` affecting ``GROUP BY`` queries (such as ``.annotate().values()``) is a common source of confusion. Such queries now issue a deprecation warning with the advice to add an ``order_by()`` to retain the current query. ``Meta.ordering`` will be ignored in such queries starting in Django 3.1. Miscellaneous ------------- * ``django.utils.timezone.FixedOffset`` is deprecated in favor of :class:`datetime.timezone`. * The undocumented ``QuerySetPaginator`` alias of ``django.core.paginator.Paginator`` is deprecated. * The ``FloatRangeField`` model and form fields in ``django.contrib.postgres`` are deprecated in favor of a new name, ``DecimalRangeField``, to match the underlying ``numrange`` data type used in the database. * The ``FILE_CHARSET`` setting is deprecated. Starting with Django 3.1, files read from disk must be UTF-8 encoded. * ``django.contrib.staticfiles.storage.CachedStaticFilesStorage`` is deprecated due to the intractable problems that it has. Use :class:`.ManifestStaticFilesStorage` or a third-party cloud storage instead. * :meth:`.RemoteUserBackend.configure_user` is now passed ``request`` as the first positional argument, if it accepts it. Support for overrides that don't accept it will be removed in Django 3.1. * The :attr:`.SimpleTestCase.allow_database_queries`, :attr:`.TransactionTestCase.multi_db`, and :attr:`.TestCase.multi_db` attributes are deprecated in favor of :attr:`.SimpleTestCase.databases`, :attr:`.TransactionTestCase.databases`, and :attr:`.TestCase.databases`. These new attributes allow databases dependencies to be declared in order to prevent unexpected queries against non-default databases to leak state between tests. The previous behavior of ``allow_database_queries=True`` and ``multi_db=True`` can be achieved by setting ``databases='__all__'``. =========================== ``` ### 2.1.11 ``` =========================== *August 1, 2019* Django 2.1.11 fixes security issues in 2.1.10. CVE-2019-14232: Denial-of-service possibility in ``django.utils.text.Truncator`` ================================================================================ If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods were passed the ``html=True`` argument, they were extremely slow to evaluate certain inputs due to a catastrophic backtracking vulnerability in a regular expression. The ``chars()`` and ``words()`` methods are used to implement the :tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template filters, which were thus vulnerable. The regular expressions used by ``Truncator`` have been simplified in order to avoid potential backtracking issues. As a consequence, trailing punctuation may now at times be included in the truncated output. CVE-2019-14233: Denial-of-service possibility in ``strip_tags()`` ================================================================= Due to the behavior of the underlying ``HTMLParser``, :func:`django.utils.html.strip_tags` would be extremely slow to evaluate certain inputs containing large sequences of nested incomplete HTML entities. The ``strip_tags()`` method is used to implement the corresponding :tfilter:`striptags` template filter, which was thus also vulnerable. ``strip_tags()`` now avoids recursive calls to ``HTMLParser`` when progress removing tags, but necessarily incomplete HTML entities, stops being made. Remember that absolutely NO guarantee is provided about the results of ``strip_tags()`` being HTML safe. So NEVER mark safe the result of a ``strip_tags()`` call without escaping it first, for example with :func:`django.utils.html.escape`. CVE-2019-14234: SQL injection possibility in key and index lookups for ``JSONField``/``HStoreField`` ==================================================================================================== :lookup:`Key and index lookups <jsonfield.key>` for :class:`~django.contrib.postgres.fields.JSONField` and :lookup:`key lookups <hstorefield.key>` for :class:`~django.contrib.postgres.fields.HStoreField` were subject to SQL injection, using a suitably crafted dictionary, with dictionary expansion, as the ``**kwargs`` passed to ``QuerySet.filter()``. CVE-2019-14235: Potential memory exhaustion in ``django.utils.encoding.uri_to_iri()`` ===================================================================================== If passed certain inputs, :func:`django.utils.encoding.uri_to_iri` could lead to significant memory usage due to excessive recursion when re-percent-encoding invalid UTF-8 octet sequences. ``uri_to_iri()`` now avoids recursion when re-percent-encoding invalid UTF-8 octet sequences. =========================== ``` ### 2.1.10 ``` =========================== *July 1, 2019* Django 2.1.10 fixes a security issue in 2.1.9. CVE-2019-12781: Incorrect HTTP detection with reverse-proxy connecting via HTTPS -------------------------------------------------------------------------------- When deployed behind a reverse-proxy connecting to Django via HTTPS, :attr:`django.http.HttpRequest.scheme` would incorrectly detect client requests made via HTTP as using HTTPS. This entails incorrect results for :meth:`~django.http.HttpRequest.is_secure`, and :meth:`~django.http.HttpRequest.build_absolute_uri`, and that HTTP requests would not be redirected to HTTPS in accordance with :setting:`SECURE_SSL_REDIRECT`. ``HttpRequest.scheme`` now respects :setting:`SECURE_PROXY_SSL_HEADER`, if it is configured, and the appropriate header is set on the request, for both HTTP and HTTPS requests. If you deploy Django behind a reverse-proxy that forwards HTTP requests, and that connects to Django via HTTPS, be sure to verify that your application correctly handles code paths relying on ``scheme``, ``is_secure()``, ``build_absolute_uri()``, and ``SECURE_SSL_REDIRECT``. ============================ ``` ### 2.1.9 ``` ============================ *June 3, 2019* Django 2.1.9 fixes security issues in 2.1.8. CVE-2019-12308: AdminURLFieldWidget XSS --------------------------------------- The clickable "Current URL" link generated by ``AdminURLFieldWidget`` displayed the provided value without validating it as a safe URL. Thus, an unvalidated value stored in the database, or a value provided as a URL query parameter payload, could result in an clickable JavaScript link. ``AdminURLFieldWidget`` now validates the provided value using :class:`~django.core.validators.URLValidator` before displaying the clickable link. You may customize the validator by passing a ``validator_class`` kwarg to ``AdminURLFieldWidget.__init__()``, e.g. when using :attr:`~django.contrib.admin.ModelAdmin.formfield_overrides`. Patched bundled jQuery for CVE-2019-11358: Prototype pollution -------------------------------------------------------------- jQuery before 3.4.0, mishandles ``jQuery.extend(true, {}, ...)`` because of ``Object.prototype`` pollution. If an unsanitized source object contained an enumerable ``__proto__`` property, it could extend the native ``Object.prototype``. The bundled version of jQuery used by the Django admin has been patched to allow for the ``select2`` library's use of ``jQuery.extend()``. ========================== ``` ### 2.1.8 ``` ========================== *April 1, 2019* Django 2.1.8 fixes a bug in 2.1.7. Bugfixes ======== * Prevented admin inlines for a ``ManyToManyField``\'s implicit through model from being editable if the user only has the view permission (:ticket:`30289`). ========================== ```Links
- PyPI: https://pypi.org/project/django - Changelog: https://pyup.io/changelogs/django/ - Homepage: https://www.djangoproject.com/