search

samdobson / monzo-coffee

Intelligent transaction tagging for data-loving Monzonauts
8 stars 2 forks source link

Scheduled monthly dependency update for August #21

Closed pyup-bot closed 5 years ago

pyup-bot commented 5 years ago

Update babel from 2.6.0 to 2.7.0.

Changelog ### 2.7.0 ``` ------------- Possibly incompatible changes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ These may be backward incompatible in some cases, as some more-or-less internal APIs have changed. Please feel free to file issues if you bump into anything strange and we'll try to help! * General: Internal uses of ``babel.util.odict`` have been replaced with ``collections.OrderedDict`` from The Python standard library. Improvements ~~~~~~~~~~~~ * CLDR: Upgrade to CLDR 35.1 - Alberto Mardegan, Aarni Koskela (626, 643) * General: allow anchoring path patterns to the start of a string - Brian Cappello (600) * General: Bumped version requirement on pytz - chrisbrake (592) * Messages: `pybabel compile`: exit with code 1 if errors were encountered - Aarni Koskela (647) * Messages: Add omit-header to update_catalog - Cédric Krier (633) * Messages: Catalog update: keep user comments from destination by default - Aarni Koskela (648) * Messages: Skip empty message when writing mo file - Cédric Krier (564) * Messages: Small fixes to avoid crashes on badly formatted .po files - Bryn Truscott (597) * Numbers: `parse_decimal()` `strict` argument and `suggestions` - Charly C (590) * Numbers: don't repeat suggestions in parse_decimal strict - Serban Constantin (599) * Numbers: implement currency formatting with long display names - Luke Plant (585) * Numbers: parse_decimal(): assume spaces are equivalent to non-breaking spaces when not in strict mode - Aarni Koskela (649) * Performance: Cache locale_identifiers() - Aarni Koskela (644) Bugfixes ~~~~~~~~ * CLDR: Skip alt=... for week data (minDays, firstDay, weekendStart, weekendEnd) - Aarni Koskela (634) * Dates: Fix wrong weeknumber for 31.12.2018 - BT-sschmid (621) * Locale: Avoid KeyError trying to get data on WindowsXP - mondeja (604) * Locale: get_display_name(): Don't attempt to concatenate variant information to None - Aarni Koskela (645) * Messages: pofile: Add comparison operators to _NormalizedString - Aarni Koskela (646) * Messages: pofile: don't crash when message.locations can't be sorted - Aarni Koskela (646) Tooling & docs ~~~~~~~~~~~~~~ * Docs: Remove all references to deprecated easy_install - Jon Dufresne (610) * Docs: Switch print statement in docs to print function - NotAFile * Docs: Update all pypi.python.org URLs to pypi.org - Jon Dufresne (587) * Docs: Use https URLs throughout project where available - Jon Dufresne (588) * Support: Add testing and document support for Python 3.7 - Jon Dufresne (611) * Support: Test on Python 3.8-dev - Aarni Koskela (642) * Support: Using ABCs from collections instead of collections.abc is deprecated. - Julien Palard (609) * Tests: Fix conftest.py compatibility with pytest 4.3 - Miro Hrončok (635) * Tests: Update pytest and pytest-cov - Miro Hrončok (635) ```
Links - PyPI: https://pypi.org/project/babel - Changelog: https://pyup.io/changelogs/babel/ - Homepage: http://babel.pocoo.org/ - Docs: https://pythonhosted.org/Babel/

Update certifi from 2018.11.29 to 2019.6.16.

The bot wasn't able to find a changelog for this release. Got an idea?

Links - PyPI: https://pypi.org/project/certifi - Homepage: https://certifi.io/

Update django from 2.1.4 to 2.2.4.

Changelog ### 2.2.4 ``` ========================== *August 1, 2019* Django 2.2.4 fixes security issues and several bugs in 2.2.3. CVE-2019-14232: Denial-of-service possibility in ``django.utils.text.Truncator`` ================================================================================ If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods were passed the ``html=True`` argument, they were extremely slow to evaluate certain inputs due to a catastrophic backtracking vulnerability in a regular expression. The ``chars()`` and ``words()`` methods are used to implement the :tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template filters, which were thus vulnerable. The regular expressions used by ``Truncator`` have been simplified in order to avoid potential backtracking issues. As a consequence, trailing punctuation may now at times be included in the truncated output. CVE-2019-14233: Denial-of-service possibility in ``strip_tags()`` ================================================================= Due to the behavior of the underlying ``HTMLParser``, :func:`django.utils.html.strip_tags` would be extremely slow to evaluate certain inputs containing large sequences of nested incomplete HTML entities. The ``strip_tags()`` method is used to implement the corresponding :tfilter:`striptags` template filter, which was thus also vulnerable. ``strip_tags()`` now avoids recursive calls to ``HTMLParser`` when progress removing tags, but necessarily incomplete HTML entities, stops being made. Remember that absolutely NO guarantee is provided about the results of ``strip_tags()`` being HTML safe. So NEVER mark safe the result of a ``strip_tags()`` call without escaping it first, for example with :func:`django.utils.html.escape`. CVE-2019-14234: SQL injection possibility in key and index lookups for ``JSONField``/``HStoreField`` ==================================================================================================== :lookup:`Key and index lookups <jsonfield.key>` for :class:`~django.contrib.postgres.fields.JSONField` and :lookup:`key lookups <hstorefield.key>` for :class:`~django.contrib.postgres.fields.HStoreField` were subject to SQL injection, using a suitably crafted dictionary, with dictionary expansion, as the ``**kwargs`` passed to ``QuerySet.filter()``. CVE-2019-14235: Potential memory exhaustion in ``django.utils.encoding.uri_to_iri()`` ===================================================================================== If passed certain inputs, :func:`django.utils.encoding.uri_to_iri` could lead to significant memory usage due to excessive recursion when re-percent-encoding invalid UTF-8 octet sequences. ``uri_to_iri()`` now avoids recursion when re-percent-encoding invalid UTF-8 octet sequences. Bugfixes ======== * Fixed a regression in Django 2.2 when ordering a ``QuerySet.union()``, ``intersection()``, or ``difference()`` by a field type present more than once results in the wrong ordering being used (:ticket:`30628`). * Fixed a migration crash on PostgreSQL when adding a check constraint with a ``contains`` lookup on :class:`~django.contrib.postgres.fields.DateRangeField` or :class:`~django.contrib.postgres.fields.DateTimeRangeField`, if the right hand side of an expression is the same type (:ticket:`30621`). * Fixed a regression in Django 2.2 where auto-reloader crashes if a file path contains nulls characters (``'\x00'``) (:ticket:`30506`). * Fixed a regression in Django 2.2 where auto-reloader crashes if a translation directory cannot be resolved (:ticket:`30647`). ========================== ``` ### 2.2.3 ``` ========================== *July 1, 2019* Django 2.2.3 fixes a security issue and several bugs in 2.2.2. Also, the latest string translations from Transifex are incorporated. CVE-2019-12781: Incorrect HTTP detection with reverse-proxy connecting via HTTPS -------------------------------------------------------------------------------- When deployed behind a reverse-proxy connecting to Django via HTTPS, :attr:`django.http.HttpRequest.scheme` would incorrectly detect client requests made via HTTP as using HTTPS. This entails incorrect results for :meth:`~django.http.HttpRequest.is_secure`, and :meth:`~django.http.HttpRequest.build_absolute_uri`, and that HTTP requests would not be redirected to HTTPS in accordance with :setting:`SECURE_SSL_REDIRECT`. ``HttpRequest.scheme`` now respects :setting:`SECURE_PROXY_SSL_HEADER`, if it is configured, and the appropriate header is set on the request, for both HTTP and HTTPS requests. If you deploy Django behind a reverse-proxy that forwards HTTP requests, and that connects to Django via HTTPS, be sure to verify that your application correctly handles code paths relying on ``scheme``, ``is_secure()``, ``build_absolute_uri()``, and ``SECURE_SSL_REDIRECT``. Bugfixes ======== * Fixed a regression in Django 2.2 where :class:`~django.db.models.Avg`, :class:`~django.db.models.StdDev`, and :class:`~django.db.models.Variance` crash with ``filter`` argument (:ticket:`30542`). * Fixed a regression in Django 2.2.2 where auto-reloader crashes with ``AttributeError``, e.g. when using ``ipdb`` (:ticket:`30588`). ========================== ``` ### 2.2.2 ``` ========================== *June 3, 2019* Django 2.2.2 fixes security issues and several bugs in 2.2.1. CVE-2019-12308: AdminURLFieldWidget XSS --------------------------------------- The clickable "Current URL" link generated by ``AdminURLFieldWidget`` displayed the provided value without validating it as a safe URL. Thus, an unvalidated value stored in the database, or a value provided as a URL query parameter payload, could result in an clickable JavaScript link. ``AdminURLFieldWidget`` now validates the provided value using :class:`~django.core.validators.URLValidator` before displaying the clickable link. You may customize the validator by passing a ``validator_class`` kwarg to ``AdminURLFieldWidget.__init__()``, e.g. when using :attr:`~django.contrib.admin.ModelAdmin.formfield_overrides`. Patched bundled jQuery for CVE-2019-11358: Prototype pollution -------------------------------------------------------------- jQuery before 3.4.0, mishandles ``jQuery.extend(true, {}, ...)`` because of ``Object.prototype`` pollution. If an unsanitized source object contained an enumerable ``__proto__`` property, it could extend the native ``Object.prototype``. The bundled version of jQuery used by the Django admin has been patched to allow for the ``select2`` library's use of ``jQuery.extend()``. Bugfixes ======== * Fixed a regression in Django 2.2 that stopped Show/Hide toggles working on dynamically added admin inlines (:ticket:`30459`). * Fixed a regression in Django 2.2 where deprecation message crashes if ``Meta.ordering`` contains an expression (:ticket:`30463`). * Fixed a regression in Django 2.2.1 where :class:`~django.contrib.postgres.search.SearchVector` generates SQL with a redundant ``Coalesce`` call (:ticket:`30488`). * Fixed a regression in Django 2.2 where auto-reloader doesn't detect changes in ``manage.py`` file when using ``StatReloader`` (:ticket:`30479`). * Fixed crash of :class:`~django.contrib.postgres.aggregates.ArrayAgg` and :class:`~django.contrib.postgres.aggregates.StringAgg` with ``ordering`` argument when used in a ``Subquery`` (:ticket:`30315`). * Fixed a regression in Django 2.2 that caused a crash of auto-reloader when an exception with custom signature is raised (:ticket:`30516`). * Fixed a regression in Django 2.2.1 where auto-reloader unnecessarily reloads translation files multiple times when using ``StatReloader`` (:ticket:`30523`). ========================== ``` ### 2.2.1 ``` ========================== *May 1, 2019* Django 2.2.1 fixes several bugs in 2.2. Bugfixes ======== * Fixed a regression in Django 2.1 that caused the incorrect quoting of database user password when using :djadmin:`dbshell` on Oracle (:ticket:`30307`). * Added compatibility for ``psycopg2`` 2.8 (:ticket:`30331`). * Fixed a regression in Django 2.2 that caused a crash when loading the template for the technical 500 debug page (:ticket:`30324`). * Fixed crash of ``ordering`` argument in :class:`~django.contrib.postgres.aggregates.ArrayAgg` and :class:`~django.contrib.postgres.aggregates.StringAgg` when it contains an expression with params (:ticket:`30332`). * Fixed a regression in Django 2.2 that caused a single instance fast-delete to not set the primary key to ``None`` (:ticket:`30330`). * Prevented :djadmin:`makemigrations` from generating infinite migrations for check constraints and partial indexes when ``condition`` contains a :class:`~python:range` object (:ticket:`30350`). * Reverted an optimization in Django 2.2 (:ticket:`29725`) that caused the inconsistent behavior of ``count()`` and ``exists()`` on a reverse many-to-many relationship with a custom manager (:ticket:`30325`). * Fixed a regression in Django 2.2 where :class:`~django.core.paginator.Paginator` crashes if ``object_list`` is a queryset ordered or aggregated over a nested ``JSONField`` key transform (:ticket:`30335`). * Fixed a regression in Django 2.2 where ``IntegerField`` validation of database limits crashes if ``limit_value`` attribute in a custom validator is callable (:ticket:`30328`). * Fixed a regression in Django 2.2 where :class:`~django.contrib.postgres.search.SearchVector` generates SQL that is not indexable (:ticket:`30385`). * Fixed a regression in Django 2.2 that caused an exception to be raised when a custom error handler could not be imported (:ticket:`30318`). * Relaxed the system check added in Django 2.2 for the admin app's dependencies to reallow use of :class:`~django.contrib.sessions.middleware.SessionMiddleware` subclasses, rather than requiring :mod:`django.contrib.sessions` to be in :setting:`INSTALLED_APPS` (:ticket:`30312`). * Increased the default timeout when using ``Watchman`` to 5 seconds to prevent falling back to ``StatReloader`` on larger projects and made it customizable via the ``DJANGO_WATCHMAN_TIMEOUT`` environment variable (:ticket:`30361`). * Fixed a regression in Django 2.2 that caused a crash when migrating permissions for proxy models if the target permissions already existed. For example, when a permission had been created manually or a model had been migrated from concrete to proxy (:ticket:`30351`). * Fixed a regression in Django 2.2 that caused a crash of :djadmin:`runserver` when URLConf modules raised exceptions (:ticket:`30323`). * Fixed a regression in Django 2.2 where changes were not reliably detected by auto-reloader when using ``StatReloader`` (:ticket:`30323`). * Fixed a migration crash on Oracle and PostgreSQL when adding a check constraint with a ``contains``, ``startswith``, or ``endswith`` lookup (or their case-insensitive variant) (:ticket:`30408`). * Fixed a migration crash on Oracle and SQLite when adding a check constraint with ``condition`` contains ``|`` (``OR``) operator (:ticket:`30412`). ======================== ``` ### 2.2 ``` ======================== *April 1, 2019* Welcome to Django 2.2! These release notes cover the :ref:`new features <whats-new-2.2>`, as well as some :ref:`backwards incompatible changes <backwards-incompatible-2.2>` you'll want to be aware of when upgrading from Django 2.1 or earlier. We've :ref:`begun the deprecation process for some features <deprecated-features-2.2>`. See the :doc:`/howto/upgrade-version` guide if you're updating an existing project. Django 2.2 is designated as a :term:`long-term support release`. It will receive security updates for at least three years after its release. Support for the previous LTS, Django 1.11, will end in April 2020. Python compatibility ==================== Django 2.2 supports Python 3.5, 3.6, and 3.7. We **highly recommend** and only officially support the latest release of each series. .. _whats-new-2.2: What's new in Django 2.2 ======================== Constraints ----------- The new :class:`~django.db.models.CheckConstraint` and :class:`~django.db.models.UniqueConstraint` classes enable adding custom database constraints. Constraints are added to models using the :attr:`Meta.constraints <django.db.models.Options.constraints>` option. Minor features -------------- :mod:`django.contrib.admin` ~~~~~~~~~~~~~~~~~~~~~~~~~~~ * Added a CSS class to the column headers of :class:`~django.contrib.admin.TabularInline`. :mod:`django.contrib.auth` ~~~~~~~~~~~~~~~~~~~~~~~~~~ * The ``HttpRequest`` is now passed as the first positional argument to :meth:`.RemoteUserBackend.configure_user`, if it accepts it. :mod:`django.contrib.gis` ~~~~~~~~~~~~~~~~~~~~~~~~~ * Added Oracle support for the :class:`~django.contrib.gis.db.models.functions.Envelope` function. * Added SpatiaLite support for the :lookup:`coveredby` and :lookup:`covers` lookups. :mod:`django.contrib.postgres` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * The new ``ordering`` argument for :class:`~django.contrib.postgres.aggregates.ArrayAgg` and :class:`~django.contrib.postgres.aggregates.StringAgg` determines the ordering of the aggregated elements. * The new :class:`~django.contrib.postgres.indexes.BTreeIndex`, :class:`~django.contrib.postgres.indexes.HashIndex` and :class:`~django.contrib.postgres.indexes.SpGistIndex` classes allow creating ``B-Tree``, ``hash``, and ``SP-GiST`` indexes in the database. * :class:`~django.contrib.postgres.indexes.BrinIndex` now has the ``autosummarize`` parameter. * The new ``search_type`` parameter of :class:`~django.contrib.postgres.search.SearchQuery` allows searching for a phrase or raw expression. :mod:`django.contrib.staticfiles` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * Added path matching to the :option:`collectstatic --ignore` option so that patterns like ``/vendor/*.js`` can be used. Database backends ~~~~~~~~~~~~~~~~~ * Added result streaming for :meth:`.QuerySet.iterator` on SQLite. Generic Views ~~~~~~~~~~~~~ * The new :meth:`View.setup <django.views.generic.base.View.setup>` hook initializes view attributes before calling :meth:`~django.views.generic.base.View.dispatch`. It allows mixins to setup instance attributes for reuse in child classes. Internationalization ~~~~~~~~~~~~~~~~~~~~ * Added support and translations for the Armenian language. Management Commands ~~~~~~~~~~~~~~~~~~~ * The new :option:`--force-color` option forces colorization of the command output. * :djadmin:`inspectdb` now creates models for foreign tables on PostgreSQL. * :option:`inspectdb --include-views` now creates models for materialized views on Oracle and PostgreSQL. * The new :option:`inspectdb --include-partitions` option allows creating models for partition tables on PostgreSQL. In older versions, models are created child tables instead the parent. * :djadmin:`inspectdb` now introspects :class:`~django.db.models.DurationField` for Oracle and PostgreSQL, and :class:`~django.db.models.AutoField` for SQLite. * On Oracle, :djadmin:`dbshell` is wrapped with ``rlwrap``, if available. ``rlwrap`` provides a command history and editing of keyboard input. * The new :option:`makemigrations --no-header` option avoids writing header comments in generated migration file(s). This option is also available for :djadmin:`squashmigrations`. * :djadmin:`runserver` can now use `Watchman <https://facebook.github.io/watchman/>`_ to improve the performance of watching a large number of files for changes. Migrations ~~~~~~~~~~ * The new :option:`migrate --plan` option prints the list of migration operations that will be performed. * ``NoneType`` can now be serialized in migrations. * You can now :ref:`register custom serializers <custom-migration-serializers>` for migrations. Models ~~~~~~ * Added support for PostgreSQL operator classes (:attr:`.Index.opclasses`). * Added support for partial indexes (:attr:`.Index.condition`). * Added the :class:`~django.db.models.functions.NullIf` and :class:`~django.db.models.functions.Reverse` database functions, as well as many :ref:`math database functions <math-functions>`. * Setting the new ``ignore_conflicts`` parameter of :meth:`.QuerySet.bulk_create` to ``True`` tells the database to ignore failure to insert rows that fail uniqueness constraints or other checks. * The new :class:`~django.db.models.functions.ExtractIsoYear` function extracts ISO-8601 week-numbering years from :class:`~django.db.models.DateField` and :class:`~django.db.models.DateTimeField`, and the new :lookup:`iso_year` lookup allows querying by an ISO-8601 week-numbering year. * The new :meth:`.QuerySet.bulk_update` method allows efficiently updating specific fields on multiple model instances. * Django no longer always starts a transaction when a single query is being performed, such as ``Model.save()``, ``QuerySet.update()``, and ``Model.delete()``. This improves the performance of autocommit by reducing the number of database round trips. * Added SQLite support for the :class:`~django.db.models.StdDev` and :class:`~django.db.models.Variance` functions. * The handling of ``DISTINCT`` aggregation is added to the :class:`~django.db.models.Aggregate` class. Adding :attr:`allow_distinct = True <django.db.models.Aggregate.allow_distinct>` as a class attribute on ``Aggregate`` subclasses allows a ``distinct`` keyword argument to be specified on initialization to ensure that the aggregate function is only called for each distinct value of ``expressions``. * The :meth:`.RelatedManager.add`, :meth:`~.RelatedManager.create`, :meth:`~.RelatedManager.remove`, :meth:`~.RelatedManager.set`, ``get_or_create()``, and ``update_or_create()`` methods are now allowed on many-to-many relationships with intermediate models. The new ``through_defaults`` argument is used to specify values for new intermediate model instance(s). Requests and Responses ~~~~~~~~~~~~~~~~~~~~~~ * Added :attr:`.HttpRequest.headers` to allow simple access to a request's headers. Serialization ~~~~~~~~~~~~~ * You can now deserialize data using natural keys containing :ref:`forward references <natural-keys-and-forward-references>` by passing ``handle_forward_references=True`` to ``serializers.deserialize()``. Additionally, :djadmin:`loaddata` handles forward references automatically. Tests ~~~~~ * The new :meth:`.SimpleTestCase.assertURLEqual` assertion checks for a given URL, ignoring the ordering of the query string. :meth:`~.SimpleTestCase.assertRedirects` uses the new assertion. * The test :class:`~.django.test.Client` now supports automatic JSON serialization of list and tuple ``data`` when ``content_type='application/json'``. * The new :setting:`ORACLE_MANAGED_FILES <TEST_ORACLE_MANAGED_FILES>` test database setting allows using Oracle Managed Files (OMF) tablespaces. * Deferrable database constraints are now checked at the end of each :class:`~django.test.TestCase` test on SQLite 3.20+, just like on other backends that support deferrable constraints. These checks aren't implemented for older versions of SQLite because they would require expensive table introspection there. * :class:`~django.test.runner.DiscoverRunner` now skips the setup of databases not :ref:`referenced by tests<testing-multi-db>`. URLs ~~~~ * The new :attr:`.ResolverMatch.route` attribute stores the route of the matching URL pattern. Validators ~~~~~~~~~~ * :class:`.MaxValueValidator`, :class:`.MinValueValidator`, :class:`.MinLengthValidator`, and :class:`.MaxLengthValidator` now accept a callable ``limit_value``. .. _backwards-incompatible-2.2: Backwards incompatible changes in 2.2 ===================================== Database backend API -------------------- This section describes changes that may be needed in third-party database backends. * Third-party database backends must implement support for table check constraints or set ``DatabaseFeatures.supports_table_check_constraints`` to ``False``. * Third party database backends must implement support for ignoring constraints or uniqueness errors while inserting or set ``DatabaseFeatures.supports_ignore_conflicts`` to ``False``. * Third party database backends must implement introspection for ``DurationField`` or set ``DatabaseFeatures.can_introspect_duration_field`` to ``False``. * ``DatabaseFeatures.uses_savepoints`` now defaults to ``True``. * Third party database backends must implement support for partial indexes or set ``DatabaseFeatures.supports_partial_indexes`` to ``False``. * ``DatabaseIntrospection.table_name_converter()`` and ``column_name_converter()`` are removed. Third party database backends may need to instead implement ``DatabaseIntrospection.identifier_converter()``. In that case, the constraint names that ``DatabaseIntrospection.get_constraints()`` returns must be normalized by ``identifier_converter()``. * SQL generation for indexes is moved from :class:`~django.db.models.Index` to ``SchemaEditor`` and these ``SchemaEditor`` methods are added: * ``_create_primary_key_sql()`` and ``_delete_primary_key_sql()`` * ``_delete_index_sql()`` (to pair with ``_create_index_sql()``) * ``_delete_unique_sql`` (to pair with ``_create_unique_sql()``) * ``_delete_fk_sql()`` (to pair with ``_create_fk_sql()``) * ``_create_check_sql()`` and ``_delete_check_sql()`` * The third argument of ``DatabaseWrapper.__init__()``, ``allow_thread_sharing``, is removed. Admin actions are no longer collected from base ``ModelAdmin`` classes ---------------------------------------------------------------------- For example, in older versions of Django:: from django.contrib import admin class BaseAdmin(admin.ModelAdmin): actions = ['a'] class SubAdmin(BaseAdmin): actions = ['b'] ``SubAdmin`` would have actions ``'a'`` and ``'b'``. Now ``actions`` follows standard Python inheritance. To get the same result as before:: class SubAdmin(BaseAdmin): actions = BaseAdmin.actions + ['b'] :mod:`django.contrib.gis` ------------------------- * Support for GDAL 1.9 and 1.10 is dropped. ``TransactionTestCase`` serialized data loading ----------------------------------------------- Initial data migrations are now loaded in :class:`~django.test.TransactionTestCase` at the end of the test, after the database flush. In older versions, this data was loaded at the beginning of the test, but this prevents the :option:`test --keepdb` option from working properly (the database was empty at the end of the whole test suite). This change shouldn't have an impact on your tests unless you've customized :class:`~django.test.TransactionTestCase`'s internals. ``sqlparse`` is required dependency ----------------------------------- To simplify a few parts of Django's database handling, `sqlparse <https://pypi.org/project/sqlparse/>`_ is now a required dependency. It's automatically installed along with Django. ``cached_property`` aliases --------------------------- In usage like:: from django.utils.functional import cached_property class A: cached_property def base(self): return ... alias = base ``alias`` is not cached. Where the problem can be detected (Python 3.6 and later), such usage now raises ``TypeError: Cannot assign the same cached_property to two different names ('base' and 'alias').`` Use this instead:: import operator class A: ... alias = property(operator.attrgetter('base')) Permissions for proxy models ---------------------------- :ref:`Permissions for proxy models <proxy-models-permissions-topic>` are now created using the content type of the proxy model rather than the content type of the concrete model. A migration will update existing permissions when you run :djadmin:`migrate`. In the admin, the change is transparent for proxy models having the same ``app_label`` as their concrete model. However, in older versions, users with permissions for a proxy model with a *different* ``app_label`` than its concrete model couldn't access the model in the admin. That's now fixed, but you might want to audit the permissions assignments for such proxy models (``[add|view|change|delete]_myproxy``) prior to upgrading to ensure the new access is appropriate. Finally, proxy model permission strings must be updated to use their own ``app_label``. For example, for ``app.MyProxyModel`` inheriting from ``other_app.ConcreteModel``, update ``user.has_perm('other_app.add_myproxymodel')`` to ``user.has_perm('app.add_myproxymodel')``. Merging of form ``Media`` assets -------------------------------- Form ``Media`` assets are now merged using a topological sort algorithm, as the old pairwise merging algorithm is insufficient for some cases. CSS and JavaScript files which don't include their dependencies may now be sorted incorrectly (where the old algorithm produced results correctly by coincidence). Audit all ``Media`` classes for any missing dependencies. For example, widgets depending on ``django.jQuery`` must specify ``js=['admin/js/jquery.init.js', ...]`` when :ref:`declaring form media assets <assets-as-a-static-definition>`. Miscellaneous ------------- * To improve readability, the ``UUIDField`` form field now displays values with dashes, e.g. ``550e8400-e29b-41d4-a716-446655440000`` instead of ``550e8400e29b41d4a716446655440000``. * On SQLite, ``PositiveIntegerField`` and ``PositiveSmallIntegerField`` now include a check constraint to prevent negative values in the database. If you have existing invalid data and run a migration that recreates a table, you'll see ``CHECK constraint failed``. * For consistency with WSGI servers, the test client now sets the ``Content-Length`` header to a string rather than an integer. * The return value of :func:`django.utils.text.slugify` is no longer marked as HTML safe. * The default truncation character used by the :tfilter:`urlizetrunc`, :tfilter:`truncatechars`, :tfilter:`truncatechars_html`, :tfilter:`truncatewords`, and :tfilter:`truncatewords_html` template filters is now the real ellipsis character (``…``) instead of 3 dots. You may have to adapt some test output comparisons. * Support for bytestring paths in the template filesystem loader is removed. * :func:`django.utils.http.urlsafe_base64_encode` now returns a string instead of a bytestring, and :func:`django.utils.http.urlsafe_base64_decode` may no longer be passed a bytestring. * Support for ``cx_Oracle`` < 6.0 is removed. * The minimum supported version of ``mysqlclient`` is increased from 1.3.7 to 1.3.13. * The minimum supported version of SQLite is increased from 3.7.15 to 3.8.3. * In an attempt to provide more semantic query data, ``NullBooleanSelect`` now renders ``<option>`` values of ``unknown``, ``true``, and ``false`` instead of ``1``, ``2``, and ``3``. For backwards compatibility, the old values are still accepted as data. * :attr:`Group.name <django.contrib.auth.models.Group.name>` ``max_length`` is increased from 80 to 150 characters. * Tests that violate deferrable database constraints now error when run on SQLite 3.20+, just like on other backends that support such constraints. * To catch usage mistakes, the test :class:`~django.test.Client` and :func:`django.utils.http.urlencode` now raise ``TypeError`` if ``None`` is passed as a value to encode because ``None`` can't be encoded in GET and POST data. Either pass an empty string or omit the value. * The :djadmin:`ping_google` management command now defaults to ``https`` instead of ``http`` for the sitemap's URL. If your site uses http, use the new :option:`ping_google --sitemap-uses-http` option. If you use the :func:`~django.contrib.sitemaps.ping_google` function, set the new ``sitemap_uses_https`` argument to ``False``. * :djadmin:`runserver` no longer supports `pyinotify` (replaced by Watchman). * The :class:`~django.db.models.Avg`, :class:`~django.db.models.StdDev`, and :class:`~django.db.models.Variance` aggregate functions now return a ``Decimal`` instead of a ``float`` when the input is ``Decimal``. * Tests will fail on SQLite if apps without migrations have relations to apps with migrations. This has been a documented restriction since migrations were added in Django 1.7, but it fails more reliably now. You'll see tests failing with errors like ``no such table: <app_label>_<model>``. This was observed with several third-party apps that had models in tests without migrations. You must add migrations for such models. .. _deprecated-features-2.2: Features deprecated in 2.2 ========================== Model ``Meta.ordering`` will no longer affect ``GROUP BY`` queries ------------------------------------------------------------------ A model's ``Meta.ordering`` affecting ``GROUP BY`` queries (such as ``.annotate().values()``) is a common source of confusion. Such queries now issue a deprecation warning with the advice to add an ``order_by()`` to retain the current query. ``Meta.ordering`` will be ignored in such queries starting in Django 3.1. Miscellaneous ------------- * ``django.utils.timezone.FixedOffset`` is deprecated in favor of :class:`datetime.timezone`. * The undocumented ``QuerySetPaginator`` alias of ``django.core.paginator.Paginator`` is deprecated. * The ``FloatRangeField`` model and form fields in ``django.contrib.postgres`` are deprecated in favor of a new name, ``DecimalRangeField``, to match the underlying ``numrange`` data type used in the database. * The ``FILE_CHARSET`` setting is deprecated. Starting with Django 3.1, files read from disk must be UTF-8 encoded. * ``django.contrib.staticfiles.storage.CachedStaticFilesStorage`` is deprecated due to the intractable problems that it has. Use :class:`.ManifestStaticFilesStorage` or a third-party cloud storage instead. * :meth:`.RemoteUserBackend.configure_user` is now passed ``request`` as the first positional argument, if it accepts it. Support for overrides that don't accept it will be removed in Django 3.1. * The :attr:`.SimpleTestCase.allow_database_queries`, :attr:`.TransactionTestCase.multi_db`, and :attr:`.TestCase.multi_db` attributes are deprecated in favor of :attr:`.SimpleTestCase.databases`, :attr:`.TransactionTestCase.databases`, and :attr:`.TestCase.databases`. These new attributes allow databases dependencies to be declared in order to prevent unexpected queries against non-default databases to leak state between tests. The previous behavior of ``allow_database_queries=True`` and ``multi_db=True`` can be achieved by setting ``databases='__all__'``. =========================== ``` ### 2.1.11 ``` =========================== *August 1, 2019* Django 2.1.11 fixes security issues in 2.1.10. CVE-2019-14232: Denial-of-service possibility in ``django.utils.text.Truncator`` ================================================================================ If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods were passed the ``html=True`` argument, they were extremely slow to evaluate certain inputs due to a catastrophic backtracking vulnerability in a regular expression. The ``chars()`` and ``words()`` methods are used to implement the :tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template filters, which were thus vulnerable. The regular expressions used by ``Truncator`` have been simplified in order to avoid potential backtracking issues. As a consequence, trailing punctuation may now at times be included in the truncated output. CVE-2019-14233: Denial-of-service possibility in ``strip_tags()`` ================================================================= Due to the behavior of the underlying ``HTMLParser``, :func:`django.utils.html.strip_tags` would be extremely slow to evaluate certain inputs containing large sequences of nested incomplete HTML entities. The ``strip_tags()`` method is used to implement the corresponding :tfilter:`striptags` template filter, which was thus also vulnerable. ``strip_tags()`` now avoids recursive calls to ``HTMLParser`` when progress removing tags, but necessarily incomplete HTML entities, stops being made. Remember that absolutely NO guarantee is provided about the results of ``strip_tags()`` being HTML safe. So NEVER mark safe the result of a ``strip_tags()`` call without escaping it first, for example with :func:`django.utils.html.escape`. CVE-2019-14234: SQL injection possibility in key and index lookups for ``JSONField``/``HStoreField`` ==================================================================================================== :lookup:`Key and index lookups <jsonfield.key>` for :class:`~django.contrib.postgres.fields.JSONField` and :lookup:`key lookups <hstorefield.key>` for :class:`~django.contrib.postgres.fields.HStoreField` were subject to SQL injection, using a suitably crafted dictionary, with dictionary expansion, as the ``**kwargs`` passed to ``QuerySet.filter()``. CVE-2019-14235: Potential memory exhaustion in ``django.utils.encoding.uri_to_iri()`` ===================================================================================== If passed certain inputs, :func:`django.utils.encoding.uri_to_iri` could lead to significant memory usage due to excessive recursion when re-percent-encoding invalid UTF-8 octet sequences. ``uri_to_iri()`` now avoids recursion when re-percent-encoding invalid UTF-8 octet sequences. =========================== ``` ### 2.1.10 ``` =========================== *July 1, 2019* Django 2.1.10 fixes a security issue in 2.1.9. CVE-2019-12781: Incorrect HTTP detection with reverse-proxy connecting via HTTPS -------------------------------------------------------------------------------- When deployed behind a reverse-proxy connecting to Django via HTTPS, :attr:`django.http.HttpRequest.scheme` would incorrectly detect client requests made via HTTP as using HTTPS. This entails incorrect results for :meth:`~django.http.HttpRequest.is_secure`, and :meth:`~django.http.HttpRequest.build_absolute_uri`, and that HTTP requests would not be redirected to HTTPS in accordance with :setting:`SECURE_SSL_REDIRECT`. ``HttpRequest.scheme`` now respects :setting:`SECURE_PROXY_SSL_HEADER`, if it is configured, and the appropriate header is set on the request, for both HTTP and HTTPS requests. If you deploy Django behind a reverse-proxy that forwards HTTP requests, and that connects to Django via HTTPS, be sure to verify that your application correctly handles code paths relying on ``scheme``, ``is_secure()``, ``build_absolute_uri()``, and ``SECURE_SSL_REDIRECT``. ============================ ``` ### 2.1.9 ``` ============================ *June 3, 2019* Django 2.1.9 fixes security issues in 2.1.8. CVE-2019-12308: AdminURLFieldWidget XSS --------------------------------------- The clickable "Current URL" link generated by ``AdminURLFieldWidget`` displayed the provided value without validating it as a safe URL. Thus, an unvalidated value stored in the database, or a value provided as a URL query parameter payload, could result in an clickable JavaScript link. ``AdminURLFieldWidget`` now validates the provided value using :class:`~django.core.validators.URLValidator` before displaying the clickable link. You may customize the validator by passing a ``validator_class`` kwarg to ``AdminURLFieldWidget.__init__()``, e.g. when using :attr:`~django.contrib.admin.ModelAdmin.formfield_overrides`. Patched bundled jQuery for CVE-2019-11358: Prototype pollution -------------------------------------------------------------- jQuery before 3.4.0, mishandles ``jQuery.extend(true, {}, ...)`` because of ``Object.prototype`` pollution. If an unsanitized source object contained an enumerable ``__proto__`` property, it could extend the native ``Object.prototype``. The bundled version of jQuery used by the Django admin has been patched to allow for the ``select2`` library's use of ``jQuery.extend()``. ========================== ``` ### 2.1.8 ``` ========================== *April 1, 2019* Django 2.1.8 fixes a bug in 2.1.7. Bugfixes ======== * Prevented admin inlines for a ``ManyToManyField``\'s implicit through model from being editable if the user only has the view permission (:ticket:`30289`). ========================== ``` ### 2.1.7 ``` ========================== *February 11, 2019* Django 2.1.7 fixes a packaging error in 2.1.6. Bugfixes ======== * Corrected packaging error from 2.1.6 (:ticket:`30175`). ========================== ``` ### 2.1.5 ``` ========================== *January 4, 2019* Django 2.1.5 fixes a security issue and several bugs in 2.1.4. CVE-2019-3498: Content spoofing possibility in the default 404 page ------------------------------------------------------------------- An attacker could craft a malicious URL that could make spoofed content appear on the default page generated by the ``django.views.defaults.page_not_found()`` view. The URL path is no longer displayed in the default 404 template and the ``request_path`` context variable is now quoted to fix the issue for custom templates that use the path. Bugfixes ======== * Fixed compatibility with mysqlclient 1.3.14 (:ticket:`30013`). * Fixed a schema corruption issue on SQLite 3.26+. You might have to drop and rebuild your SQLite database if you applied a migration while using an older version of Django with SQLite 3.26 or later (:ticket:`29182`). * Prevented SQLite schema alterations while foreign key checks are enabled to avoid the possibility of schema corruption (:ticket:`30023`). * Fixed a regression in Django 2.1.4 (which enabled keep-alive connections) where request body data isn't properly consumed for such connections (:ticket:`30015`). * Fixed a regression in Django 2.1.4 where ``InlineModelAdmin.has_change_permission()`` is incorrectly called with a non-``None`` ``obj`` argument during an object add (:ticket:`30050`). ========================== ```
Links - PyPI: https://pypi.org/project/django - Changelog: https://pyup.io/changelogs/django/ - Homepage: https://www.djangoproject.com/

Update docutils from 0.14 to 0.15.2.

Changelog ### 0.15 ``` ============ .. Note:: Docutils 0.14.x is the last version supporting Python 2.4, 2.5, 3.1, and 3.2. Docutils 0.15.x is compatible with Python versions 2.6, 2.7 and 3.3 to 3.5 (cf. `Python 3 compatibility`_). * reStructuredText: - Allow embedded colons in field list field names (before, tokens like ``:this:example:`` were considered ordinary text). - Fixed a bug with the "trim" options of the "unicode" directive. * languages: Added Korean (ko) mappings and latin. * Several fixes to keep mor information on source in parsed elements, isolate documents roles from other documents parsed, smartquotes, table gets width and latex table multicolumn cells, ... ```
Links - PyPI: https://pypi.org/project/docutils - Changelog: https://pyup.io/changelogs/docutils/ - Homepage: http://docutils.sourceforge.net/

Update jinja2 from 2.10 to 2.10.1.

The bot wasn't able to find a changelog for this release. Got an idea?

Links - PyPI: https://pypi.org/project/jinja2 - Homepage: http://jinja.pocoo.org/

Update livereload from 2.6.0 to 2.6.1.

Changelog ### 2.6.1 ``` ------------- Released on May 7, 2019 1. Fixed bugs ```
Links - PyPI: https://pypi.org/project/livereload - Changelog: https://pyup.io/changelogs/livereload/ - Repo: https://github.com/lepture/python-livereload

Update markupsafe from 1.1.0 to 1.1.1.

The bot wasn't able to find a changelog for this release. Got an idea?

Links - PyPI: https://pypi.org/project/markupsafe - Changelog: https://pyup.io/changelogs/markupsafe/ - Homepage: https://palletsprojects.com/p/markupsafe/

Update packaging from 18.0 to 19.1.

Changelog ### 19.0 ``` ~~~~~~~~~~~~~~~~~ * Fix string representation of PEP 508 direct URL requirements with markers. * Better handling of file URLs This allows for using ``file:///absolute/path``, which was previously prevented due to the missing ``netloc``. This allows for all file URLs that ``urlunparse`` turns back into the original URL to be valid. ```
Links - PyPI: https://pypi.org/project/packaging - Changelog: https://pyup.io/changelogs/packaging/ - Repo: https://github.com/pypa/packaging

Update port-for from 0.3.1 to 0.4.

The bot wasn't able to find a changelog for this release. Got an idea?

Links - PyPI: https://pypi.org/project/port-for - Repo: https://github.com/kmike/port-for/

Update psycopg2 from 2.7.6.1 to 2.8.3.

Changelog ### 2.8.3 ``` ^^^^^^^^^^^^^^^^^^^^^^^^^^^ - Added *interval_status* parameter to `~psycopg2.extras.ReplicationCursor.start_replication()` method and other facilities to send automatic replication keepalives at periodic intervals (:ticket:`913`). - Fixed namedtuples caching introduced in 2.8 (:ticket:`928`). ``` ### 2.8.2 ``` ^^^^^^^^^^^^^^^^^^^^^^^^^^^ - Fixed `~psycopg2.extras.RealDictCursor` when there are repeated columns (:ticket:`884`). - Binary packages built with openssl 1.1.1b. Should fix concurrency problems (:tickets:`543, 836`). ``` ### 2.8.1 ``` ^^^^^^^^^^^^^^^^^^^^^^^^^^^ - Fixed `~psycopg2.extras.RealDictRow` modifiability (:ticket:`886`). - Fixed "there's no async cursor" error polling a connection with no cursor (:ticket:`887`). ``` ### 2.8 ``` ------------------------- New features: - Added `~psycopg2.errors` module. Every PostgreSQL error is converted into a specific exception class (:ticket:`682`). - Added `~psycopg2.extensions.encrypt_password()` function (:ticket:`576`). - Added `~psycopg2.extensions.BYTES` adapter to manage databases with mixed encodings on Python 3 (:ticket:`835`). - Added `~psycopg2.extensions.Column.table_oid` and `~psycopg2.extensions.Column.table_column` attributes on `cursor.description` items (:ticket:`661`). - Added `connection.info` object to retrieve various PostgreSQL connection information (:ticket:`726`). - Added `~connection.get_native_connection()` to expose the raw ``PGconn`` structure to C extensions via Capsule (:ticket:`782`). - Added `~connection.pgconn_ptr` and `~cursor.pgresult_ptr` to expose raw C structures to Python and interact with libpq via ctypes (:ticket:`782`). - `~psycopg2.sql.Identifier` can represent qualified names in SQL composition (:ticket:`732`). - Added `!ReplicationCursor`.\ `~psycopg2.extras.ReplicationCursor.wal_end` attribute (:ticket:`800`). - Added *fetch* parameter to `~psycopg2.extras.execute_values()` function (:ticket:`813`). - `!str()` on `~psycopg2.extras.Range` produces a human-readable representation (:ticket:`773`). - `~psycopg2.extras.DictCursor` and `~psycopg2.extras.RealDictCursor` rows maintain columns order (:ticket:`177`). - Added `~psycopg2.extensions.Diagnostics.severity_nonlocalized` attribute on the `~psycopg2.extensions.Diagnostics` object (:ticket:`783`). - More efficient `~psycopg2.extras.NamedTupleCursor` (:ticket:`838`). Bug fixes: - Fixed connections occasionally broken by the unrelated use of the multiprocessing module (:ticket:`829`). - Fixed async communication blocking if results are returned in different chunks, e.g. with notices interspersed to the results (:ticket:`856`). - Fixed adaptation of numeric subclasses such as `~enum.IntEnum` (:ticket:`591`). Other changes: - Dropped support for Python 2.6, 3.2, 3.3. - Dropped `psycopg1` module. - Dropped deprecated `!register_tstz_w_secs()` (was previously a no-op). - Dropped deprecated `!PersistentConnectionPool`. This pool class was mostly designed to interact with Zope. Use `!ZPsycopgDA.pool` instead. - Binary packages no longer installed by default. The 'psycopg2-binary' package must be used explicitly. - Dropped `!PSYCOPG_DISPLAY_SIZE` build parameter. - Dropped support for mxDateTime as the default date and time adapter. mxDatetime support continues to be available as an alternative to Python's builtin datetime. - No longer use 2to3 during installation for Python 2 & 3 compatibility. All source files are now compatible with Python 2 & 3 as is. - The `!psycopg2.test` package is no longer installed by ``python setup.py install``. - Wheel package compiled against OpenSSL 1.0.2r and PostgreSQL 11.2 libpq. ``` ### 2.7.7 ``` ^^^^^^^^^^^^^^^^^^^^^^^^^^^ - Cleanup of the cursor results assignment code, which might have solved double free and inconsistencies in concurrent usage (:tickets:`346, 384`). - Wheel package compiled against OpenSSL 1.0.2q. ```
Links - PyPI: https://pypi.org/project/psycopg2 - Changelog: https://pyup.io/changelogs/psycopg2/ - Homepage: http://initd.org/psycopg/

Update pygments from 2.3.0 to 2.4.2.

The bot wasn't able to find a changelog for this release. Got an idea?

Links - PyPI: https://pypi.org/project/pygments - Homepage: http://pygments.org/

Update pyparsing from 2.3.0 to 2.4.2.

Changelog ### 2.4.2 ``` - API change adding support for `expr[...]` - the original code in 2.4.1 incorrectly implemented this as OneOrMore. Code using this feature under this relase should explicitly use `expr[0, ...]` for ZeroOrMore and `expr[1, ...]` for OneOrMore. In 2.4.2 you will be able to write `expr[...]` equivalent to `ZeroOrMore(expr)`. - Bug if composing And, Or, MatchFirst, or Each expressions using an expression. This only affects code which uses explicit expression construction using the And, Or, etc. classes instead of using overloaded operators '+', '^', and so on. If constructing an And using a single expression, you may get an error that "cannot multiply ParserElement by 0 or (0, 0)" or a Python `IndexError`. Change code like cmd = Or(Word(alphas)) to cmd = Or([Word(alphas)]) (Note that this is not the recommended style for constructing Or expressions.) - Some newly-added `__diag__` switches are enabled by default, which may give rise to noisy user warnings for existing parsers. You can disable them using: import pyparsing as pp pp.__diag__.warn_multiple_tokens_in_named_alternation = False pp.__diag__.warn_ungrouped_named_tokens_in_collection = False pp.__diag__.warn_name_set_on_empty_Forward = False pp.__diag__.warn_on_multiple_string_args_to_oneof = False pp.__diag__.enable_debug_on_named_expressions = False In 2.4.2 these will all be set to False by default. ``` ### 2.4.2a1 ``` ---------------------------- It turns out I got the meaning of `[...]` absolutely backwards, so I've deleted 2.4.1 and am repushing this release as 2.4.2a1 for people to give it a try before I can call it ready to go. The `expr[...]` notation was pushed out to be synonymous with `OneOrMore(expr)`, but this is really counter to most Python notations (and even other internal pyparsing notations as well). It should have been defined to be equivalent to ZeroOrMore(expr). - Changed [...] to emit ZeroOrMore instead of OneOrMore. - Removed code that treats ParserElements like iterables. - Change all __diag__ switches to False. ``` ### 2.4.1.1 ``` ------------------------------- This is a re-release of version 2.4.1 to restore the release history in PyPI, since the 2.4.1 release was deleted. There are 3 known issues in this release, which are fixed in ``` ### 2.4.1 ``` -------------------------- - NOTE: Deprecated functions and features that will be dropped in pyparsing 2.5.0 (planned next release): . support for Python 2 - ongoing users running with Python 2 can continue to use pyparsing 2.4.1 . ParseResults.asXML() - if used for debugging, switch to using ParseResults.dump(); if used for data transfer, use ParseResults.asDict() to convert to a nested Python dict, which can then be converted to XML or JSON or other transfer format . operatorPrecedence synonym for infixNotation - convert to calling infixNotation . commaSeparatedList - convert to using pyparsing_common.comma_separated_list . upcaseTokens and downcaseTokens - convert to using pyparsing_common.upcaseTokens and downcaseTokens . __compat__.collect_all_And_tokens will not be settable to False to revert to pre-2.3.1 results name behavior - review use of names for MatchFirst and Or expressions containing And expressions, as they will return the complete list of parsed tokens, not just the first one. Use __diag__.warn_multiple_tokens_in_named_alternation (described below) to help identify those expressions in your parsers that will have changed as a result. - A new shorthand notation has been added for repetition expressions: expr[min, max], with '...' valid as a min or max value: - expr[...] is equivalent to OneOrMore(expr) - expr[0, ...] is equivalent to ZeroOrMore(expr) - expr[1, ...] is equivalent to OneOrMore(expr) - expr[n, ...] or expr[n,] is equivalent to expr*n + ZeroOrMore(expr) (read as "n or more instances of expr") - expr[..., n] is equivalent to expr*(0, n) - expr[m, n] is equivalent to expr*(m, n) Note that expr[..., n] and expr[m, n] do not raise an exception if more than n exprs exist in the input stream. If this behavior is desired, then write expr[..., n] + ~expr. - '...' can also be used as short hand for SkipTo when used in adding parse expressions to compose an And expression. Literal('start') + ... + Literal('end') And(['start', ..., 'end']) are both equivalent to: Literal('start') + SkipTo('end')("_skipped*") + Literal('end') The '...' form has the added benefit of not requiring repeating the skip target expression. Note that the skipped text is returned with '_skipped' as a results name, and that the contents of `_skipped` will contain a list of text from all `...`s in the expression. - '...' can also be used as a "skip forward in case of error" expression: expr = "start" + (Word(nums).setName("int") | ...) + "end" expr.parseString("start 456 end") ['start', '456', 'end'] expr.parseString("start 456 foo 789 end") ['start', '456', 'foo 789 ', 'end'] - _skipped: ['foo 789 '] expr.parseString("start foo end") ['start', 'foo ', 'end'] - _skipped: ['foo '] expr.parseString("start end") ['start', '', 'end'] - _skipped: ['missing <int>'] Note that in all the error cases, the '_skipped' results name is present, showing a list of the extra or missing items. This form is only valid when used with the '|' operator. - Improved exception messages to show what was actually found, not just what was expected. word = pp.Word(pp.alphas) pp.OneOrMore(word).parseString("aaa bbb 123", parseAll=True) Former exception message: pyparsing.ParseException: Expected end of text (at char 8), (line:1, col:9) New exception message: pyparsing.ParseException: Expected end of text, found '1' (at char 8), (line:1, col:9) - Added diagnostic switches to help detect and warn about common parser construction mistakes, or enable additional parse debugging. Switches are attached to the pyparsing.__diag__ namespace object: - warn_multiple_tokens_in_named_alternation - flag to enable warnings when a results name is defined on a MatchFirst or Or expression with one or more And subexpressions (default=True) - warn_ungrouped_named_tokens_in_collection - flag to enable warnings when a results name is defined on a containing expression with ungrouped subexpressions that also have results names (default=True) - warn_name_set_on_empty_Forward - flag to enable warnings whan a Forward is defined with a results name, but has no contents defined (default=False) - warn_on_multiple_string_args_to_oneof - flag to enable warnings whan oneOf is incorrectly called with multiple str arguments (default=True) - enable_debug_on_named_expressions - flag to auto-enable debug on all subsequent calls to ParserElement.setName() (default=False) warn_multiple_tokens_in_named_alternation is intended to help those who currently have set __compat__.collect_all_And_tokens to False as a workaround for using the pre-2.3.1 code with named MatchFirst or Or expressions containing an And expression. - Added ParseResults.from_dict classmethod, to simplify creation of a ParseResults with results names using a dict, which may be nested. This makes it easy to add a sub-level of named items to the parsed tokens in a parse action. - Added asKeyword argument (default=False) to oneOf, to force keyword-style matching on the generated expressions. - ParserElement.runTests now accepts an optional 'file' argument to redirect test output to a file-like object (such as a StringIO, or opened file). Default is to write to sys.stdout. - conditionAsParseAction is a helper method for constructing a parse action method from a predicate function that simply returns a boolean result. Useful for those places where a predicate cannot be added using addCondition, but must be converted to a parse action (such as in infixNotation). May be used as a decorator if default message and exception types can be used. See ParserElement.addCondition for more details about the expected signature and behavior for predicate condition methods. - While investigating issue 93, I found that Or and addCondition could interact to select an alternative that is not the longest match. This is because Or first checks all alternatives for matches without running attached parse actions or conditions, orders by longest match, and then rechecks for matches with conditions and parse actions. Some expressions, when checking with conditions, may end up matching on a shorter token list than originally matched, but would be selected because of its original priority. This matching code has been expanded to do more extensive searching for matches when a second-pass check matches a smaller list than in the first pass. - Fixed issue 87, a regression in indented block. Reported by Renz Bagaporo, who submitted a very nice repro example, which makes the bug-fixing process a lot easier, thanks! - Fixed MemoryError issue 85 and 91 with str generation for Forwards. Thanks decalage2 and Harmon758 for your patience. - Modified setParseAction to accept None as an argument, indicating that all previously-defined parse actions for the expression should be cleared. - Modified pyparsing_common.real and sci_real to parse reals without leading integer digits before the decimal point, consistent with Python real number formats. Original PR 98 submitted by ansobolev. - Modified runTests to call postParse function before dumping out the parsed results - allows for postParse to add further results, such as indications of additional validation success/failure. - Updated statemachine example: refactored state transitions to use overridden classmethods; added <statename>Mixin class to simplify definition of application classes that "own" the state object and delegate to it to model state-specific properties and behavior. - Added example nested_markup.py, showing a simple wiki markup with nested markup directives, and illustrating the use of '...' for skipping over input to match the next expression. (This example uses syntax that is not valid under Python 2.) - Rewrote delta_time.py example (renamed from deltaTime.py) to fix some omitted formats and upgrade to latest pyparsing idioms, beginning with writing an actual BNF. - With the help and encouragement from several contributors, including Matěj Cepl and Cengiz Kaygusuz, I've started cleaning up the internal coding styles in core pyparsing, bringing it up to modern coding practices from pyparsing's early development days dating back to 2003. Whitespace has been largely standardized along PEP8 guidelines, removing extra spaces around parentheses, and adding them around arithmetic operators and after colons and commas. I was going to hold off on doing this work until after 2.4.1, but after cleaning up a few trial classes, the difference was so significant that I continued on to the rest of the core code base. This should facilitate future work and submitted PRs, allowing them to focus on substantive code changes, and not get sidetracked by whitespace issues. ``` ### 2.4.0 ``` --------------------------- - Well, it looks like the API change that was introduced in 2.3.1 was more drastic than expected, so for a friendlier forward upgrade path, this release: . Bumps the current version number to 2.4.0, to reflect this incompatible change. . Adds a pyparsing.__compat__ object for specifying compatibility with future breaking changes. . Conditionalizes the API-breaking behavior, based on the value pyparsing.__compat__.collect_all_And_tokens. By default, this value will be set to True, reflecting the new bugfixed behavior. To set this value to False, add to your code: import pyparsing pyparsing.__compat__.collect_all_And_tokens = False . User code that is dependent on the pre-bugfix behavior can restore it by setting this value to False. In 2.5 and later versions, the conditional code will be removed and setting the flag to True or False in these later versions will have no effect. - Updated unitTests.py and simple_unit_tests.py to be compatible with "python setup.py test". To run tests using setup, do: python setup.py test python setup.py test -s unitTests.suite python setup.py test -s simple_unit_tests.suite Prompted by issue 83 and PR submitted by bdragon28, thanks. - Fixed bug in runTests handling '\n' literals in quoted strings. - Added tag_body attribute to the start tag expressions generated by makeHTMLTags, so that you can avoid using SkipTo to roll your own tag body expression: a, aEnd = pp.makeHTMLTags('a') link = a + a.tag_body("displayed_text") + aEnd for t in s.searchString(html_page): print(t.displayed_text, '->', t.startA.href) - indentedBlock failure handling was improved; PR submitted by TMiguelT, thanks! - Address Py2 incompatibility in simpleUnitTests, plus explain() and Forward str() cleanup; PRs graciously provided by eswald. - Fixed docstring with embedded '\w', which creates SyntaxWarnings in Py3.8, issue 80. - Examples: - Added example parser for rosettacode.org tutorial compiler. - Added example to show how an HTML table can be parsed into a collection of Python lists or dicts, one per row. - Updated SimpleSQL.py example to handle nested selects, reworked '