Changelog
### 1.3.19
```
:released: August 17, 2020
.. change::
:tags: usecase, py3k
:tickets: 5357
Added a ``**kw`` argument to the :meth:`.DeclarativeMeta.__init__` method.
This allows a class to support the :pep:`487` metaclass hook
``__init_subclass__``. Pull request courtesy Ewen Gillies.
.. change::
:tags: bug, sql
:tickets: 5470
Repaired an issue where the "ORDER BY" clause rendering a label name rather
than a complete expression, which is particularly important for SQL Server,
would fail to occur if the expression were enclosed in a parenthesized
grouping in some cases. This case has been added to test support. The
change additionally adjusts the "automatically add ORDER BY columns when
DISTINCT is present" behavior of ORM query, deprecated in 1.4, to more
accurately detect column expressions that are already present.
.. change::
:tags: usecase, mysql
:tickets: 5481
The MySQL dialect will render FROM DUAL for a SELECT statement that has no
FROM clause but has a WHERE clause. This allows things like "SELECT 1 WHERE
EXISTS (subquery)" kinds of queries to be used as well as other use cases.
.. change::
:tags: bug, mssql, sql
:tickets: 5467
Fixed bug where the mssql dialect incorrectly escaped object names that
contained ']' character(s).
.. change::
:tags: bug, reflection, sqlite, mssql
:tickets: 5456
Applied a sweep through all included dialects to ensure names that contain
single or double quotes are properly escaped when querying system tables,
for all :class:`.Inspector` methods that accept object names as an argument
(e.g. table names, view names, etc). SQLite and MSSQL contained two
quoting issues that were repaired.
.. change::
:tags: bug, mysql
:tickets: 5411
Fixed an issue where CREATE TABLE statements were not specifying the
COLLATE keyword correctly.
.. change::
:tags: bug, datatypes, sql
:tickets: 4733
The ``LookupError`` message will now provide the user with up to four
possible values that a column is constrained to via the :class:`.Enum`.
Values longer than 11 characters will be truncated and replaced with
ellipses. Pull request courtesy Ramon Williams.
.. change::
:tags: bug, postgresql
:tickets: 5476
Fixed issue where the return type for the various RANGE comparison
operators would itself be the same RANGE type rather than BOOLEAN, which
would cause an undesirable result in the case that a
:class:`.TypeDecorator` that defined result-processing behavior were in
use. Pull request courtesy Jim Bosch.
.. change::
:tags: bug, mysql
:tickets: 5493
Added MariaDB code 1927 to the list of "disconnect" codes, as recent
MariaDB versions apparently use this code when the database server was
stopped.
.. change::
:tags: usecase, declarative, orm
:tickets: 5513
The name of the virtual column used when using the
:class:`_declarative.AbstractConcreteBase` and
:class:`_declarative.ConcreteBase` classes can now be customized, to allow
for models that have a column that is actually named ``type``. Pull
request courtesy Jesse-Bakker.
.. change::
:tags: usecase, orm
:tickets: 5494
Adjusted the workings of the :meth:`_orm.Mapper.all_orm_descriptors`
accessor to represent the attributes in the order that they are located in
a deterministic way, assuming the use of Python 3.6 or higher which
maintains the sorting order of class attributes based on how they were
declared. This sorting is not guaranteed to match the declared order of
attributes in all cases however; see the method documentation for the exact
scheme.
.. change::
:tags: bug, sql
:tickets: 5500
Fixed issue where the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would not take effect when the :meth:`_schema.Sequence.next_value`
function function for a :class:`_schema.Sequence` were used in the
:paramref:`_schema.Column.server_default` parameter and the create table
DDL were emitted.
.. changelog::
```
### 1.3.18
```
:released: June 25, 2020
.. change::
:tags: bug, sqlite
:tickets: 5395
Added "exists" to the list of reserved words for SQLite so that this word
will be quoted when used as a label or column name. Pull request courtesy
Thodoris Sotiropoulos.
.. change::
:tags: bug, mssql
:tickets: 5366, 5364
Refined the logic used by the SQL Server dialect to interpret multi-part
schema names that contain many dots, to not actually lose any dots if the
name does not have bracking or quoting used, and additionally to support a
"dbname" token that has many parts including that it may have multiple,
independently-bracketed sections.
.. change::
:tags: bug, mssql, pyodbc
:tickets: 5346
Fixed an issue in the pyodbc connector such that a warning about pyodbc
"drivername" would be emitted when using a totally empty URL. Empty URLs
are normal when producing a non-connected dialect object or when using the
"creator" argument to create_engine(). The warning now only emits if the
driver name is missing but other parameters are still present.
.. change::
:tags: bug, mssql
:tickets: 5373
Fixed issue with assembling the ODBC connection string for the pyodbc
DBAPI. Tokens containing semicolons and/or braces "{}" were not being
correctly escaped, causing the ODBC driver to misinterpret the
connection string attributes.
.. change::
:tags: usecase, orm
:tickets: 5326
Improve error message when using :meth:`_query.Query.filter_by` in
a query where the first entity is not a mapped class.
.. change::
:tags: sql, schema
:tickets: 5324
Introduce :class:`.IdentityOptions` to store common parameters for
sequences and identity columns.
.. change::
:tags: usecase, sql
:tickets: 5309
Added a ".schema" parameter to the :func:`_expression.table` construct,
allowing ad-hoc table expressions to also include a schema name.
Pull request courtesy Dylan Modesitt.
.. change::
:tags: bug, mssql
:tickets: 5339
Fixed issue where ``datetime.time`` parameters were being converted to
``datetime.datetime``, making them incompatible with comparisons like
``>=`` against an actual :class:`_mssql.TIME` column.
.. change::
:tags: bug, mssql
:tickets: 5359
Fixed an issue where the ``is_disconnect`` function in the SQL Server
pyodbc dialect was incorrectly reporting the disconnect state when the
exception messsage had a substring that matched a SQL Server ODBC error
code.
.. change::
:tags: bug, engine
:tickets: 5326
Further refinements to the fixes to the "reset" agent fixed in
:ticket:`5326`, which now emits a warning when it is not being correctly
invoked and corrects for the behavior. Additional scenarios have been
identified and fixed where this warning was being emitted.
.. change::
:tags: usecase, sqlite
:tickets: 5297
SQLite 3.31 added support for computed column. This change
enables their support in SQLAlchemy when targeting SQLite.
.. change::
:tags: bug, schema
:tickets: 5276
Fixed issue where ``dialect_options`` were omitted when a
database object (e.g., :class:`.Table`) was copied using
:func:`.tometadata`.
.. change::
:tags: bug, sql
:tickets: 5344
Correctly apply self_group in type_coerce element.
The type coerce element did not correctly apply grouping rules when using
in an expression
.. change::
:tags: bug, oracle, reflection
:tickets: 5421
Fixed bug in Oracle dialect where indexes that contain the full set of
primary key columns would be mistaken as the primary key index itself,
which is omitted, even if there were multiples. The check has been refined
to compare the name of the primary key constraint against the index name
itself, rather than trying to guess based on the columns present in the
index.
.. change::
:tags: change, sql, sybase
:tickets: 5294
Added ``.offset`` support to sybase dialect.
Pull request courtesy Alan D. Snow.
.. change::
:tags: bug, engine
:tickets: 5341
Fixed issue in :class:`.URL` object where stringifying the object
would not URL encode special characters, preventing the URL from being
re-consumable as a real URL. Pull request courtesy Miguel Grinberg.
.. change::
:tags: usecase, mysql
:tickets: 4860
Implemented row-level locking support for mysql. Pull request courtesy
Quentin Somerville.
.. change::
:tags: change, mssql
:tickets: 5321
Moved the ``supports_sane_rowcount_returning = False`` requirement from
the ``PyODBCConnector`` level to the ``MSDialect_pyodbc`` since pyodbc
does work properly in some circumstances.
.. change::
:tags: change, examples
Added new option ``--raw`` to the examples.performance suite
which will dump the raw profile test for consumption by any
number of profiling visualizer tools. Removed the "runsnake"
option as runsnake is very hard to build at this point;
.. change::
:tags: bug, sql
:tickets: 5353
Added :meth:`.Select.with_hint` output to the generic SQL string that is
produced when calling ``str()`` on a statement. Previously, this clause
would be omitted under the assumption that it was dialect specific.
The hint text is presented within brackets to indicate the rendering
of such hints varies among backends.
.. change::
:tags: usecase, orm
:tickets: 5198
Added a new parameter :paramref:`_orm.query_expression.default_expr` to the
:func:`_orm.query_expression` construct, which will be appled to queries
automatically if the :func:`_orm.with_expression` option is not used. Pull
request courtesy Haoyu Sun.
.. changelog::
```
### 1.3.17
```
:released: May 13, 2020
.. change::
:tags: bug, oracle
:tickets: 5246
Some modifications to how the cx_oracle dialect sets up per-column
outputtype handlers for LOB and numeric datatypes to adjust for potential
changes coming in cx_Oracle 8.
.. change::
:tags: bug, orm
:tickets: 5288
Fixed bug where using :func:`.with_polymorphic` as the target of a join via
:meth:`.RelationshipComparator.of_type` on a mapper that already has a
subquery-based with_polymorphic setting that's equivalent to the one
requested would not correctly alias the ON clause in the join.
.. change::
:tags: bug, oracle, performance
:tickets: 5314
Changed the implementation of fetching CLOB and BLOB objects to use
cx_Oracle's native implementation which fetches CLOB/BLOB objects inline
with other result columns, rather than performing a separate fetch. As
always, this can be disabled by setting auto_convert_lobs to False.
As part of this change, the behavior of a CLOB that was given a blank
string on INSERT now returns None on SELECT, which is now consistent with
that of VARCHAR on Oracle.
.. change::
:tags: usecase, postgresql
:tickets: 5265
Added support for columns or type :class:`.ARRAY` of :class:`.Enum`,
:class:`.JSON` or :class:`_postgresql.JSONB` in PostgreSQL.
Previously a workaround was required in these use cases.
.. change::
:tags: schema
:tickets: 4138
Add ``comment`` attribute to :class:`_schema.Column` ``__repr__`` method.
.. change::
:tags: bug, orm
:tickets: 5303
Fixed issue in the area of where loader options such as selectinload()
interact with the baked query system, such that the caching of a query is
not supposed to occur if the loader options themselves have elements such
as with_polymorphic() objects in them that currently are not
cache-compatible. The baked loader could sometimes not fully invalidate
itself in these some of these scenarios leading to missed eager loads.
.. change::
:tags: bug, engine
:tickets: 5326
Fixed fairly critical issue where the DBAPI connection could be returned to
the connection pool while still in an un-rolled-back state. The reset agent
responsible for rolling back the connection could be corrupted in the case
that the transaction was "closed" without being rolled back or committed,
which can occur in some scenarios when using ORM sessions and emitting
.close() in a certain pattern involving savepoints. The fix ensures that
the reset agent is always active.
.. change::
:tags: bug, orm
:tickets: 5304
Modified the internal "identity set" implementation, which is a set that
hashes objects on their id() rather than their hash values, to not actually
call the ``__hash__()`` method of the objects, which are typically
user-mapped objects. Some methods were calling this method as a side
effect of the implementation.
.. change::
:tags: usecase, postgresql
:tickets: 5266
Raise an explicit :class:`.exc.CompileError` when adding a table with a
column of type :class:`.ARRAY` of :class:`.Enum` configured with
:paramref:`.Enum.native_enum` set to ``False`` when
:paramref:`.Enum.create_constraint` is not set to ``False``
.. change::
:tags: bug, schema
:tickets: 5298
Fixed issue where an :class:`.Index` that is deferred in being associated
with a table, such as as when it contains a :class:`.Column` that is not
associated with any :class:`.Table` yet, would fail to attach correctly if
it also contained a non table-oriented expression.
.. change::
:tags: change, firebird
:tickets: 5278
Adjusted dialect loading for ``firebird://`` URIs so the external
sqlalchemy-firebird dialect will be used if it has been installed,
otherwise fall back to the (now deprecated) internal Firebird dialect.
.. change::
:tags: bug, mssql, reflection
:tickets: 5255
Fix a regression introduced by the reflection of computed column in
MSSQL when using the legacy TDS version 4.2. The dialect will try
to detect the protocol version of first connect and run in compatibility
mode if it cannot detect it.
.. change::
:tags: bug, mssql, reflection
:tickets: 5271
Fix a regression introduced by the reflection of computed column in
MSSQL when using SQL server versions before 2012, which does not support
the ``concat`` function.
.. change::
:tags: bug, orm
:tickets: 5269
An informative error message is raised when an ORM many-to-one comparison
is attempted against an object that is not an actual mapped instance.
Comparisons such as those to scalar subqueries aren't supported;
generalized comparison with subqueries is better achieved using
:meth:`~.RelationshipProperty.Comparator.has`.
.. change::
:tags: usecase, orm
:tickets: 5262
Added an accessor :attr:`.ColumnProperty.Comparator.expressions` which
provides access to the group of columns mapped under a multi-column
:class:`.ColumnProperty` attribute.
.. change::
:tags: bug, schema
:tickets: 5316
A warning is emitted when making use of the :attr:`.MetaData.sorted_tables`
attribute as well as the :func:`_schema.sort_tables` function, and the
given tables cannot be correctly sorted due to a cyclic dependency between
foreign key constraints. In this case, the functions will no longer sort
the involved tables by foreign key, and a warning will be emitted. Other
tables that are not part of the cycle will still be returned in dependency
order. Previously, the sorted_table routines would return a collection that
would unconditionally omit all foreign keys when a cycle was detected, and
no warning was emitted.
.. change::
:tags: orm, usecase
:tickets: 5237
Introduce :paramref:`_orm.relationship.sync_backref` flag in a relationship
to control if the synchronization events that mutate the in-Python
attributes are added. This supersedes the previous change :ticket:`5149`,
which warned that ``viewonly=True`` relationship target of a
back_populates or backref configuration would be disallowed.
.. changelog::
```
### 1.3.16
```
:released: April 7, 2020
.. change::
:tags: oracle, usecase
:tickets: 5200
Implemented AUTOCOMMIT isolation level for Oracle when using cx_Oracle.
Also added a fixed default isolation level of READ COMMITTED for Oracle.
.. change::
:tags: bug, mysql
:tickets: 5239
Fixed issue in MySQL dialect when connecting to a pseudo-MySQL database
such as that provided by ProxySQL, the up front check for isolation level
when it returns no row will not prevent the dialect from continuing to
connect. A warning is emitted that the isolation level could not be
detected.
.. change::
:tags: bug, tests
:tickets: 5201
Fixed an issue that prevented the test suite from running with the
recently released py.test 5.4.0.
.. change::
:tags: bug, oracle, reflection
:tickets: 5146
Fixed regression / incorrect fix caused by fix for :ticket:`5146` where the
Oracle dialect reads from the "all_tab_comments" view to get table comments
but fails to accommodate for the current owner of the table being
requested, causing it to read the wrong comment if multiple tables of the
same name exist in multiple schemas.
.. change::
:tags: types, enum
:tickets: 5183
The :class:`.Enum` type now supports the parameter :paramref:`.Enum.length`
to specify the length of the VARCHAR column to create when using
non native enums by setting :paramref:`.Enum.native_enum` to ``False``
.. change::
:tags: bug, orm
:tickets: 5228
Fixed bug in :func:`_orm.selectinload` loading option where two or more
loaders that represent different relationships with the same string key
name as referenced from a single :func:`_orm.with_polymorphic` construct
with multiple subclass mappers would fail to invoke each subqueryload
separately, instead making use of a single string-based slot that would
prevent the other loaders from being invoked.
.. change::
:tags: schema, reflection
:tickets: 5063
Added support for reflection of "computed" columns, which are now returned
as part of the structure returned by :meth:`_reflection.Inspector.get_columns`.
When reflecting full :class:`_schema.Table` objects, computed columns will
be represented using the :class:`.Computed` construct.
.. change::
:tags: orm, performance
:tickets: 5162
Modified the queries used by subqueryload and selectinload to no longer
ORDER BY the primary key of the parent entity; this ordering was there to
allow the rows as they come in to be copied into lists directly with a
minimal level of Python-side collation. However, these ORDER BY clauses
can negatively impact the performance of the query as in many scenarios
these columns are derived from a subquery or are otherwise not actual
primary key columns such that SQL planners cannot make use of indexes. The
Python-side collation uses the native itertools.group_by() to collate the
incoming rows, and has been modified to allow multiple
row-groups-per-parent to be assembled together using list.extend(), which
should still allow for relatively fast Python-side performance. There will
still be an ORDER BY present for a relationship that includes an explicit
order_by parameter, however this is the only ORDER BY that will be added to
the query for both kinds of loading.
.. change::
:tags: mssql, mysql, oracle, usecase
:tickets: 5137
Added support for :meth:`.ColumnOperators.is_distinct_from` and
:meth:`.ColumnOperators.isnot_distinct_from` to SQL Server,
MySQL, and Oracle.
.. change::
:tags: sqlite, usecase
:tickets: 5164
Implemented AUTOCOMMIT isolation level for SQLite when using pysqlite.
.. change::
:tags: bug, postgresql
:tickets: 5205
Fixed issue where a "covering" index, e.g. those which have an INCLUDE
clause, would be reflected including all the columns in INCLUDE clause as
regular columns. A warning is now emitted if these additional columns are
detected indicating that they are currently ignored. Note that full
support for "covering" indexes is part of :ticket:`4458`. Pull request
courtesy Marat Sharafutdinov.
.. change::
:tags: sql, types
:tickets: 5052
Add ability to literal compile a :class:`DateTime`, :class:`Date`
or :class:`Time` when using the string dialect for debugging purposes.
This change does not impact real dialect implementation that retain
their current behavior.
.. change::
:tags: installer
:tickets: 5207
Ensured that the "pyproject.toml" file is not included in builds, as the
presence of this file indicates to pip that a pep-517 installation process
should be used. As this mode of operation appears to be not well supported
by current tools / distros, these problems are avoided within the scope
of SQLAlchemy installation by omitting the file.
.. change::
:tags: bug, orm
:tickets: 5210
Fixed issue where a lazyload that uses session-local "get" against a target
many-to-one relationship where an object with the correct primary key is
present, however it's an instance of a sibling class, does not correctly
return None as is the case when the lazy loader actually emits a load for
that row.
.. change::
:tags: bug, orm, declarative
:tickets: 5238
The string argument accepted as the first positional argument by the
:func:`_orm.relationship` function when using the Declarative API is no longer
interpreted using the Python ``eval()`` function; instead, the name is dot
separated and the names are looked up directly in the name resolution
dictionary without treating the value as a Python expression. However,
passing a string argument to the other :func:`_orm.relationship` parameters
that necessarily must accept Python expressions will still use ``eval()``;
the documentation has been clarified to ensure that there is no ambiguity
that this is in use.
.. seealso::
:ref:`declarative_relationship_eval` - details on string evaluation
.. changelog::
```
### 1.3.15
```
:released: March 11, 2020
.. change::
:tags: bug, orm
:tickets: 5194
Adjusted the error message emitted by :meth:`_query.Query.join` when a left hand
side can't be located that the :meth:`_query.Query.select_from` method is the
best way to resolve the issue. Also, within the 1.3 series, used a
deterministic ordering when determining the FROM clause from a given column
entity passed to :class:`_query.Query` so that the same expression is determined
each time.
.. change::
:tags: orm, bug
:tickets: 5196
Fixed regression in 1.3.14 due to :ticket:`4849` where a sys.exc_info()
call failed to be invoked correctly when a flush error would occur. Test
coverage has been added for this exception case.
.. changelog::
```
### 1.3.14
```
:released: March 10, 2020
.. change::
:tags: bug, sql, postgresql
:tickets: 5181
Fixed bug where a CTE of an INSERT/UPDATE/DELETE that also uses RETURNING
could then not be SELECTed from directly, as the internal state of the
compiler would try to treat the outer SELECT as a DELETE statement itself
and access nonexistent state.
.. change::
:tags: bug, orm
:tickets: 5110
Fixed regression caused in 1.3.13 by :ticket:`5056` where a refactor of the
ORM path registry system made it such that a path could no longer be
compared to an empty tuple, which can occur in a particular kind of joined
eager loading path. The "empty tuple" use case has been resolved so that
the path registry is compared to a path registry in all cases; the
:class:`.PathRegistry` object itself now implements ``__eq__()`` and
``__ne__()`` methods which will take place for all equality comparisons and
continue to succeed in the not anticipated case that a non-
:class:`.PathRegistry` object is compared, while emitting a warning that
this object should not be the subject of the comparison.
.. change::
:tags: bug, orm
:tickets: 5149
Setting a relationship to viewonly=True which is also the target of a
back_populates or backref configuration will now emit a warning and
eventually be disallowed. back_populates refers specifically to mutation
of an attribute or collection, which is disallowed when the attribute is
subject to viewonly=True. The viewonly attribute is not subject to
persistence behaviors which means it will not reflect correct results
when it is locally mutated.
.. change::
:tags: bug, oracle
:tickets: 5146
Fixed a reflection bug where table comments could only be retrieved for
tables actually owned by the user but not for tables visible to the user
but owned by someone else. Pull request courtesy Dave Hirschfeld.
.. change::
:tags: bug, performance
:tickets: 5180
Revised an internal change to the test system added as a result of
:ticket:`5085` where a testing-related module per dialect would be loaded
unconditionally upon making use of that dialect, pulling in SQLAlchemy's
testing framework as well as the ORM into the module import space. This
would only impact initial startup time and memory to a modest extent,
however it's best that these additional modules aren't reverse-dependent on
straight Core usage.
.. change::
:tags: bug, installation
:tickets: 5138
Vendored the ``inspect.formatannotation`` function inside of
``sqlalchemy.util.compat``, which is needed for the vendored version of
``inspect.formatargspec``. The function is not documented in cPython and
is not guaranteed to be available in future Python versions.
.. change::
:tags: bug, mssql
:tickets: 5132
Fixed issue where the :class:`_mssql.DATETIMEOFFSET` type would not
accommodate for the ``None`` value, introduced as part of the series of
fixes for this type first introduced in :ticket:`4983`, :ticket:`5045`.
Additionally, added support for passing a backend-specific date formatted
string through this type, as is typically allowed for date/time types on
most other DBAPIs.
.. change::
:tags: bug, engine
:tickets: 5182
Expanded the scope of cursor/connection cleanup when a statement is
executed to include when the result object fails to be constructed, or an
after_cursor_execute() event raises an error, or autocommit / autoclose
fails. This allows the DBAPI cursor to be cleaned up on failure and for
connectionless execution allows the connection to be closed out and
returned to the connection pool, where previously it waiting until garbage
collection would trigger a pool return.
.. change::
:tags: bug, postgresql
:tickets: 5158
Fixed issue where the "schema_translate_map" feature would not work with a
PostgreSQL native enumeration type (i.e. :class:`.Enum`,
:class:`_postgresql.ENUM`) in that while the "CREATE TYPE" statement would
be emitted with the correct schema, the schema would not be rendered in
the CREATE TABLE statement at the point at which the enumeration was
referenced.
.. change::
:tags: usecase, ext
:tickets: 5114
Added keyword arguments to the :meth:`.MutableList.sort` function so that a
key function as well as the "reverse" keyword argument can be provided.
.. change::
:tags: bug, general, py3k
:tickets: 4849
Applied an explicit "cause" to most if not all internally raised exceptions
that are raised from within an internal exception catch, to avoid
misleading stacktraces that suggest an error within the handling of an
exception. While it would be preferable to suppress the internally caught
exception in the way that the ``__suppress_context__`` attribute would,
there does not as yet seem to be a way to do this without suppressing an
enclosing user constructed context, so for now it exposes the internally
caught exception as the cause so that full information about the context
of the error is maintained.
.. change::
:tags: orm, bug
:tickets: 5121
Fixed an additional regression in the same area as that of :ticket:`5080`
introduced in 1.3.0b3 via :ticket:`4468` where the ability to create a
joined option across a :func:`.with_polymorphic` into a relationship
against the base class of that with_polymorphic, and then further into
regular mapped relationships would fail as the base class component would
not add itself to the load path in a way that could be located by the
loader strategy. The changes applied in :ticket:`5080` have been further
refined to also accommodate this scenario.
.. change::
:tags: bug, postgresql, reflection
:tickets: 5170
Fixed bug where PostgreSQL reflection of CHECK constraints would fail to
parse the constraint if the SQL text contained newline characters. The
regular expression has been adjusted to accommodate for this case. Pull
request courtesy Eric Borczuk.
.. change::
:tags: usecase, orm
:tickets: 5129
Added a new flag :paramref:`.InstanceEvents.restore_load_context` and
:paramref:`.SessionEvents.restore_load_context` which apply to the
:meth:`.InstanceEvents.load`, :meth:`.InstanceEvents.refresh`, and
:meth:`.SessionEvents.loaded_as_persistent` events, which when set will
restore the "load context" of the object after the event hook has been
called. This ensures that the object remains within the "loader context"
of the load operation that is already ongoing, rather than the object being
transferred to a new load context due to refresh operations which may have
occurred in the event. A warning is now emitted when this condition occurs,
which recommends use of the flag to resolve this case. The flag is
"opt-in" so that there is no risk introduced to existing applications.
The change additionally adds support for the ``raw=True`` flag to
session lifecycle events.
.. change::
:tags: bug, mysql
:tickets: 5173
Fixed issue in MySQL :meth:`.mysql.Insert.on_duplicate_key_update` construct
where using a SQL function or other composed expression for a column argument
would not properly render the ``VALUES`` keyword surrounding the column
itself.
.. changelog::
```
### 1.3.13
```
:released: January 22, 2020
.. change::
:tags: bug, postgresql
:tickets: 5039
Fixed issue where the PostgreSQL dialect would fail to parse a reflected
CHECK constraint that was a boolean-valued function (as opposed to a
boolean-valued expression).
.. change::
:tags: bug, ext
:tickets: 5086
Fixed bug in sqlalchemy.ext.serializer where a unique
:class:`.BindParameter` object could conflict with itself if it were
present in the mapping itself, as well as the filter condition of the
query, as one side would be used against the non-deserialized version and
the other side would use the deserialized version. Logic is added to
:class:`.BindParameter` similar to its "clone" method which will uniquify
the parameter name upon deserialize so that it doesn't conflict with its
original.
.. change::
:tags: usecase, sql
:tickets: 5079
A function created using :class:`.GenericFunction` can now specify that the
name of the function should be rendered with or without quotes by assigning
the :class:`.quoted_name` construct to the .name element of the object.
Prior to 1.3.4, quoting was never applied to function names, and some
quoting was introduced in :ticket:`4467` but no means to force quoting for
a mixed case name was available. Additionally, the :class:`.quoted_name`
construct when used as the name will properly register its lowercase name
in the function registry so that the name continues to be available via the
``func.`` registry.
.. seealso::
:class:`.GenericFunction`
.. change::
:tags: bug, engine
:tickets: 5048
Fixed issue where the collection of value processors on a
:class:`.Compiled` object would be mutated when "expanding IN" parameters
were used with a datatype that has bind value processors; in particular,
this would mean that when using statement caching and/or baked queries, the
same compiled._bind_processors collection would be mutated concurrently.
Since these processors are the same function for a given bind parameter
namespace every time, there was no actual negative effect of this issue,
however, the execution of a :class:`.Compiled` object should never be
causing any changes in its state, especially given that they are intended
to be thread-safe and reusable once fully constructed.
.. change::
:tags: tests, postgresql
:tickets: 5057
Improved detection of two phase transactions requirement for the PostgreSQL
database by testing that max_prepared_transactions is set to a value
greater than 0. Pull request courtesy Federico Caselli.
.. change::
:tags: bug, orm, engine
:tickets: 5056, 5050, 5071
Added test support and repaired a wide variety of unnecessary reference
cycles created for short-lived objects, mostly in the area of ORM queries.
Thanks much to Carson Ip for the help on this.
.. change::
:tags: orm, bug
:tickets: 5107
Fixed regression in loader options introduced in 1.3.0b3 via :ticket:`4468`
where the ability to create a loader option using
:meth:`.PropComparator.of_type` targeting an aliased entity that is an
inheriting subclass of the entity which the preceding relationship refers
to would fail to produce a matching path. See also :ticket:`5082` fixed
in this same release which involves a similar kind of issue.
.. change::
:tags: bug, tests
:tickets: 4946
Fixed a few test failures which would occur on Windows due to SQLite file
locking issues, as well as some timing issues in connection pool related
tests; pull request courtesy Federico Caselli.
.. change::
:tags: orm, bug
:tickets: 5082
Fixed regression in joined eager loading introduced in 1.3.0b3 via
:ticket:`4468` where the ability to create a joined option across a
:func:`.with_polymorphic` into a polymorphic subclass using
:meth:`.RelationshipProperty.of_type` and then further along regular mapped
relationships would fail as the polymorphic subclass would not add itself
to the load path in a way that could be located by the loader strategy. A
tweak has been made to resolve this scenario.
.. change::
:tags: performance, orm
Identified a performance issue in the system by which a join is constructed
based on a mapped relationship. The clause adaption system would be used
for the majority of join expressions including in the common case where no
adaptation is needed. The conditions under which this adaptation occur
have been refined so that average non-aliased joins along a simple
relationship without a "secondary" table use about 70% less function calls.
.. change::
:tags: usecase, postgresql
:tickets: 5040
Added support for prefixes to the :class:`_expression.CTE` construct, to allow
support for Postgresql 12 "MATERIALIZED" and "NOT MATERIALIZED" phrases.
Pull request courtesy Marat Sharafutdinov.
.. seealso::
:meth:`_expression.HasCTE.cte`
.. change::
:tags: bug, mssql
:tickets: 5045
Fixed issue where a timezone-aware ``datetime`` value being converted to
string for use as a parameter value of a :class:`_mssql.DATETIMEOFFSET`
column was omitting the fractional seconds.
.. change::
:tags: bug, orm
:tickets: 5068
Repaired a warning in the ORM flush process that was not covered by test
coverage when deleting objects that use the "version_id" feature. This
warning is generally unreachable unless using a dialect that sets the
"supports_sane_rowcount" flag to False, which is not typically the case
however is possible for some MySQL configurations as well as older Firebird
drivers, and likely some third party dialects.
.. change::
:tags: bug, orm
:tickets: 5065
Fixed bug where usage of joined eager loading would not properly wrap the
query inside of a subquery when :meth:`_query.Query.group_by` were used against
the query. When any kind of result-limiting approach is used, such as
DISTINCT, LIMIT, OFFSET, joined eager loading embeds the row-limited query
inside of a subquery so that the collection results are not impacted. For
some reason, the presence of GROUP BY was never included in this criterion,
even though it has a similar effect as using DISTINCT. Additionally, the
bug would prevent using GROUP BY at all for a joined eager load query for
most database platforms which forbid non-aggregated, non-grouped columns
from being in the query, as the additional columns for the joined eager
load would not be accepted by the database.
.. changelog::
```
### 1.3.12
```
:released: December 16, 2019
.. change::
:tags: bug, sql
:tickets: 5028
Fixed bug where "distinct" keyword passed to :func:`_expression.select` would not
treat a string value as a "label reference" in the same way that the
:meth:`_expression.select.distinct` does; it would instead raise unconditionally. This
keyword argument and the others passed to :func:`_expression.select` will ultimately
be deprecated for SQLAlchemy 2.0.
.. change::
:tags: bug, orm
:tickets: 4997
Fixed issue involving ``lazy="raise"`` strategy where an ORM delete of an
object would raise for a simple "use-get" style many-to-one relationship
that had lazy="raise" configured. This is inconsistent vs. the change
introduced in 1.3 as part of :ticket:`4353`, where it was established that
a history operation that does not expect emit SQL should bypass the
``lazy="raise"`` check, and instead effectively treat it as
``lazy="raise_on_sql"`` for this case. The fix adjusts the lazy loader
strategy to not raise for the case where the lazy load was instructed that
it should not emit SQL if the object were not present.
.. change::
:tags: bug, sql
Changed the text of the exception for "Can't resolve label reference" to
include other kinds of label coercions, namely that "DISTINCT" is also in
this category under the PostgreSQL dialect.
.. change::
:tags: bug, orm
:tickets: 5000
Fixed regression introduced in 1.3.0 related to the association proxy
refactor in :ticket:`4351` that prevented :func:`.composite` attributes
from working in terms of an association proxy that references them.
.. change::
:tags: bug, mssql
:tickets: 4983
Repaired support for the :class:`_mssql.DATETIMEOFFSET` datatype on PyODBC,
by adding PyODBC-level result handlers as it does not include native
support for this datatype. This includes usage of the Python 3 "timezone"
tzinfo subclass in order to set up a timezone, which on Python 2 makes
use of a minimal backport of "timezone" in sqlalchemy.util.
.. change::
:tags: bug, orm
:tickets: 4993
Setting persistence-related flags on :func:`_orm.relationship` while also
setting viewonly=True will now emit a regular warning, as these flags do
not make sense for a viewonly=True relationship. In particular, the
"cascade" settings have their own warning that is generated based on the
individual values, such as "delete, delete-orphan", that should not apply
to a viewonly relationship. Note however that in the case of "cascade",
these settings are still erroneously taking effect even though the
relationship is set up as "viewonly". In 1.4, all persistence-related
cascade settings will be disallowed on a viewonly=True relationship in
order to resolve this issue.
.. change::
:tags: bug, sqlite
:tickets: 5014
Fixed issue to workaround SQLite's behavior of assigning "numeric" affinity
to JSON datatypes, first described at :ref:`change_3850`, which returns
scalar numeric JSON values as a number and not as a string that can be JSON
deserialized. The SQLite-specific JSON deserializer now gracefully
degrades for this case as an exception and bypasses deserialization for
single numeric values, as from a JSON perspective they are already
deserialized.
.. change::
:tags: bug, orm, py3k
:tickets: 4990
Fixed issue where when assigning a collection to itself as a slice, the
mutation operation would fail as it would first erase the assigned
collection inadvertently. As an assignment that does not change the
contents should not generate events, the operation is now a no-op. Note
that the fix only applies to Python 3; in Python 2, the ``__setitem__``
hook isn't called in this case; ``__setslice__`` is used instead which
recreates the list item-by-item in all cases.
.. change::
:tags: bug, orm
:tickets: 5034
Fixed issue where by if the "begin" of a transaction failed at the Core
engine/connection level, such as due to network error or database is locked
for some transactional recipes, within the context of the :class:`.Session`
procuring that connection from the connection pool and then immediately
returning it, the ORM :class:`.Session` would not close the connection
despite this connection not being stored within the state of that
:class:`.Session`. This would lead to the connection being cleaned out by
the connection pool weakref handler within garbage collection which is an
unpreferred codepath that in some special configurations can emit errors in
standard error.
.. changelog::
```
### 1.3.11
```
:released: November 11, 2019
.. change::
:tags: bug, mssql
:tickets: 4973
Fixed issue in MSSQL dialect where an expression-based OFFSET value in a
SELECT would be rejected, even though the dialect can render this
expression inside of a ROW NUMBER-oriented LIMIT/OFFSET construct.
.. change::
:tags: orm, usecase
:tickets: 4934
Added accessor :meth:`_query.Query.is_single_entity` to :class:`_query.Query`, which
will indicate if the results returned by this :class:`_query.Query` will be a
list of ORM entities, or a tuple of entities or column expressions.
SQLAlchemy hopes to improve upon the behavior of single entity / tuples in
future releases such that the behavior would be explicit up front, however
this attribute should be helpful with the current behavior. Pull request
courtesy Patrick Hayes.
.. change::
:tags: bug, mysql
:tickets: 4945
Added "Connection was killed" message interpreted from the base
pymysql.Error class in order to detect closed connection, based on reports
that this message is arriving via a pymysql.InternalError() object which
indicates pymysql is not handling it correctly.
.. change::
:tags: bug, orm
:tickets: 4954
The :paramref:`_orm.relationship.omit_join` flag was not intended to be
manually set to True, and will now emit a warning when this occurs. The
omit_join optimization is detected automatically, and the ``omit_join``
flag was only intended to disable the optimization in the hypothetical case
that the optimization may have interfered with correct results, which has
not been observed with the modern version of this feature. Setting the
flag to True when it is not automatically detected may cause the selectin
load feature to not work correctly when a non-default primary join
condition is in use.
.. change::
:tags: bug, orm
:tickets: 4915
A warning is emitted if a primary key value is passed to :meth:`_query.Query.get`
that consists of None for all primary key column positions. Previously,
passing a single None outside of a tuple would raise a ``TypeError`` and
passing a composite None (tuple of None values) would silently pass
through. The fix now coerces the single None into a tuple where it is
handled consistently with the other None conditions. Thanks to Lev
Izraelit for the help with this.
.. change::
:tags: bug, orm
:tickets: 4947
The :class:`.BakedQuery` will not cache a query that was modified by a
:meth:`.QueryEvents.before_compile` event, so that compilation hooks that
may be applying ad-hoc modifications to queries will take effect on each
run. In particular this is helpful for events that modify queries used in
lazy loading as well as eager loading such as "select in" loading. In
order to re-enable caching for a query modified by this event, a new
flag ``bake_ok`` is added; see :ref:`baked_with_before_compile` for
details.
A longer term plan to provide a new form of SQL caching should solve this
kind of issue more comprehensively.
.. change::
:tags: bug, tests
:tickets: 4920
Fixed test failures which would occur with newer SQLite as of version 3.30
or greater, due to their addition of nulls ordering syntax as well as new
restrictions on aggregate functions. Pull request courtesy Nils Philippsen.
.. change::
:tags: bug, installation, windows
:tickets: 4967
Added a workaround for a setuptools-related failure that has been observed
as occurring on Windows installations, where setuptools is not correctly
reporting a build error when the MSVC build dependencies are not installed
and therefore not allowing graceful degradation into non C extensions
builds.
.. change::
:tags: bug, sql, py3k
:tickets: 4931
Changed the ``repr()`` of the :class:`.quoted_name` construct to use
regular string repr() under Python 3, rather than running it through
"backslashreplace" escaping, which can be misleading.
.. change::
:tags: bug, oracle, firebird
:tickets: 4931
Modified the approach of "name normalization" for the Oracle and Firebird
dialects, which converts from the UPPERCASE-as-case-insensitive convention
of these dialects into lowercase-as-case-insensitive for SQLAlchemy, to not
automatically apply the :class:`.quoted_name` construct to a name that
matches itself under upper or lower case conversion, as is the case for
many non-european characters. All names used within metadata structures
are converted to :class:`.quoted_name` objects in any case; the change
here would only affect the output of some inspection functions.
.. change::
:tags: bug, schema
:tickets: 4911
Fixed bug where a table that would have a column label overlap with a plain
column name, such as "foo.id AS foo_id" vs. "foo.foo_id", would prematurely
generate the ``._label`` attribute for a column before this overlap could
be detected due to the use of the ``index=True`` or ``unique=True`` flag on
the column in conjunction with the default naming convention of
``"column_0_label"``. This would then lead to failures when ``._label``
were used later to generate a bound parameter name, in particular those
used by the ORM when generating the WHERE clause for an UPDATE statement.
The issue has been fixed by using an alternate ``._label`` accessor for DDL
generation that does not affect the state of the :class:`_schema.Column`. The
accessor also bypasses the key-deduplication step as it is not necessary
for DDL, the naming is now consistently ``"<tablename>_<columnname>"``
without any subsequent numeric symbols when used in DDL.
.. change::
:tags: bug, engine
:tickets: 4902
Fixed bug where parameter repr as used in logging and error reporting needs
additional context in order to distinguish between a list of parameters for
a single statement and a list of parameter lists, as the "list of lists"
structure could also indicate a single parameter list where the first
parameter itself is a list, such as for an array parameter. The
engine/connection now passes in an additional boolean indicating how the
parameters should be considered. The only SQLAlchemy backend that expects
arrays as parameters is that of psycopg2 which uses pyformat parameters,
so this issue has not been too apparent, however as other drivers that use
positional gain more features it is important that this be supported. It
also eliminates the need for the parameter repr function to guess based on
the parameter structure passed.
.. change::
:tags: usecase, schema
:tickets: 4894
Added DDL support for "computed columns"; these are DDL column
specifications for columns that have a server-computed value, either upon
SELECT (known as "virtual") or at the point of which they are INSERTed or
UPDATEd (known as "stored"). Support is established for Postgresql, MySQL,
Oracle SQL Server and Firebird. Thanks to Federico Caselli for lots of work
on this one.
.. seealso::
:ref:`computed_ddl`
.. change::
:tags: bug, engine, postgresql
:tickets: 4955
Fixed bug in :class:`_reflection.Inspector` where the cache key generation did not
take into account arguments passed in the form of tuples, such as the tuple
of view name styles to return for the PostgreSQL dialect. This would lead
the inspector to cache too generally for a more specific set of criteria.
The logic has been adjusted to include every keyword element in the cache,
as every argument is expected to be appropriate for a cache else the
caching decorator should be bypassed by the dialect.
.. change::
:tags: bug, mssql
:tickets: 4923
Fixed an issue in the :meth:`_engine.Engine.table_names` method where it would
feed the dialect's default schema name back into the dialect level table
function, which in the case of SQL Server would interpret it as a
dot-tokenized schema name as viewed by the mssql dialect, which would
cause the method to fail in the case where the database username actually
had a dot inside of it. In 1.3, this method is still used by the
:meth:`_schema.MetaData.reflect` function so is a prominent codepath. In 1.4,
which is the current master development branch, this issue doesn't exist,
both because :meth:`_schema.MetaData.reflect` isn't using this method nor does the
method pass the default schema name explicitly. The fix nonetheless
guards against the default server name value returned by the dialect from
being interpreted as dot-tokenized name under any circumstances by
wrapping it in quoted_name().
.. change::
:tags: bug, orm
:tickets: 4974
Fixed ORM bug where a "secondary" table that referred to a selectable which
in some way would refer to the local primary table would apply aliasing to
both sides of the join condition when a relationship-related join, either
via :meth:`_query.Query.join` or by :func:`_orm.joinedload`, were generated. The
"local" side is now excluded.
.. change::
:tags: usecase, sql
:tickets: 4276
Added new accessors to expressions of type :class:`_types.JSON` to allow for
specific datatype access and comparison, covering strings, integers,
numeric, boolean elements. This revises the documented approach of
CASTing to string when comparing values, instead adding specific
functionality into the PostgreSQL, SQlite, MySQL dialects to reliably
deliver these basic types in all cases.
.. seealso::
:class:`_types.JSON`
:meth:`.JSON.Comparator.as_string`
:meth:`.JSON.Comparator.as_boolean`
:meth:`.JSON.Comparator.as_float`
:meth:`.JSON.Comparator.as_integer`
.. change::
:tags: usecase, oracle
:tickets: 4799
Added dialect-level flag ``encoding_errors`` to the cx_Oracle dialect,
which can be specified as part of :func:`_sa.create_engine`. This is passed
to SQLAlchemy's unicode decoding converter under Python 2, and to
cx_Oracle's ``cursor.var()`` object as the ``encodingErrors`` parameter
under Python 3, for the very unusual case that broken encodings are present
in the target database which cannot be fetched unless error handling is
relaxed. The value is ultimately one of the Python "encoding errors"
parameters passed to ``decode()``.
.. change::
:tags: usecase, sql
:tickets: 4933
The :func:`_expression.text` construct now supports "unique" bound parameters, which
will dynamically uniquify themselves on compilation thus allowing multiple
:func:`_expression.text` constructs with the same bound parameter names to be combined
together.
.. change::
:tags: bug, oracle
:tickets: 4913
The :class:`_types.NCHAR` datatype will now bind to the
``cx_Oracle.FIXED_NCHAR`` DBAPI data bindings when used in a bound
parameter, which supplies proper comparison behavior against a
variable-length string. Previously, the :class:`_types.NCHAR` datatype
would bind to ``cx_oracle.NCHAR`` which is not fixed length; the
:class:`_types.CHAR` datatype already binds to ``cx_Oracle.FIXED_CHAR``
so it is now consistent that :class:`_types.NCHAR` binds to
``cx_Oracle.FIXED_NCHAR``.
.. change::
:tags: bug, firebird
:tickets: 4903
Added additional "disconnect" message "Error writing data to the
connection" to Firebird disconnection detection. Pull request courtesy
lukens.
.. changelog::
```
### 1.3.10
```
:released: October 9, 2019
.. change::
:tags: bug, mssql
:tickets: 4857
Fixed bug in SQL Server dialect with new "max_identifier_length" feature
where the mssql dialect already featured this flag, and the implementation
did not accommodate for the new initialization hook correctly.
.. change::
:tags: bug, oracle
:tickets: 4898, 4857
Fixed regression in Oracle dialect that was inadvertently using max
identifier length of 128 characters on Oracle server 12.2 and greater even
though the stated contract for the remainder of the 1.3 series is that
this value stays at 30 until version SQLAlchemy 1.4. Also repaired issues
with the retrieval of the "compatibility" version, and removed the warning
emitted when the "v$parameter" view was not accessible as this was causing
user confusion.
.. changelog::
```
### 1.3.9
```
:released: October 4, 2019
.. change::
:tags: usecase, engine
:tickets: 4857
Added new :func:`_sa.create_engine` parameter
:paramref:`_sa.create_engine.max_identifier_length`. This overrides the
dialect-coded "max identifier length" in order to accommodate for databases
that have recently changed this length and the SQLAlchemy dialect has
not yet been adjusted to detect for that version. This parameter interacts
with the existing :paramref:`_sa.create_engine.label_length` parameter in that
it establishes the maximum (and default) value for anonymously generated
labels. Additionally, post-connection detection of max identifier lengths
has been added to the dialect system. This feature is first being used
by the Oracle dialect.
.. seealso::
:ref:`oracle_max_identifier_lengths` - in the Oracle dialect documentation
.. change::
:tags: usecase, oracle
:tickets: 4857
The Oracle dialect now emits a warning if Oracle version 12.2 or greater is
used, and the :paramref:`_sa.create_engine.max_identifier_length` parameter is
not set. The version in this specific case defaults to that of the
"compatibility" version set in the Oracle server configuration, not the
actual server version. In version 1.4, the default max_identifier_length
for 12.2 or greater will move to 128 characters. In order to maintain
forwards compatibility, applications should set
:paramref:`_sa.create_engine.max_identifier_length` to 30 in order to maintain
the same length behavior, or to 128 in order to test the upcoming behavior.
This length determines among other things how generated constraint names
are truncated for statements like ``CREATE CONSTRAINT`` and ``DROP
CONSTRAINT``, which means a the new length may produce a name-mismatch
against a name that was generated with the old length, impacting database
migrations.
.. seealso::
:ref:`oracle_max_identifier_lengths` - in the Oracle dialect documentation
.. change::
:tags: usecase, sqlite
:tickets: 4863
Added support for sqlite "URI" connections, which allow for sqlite-specific
flags to be passed in the query string such as "read only" for Python
sqlite3 drivers that support this.
.. seealso::
:ref:`pysqlite_uri_connections`
.. change::
:tags: bug, tests
:tickets: 4285
Fixed unit test regression released in 1.3.8 that would cause failure for
Oracle, SQL Server and other non-native ENUM platforms due to new
enumeration tests added as part of :ticket:`4285` enum sortability in the
unit of work; the enumerations created constraints that were duplicated on
name.
.. change::
:tags: bug, oracle
:tickets: 4886
Restored adding cx_Oracle.DATETIME to the setinputsizes() call when a
SQLAlchemy :class:`.Date`, :class:`.DateTime` or :class:`.Time` datatype is
used, as some complex queries require this to be present. This was removed
in the 1.2 series for arbitrary reasons.
.. change::
:tags: bug, mssql
:tickets: 4883
Added identifier quoting to the schema name applied to the "use" statement
which is invoked when a SQL Server multipart schema name is used within a
:class:`_schema.Table` that is being reflected, as well as for :class:`_reflection.Inspector`
methods such as :meth:`_reflection.Inspector.get_table_names`; this accommodates for
special characters or spaces in the database name. Additionally, the "use"
statement is not emitted if the current database matches the target owner
database name being passed.
.. change::
:tags: bug, orm
:tickets: 4872
Fixed regression in selectinload loader strategy caused by :ticket:`4775`
(released in version 1.3.6) where a many-to-one attribute of None would no
longer be populated by the loader. While this was usually not noticeable
due to the lazyloader populating None upon get, it would lead to a detached
instance error if the object were detached.
.. change::
:tags: bug, orm
:tickets: 4873
Passing a plain string expression to :meth:`.Session.query` is deprecated,
as all string coercions were removed in :ticket:`4481` and this one should
have been included. The :func:`_expression.literal_column` function may be used to
produce a textual column expression.
.. change::
:tags: usecase, sql
:tickets: 4847
Added an explicit error message for the case when objects passed to
:c
pyup-bot
commented
4 years ago
Update SQLAlchemy from 1.1.13 to 1.3.19.
Changelog
### 1.3.19 ``` :released: August 17, 2020 .. change:: :tags: usecase, py3k :tickets: 5357 Added a ``**kw`` argument to the :meth:`.DeclarativeMeta.__init__` method. This allows a class to support the :pep:`487` metaclass hook ``__init_subclass__``. Pull request courtesy Ewen Gillies. .. change:: :tags: bug, sql :tickets: 5470 Repaired an issue where the "ORDER BY" clause rendering a label name rather than a complete expression, which is particularly important for SQL Server, would fail to occur if the expression were enclosed in a parenthesized grouping in some cases. This case has been added to test support. The change additionally adjusts the "automatically add ORDER BY columns when DISTINCT is present" behavior of ORM query, deprecated in 1.4, to more accurately detect column expressions that are already present. .. change:: :tags: usecase, mysql :tickets: 5481 The MySQL dialect will render FROM DUAL for a SELECT statement that has no FROM clause but has a WHERE clause. This allows things like "SELECT 1 WHERE EXISTS (subquery)" kinds of queries to be used as well as other use cases. .. change:: :tags: bug, mssql, sql :tickets: 5467 Fixed bug where the mssql dialect incorrectly escaped object names that contained ']' character(s). .. change:: :tags: bug, reflection, sqlite, mssql :tickets: 5456 Applied a sweep through all included dialects to ensure names that contain single or double quotes are properly escaped when querying system tables, for all :class:`.Inspector` methods that accept object names as an argument (e.g. table names, view names, etc). SQLite and MSSQL contained two quoting issues that were repaired. .. change:: :tags: bug, mysql :tickets: 5411 Fixed an issue where CREATE TABLE statements were not specifying the COLLATE keyword correctly. .. change:: :tags: bug, datatypes, sql :tickets: 4733 The ``LookupError`` message will now provide the user with up to four possible values that a column is constrained to via the :class:`.Enum`. Values longer than 11 characters will be truncated and replaced with ellipses. Pull request courtesy Ramon Williams. .. change:: :tags: bug, postgresql :tickets: 5476 Fixed issue where the return type for the various RANGE comparison operators would itself be the same RANGE type rather than BOOLEAN, which would cause an undesirable result in the case that a :class:`.TypeDecorator` that defined result-processing behavior were in use. Pull request courtesy Jim Bosch. .. change:: :tags: bug, mysql :tickets: 5493 Added MariaDB code 1927 to the list of "disconnect" codes, as recent MariaDB versions apparently use this code when the database server was stopped. .. change:: :tags: usecase, declarative, orm :tickets: 5513 The name of the virtual column used when using the :class:`_declarative.AbstractConcreteBase` and :class:`_declarative.ConcreteBase` classes can now be customized, to allow for models that have a column that is actually named ``type``. Pull request courtesy Jesse-Bakker. .. change:: :tags: usecase, orm :tickets: 5494 Adjusted the workings of the :meth:`_orm.Mapper.all_orm_descriptors` accessor to represent the attributes in the order that they are located in a deterministic way, assuming the use of Python 3.6 or higher which maintains the sorting order of class attributes based on how they were declared. This sorting is not guaranteed to match the declared order of attributes in all cases however; see the method documentation for the exact scheme. .. change:: :tags: bug, sql :tickets: 5500 Fixed issue where the :paramref:`_engine.Connection.execution_options.schema_translate_map` feature would not take effect when the :meth:`_schema.Sequence.next_value` function function for a :class:`_schema.Sequence` were used in the :paramref:`_schema.Column.server_default` parameter and the create table DDL were emitted. .. changelog:: ``` ### 1.3.18 ``` :released: June 25, 2020 .. change:: :tags: bug, sqlite :tickets: 5395 Added "exists" to the list of reserved words for SQLite so that this word will be quoted when used as a label or column name. Pull request courtesy Thodoris Sotiropoulos. .. change:: :tags: bug, mssql :tickets: 5366, 5364 Refined the logic used by the SQL Server dialect to interpret multi-part schema names that contain many dots, to not actually lose any dots if the name does not have bracking or quoting used, and additionally to support a "dbname" token that has many parts including that it may have multiple, independently-bracketed sections. .. change:: :tags: bug, mssql, pyodbc :tickets: 5346 Fixed an issue in the pyodbc connector such that a warning about pyodbc "drivername" would be emitted when using a totally empty URL. Empty URLs are normal when producing a non-connected dialect object or when using the "creator" argument to create_engine(). The warning now only emits if the driver name is missing but other parameters are still present. .. change:: :tags: bug, mssql :tickets: 5373 Fixed issue with assembling the ODBC connection string for the pyodbc DBAPI. Tokens containing semicolons and/or braces "{}" were not being correctly escaped, causing the ODBC driver to misinterpret the connection string attributes. .. change:: :tags: usecase, orm :tickets: 5326 Improve error message when using :meth:`_query.Query.filter_by` in a query where the first entity is not a mapped class. .. change:: :tags: sql, schema :tickets: 5324 Introduce :class:`.IdentityOptions` to store common parameters for sequences and identity columns. .. change:: :tags: usecase, sql :tickets: 5309 Added a ".schema" parameter to the :func:`_expression.table` construct, allowing ad-hoc table expressions to also include a schema name. Pull request courtesy Dylan Modesitt. .. change:: :tags: bug, mssql :tickets: 5339 Fixed issue where ``datetime.time`` parameters were being converted to ``datetime.datetime``, making them incompatible with comparisons like ``>=`` against an actual :class:`_mssql.TIME` column. .. change:: :tags: bug, mssql :tickets: 5359 Fixed an issue where the ``is_disconnect`` function in the SQL Server pyodbc dialect was incorrectly reporting the disconnect state when the exception messsage had a substring that matched a SQL Server ODBC error code. .. change:: :tags: bug, engine :tickets: 5326 Further refinements to the fixes to the "reset" agent fixed in :ticket:`5326`, which now emits a warning when it is not being correctly invoked and corrects for the behavior. Additional scenarios have been identified and fixed where this warning was being emitted. .. change:: :tags: usecase, sqlite :tickets: 5297 SQLite 3.31 added support for computed column. This change enables their support in SQLAlchemy when targeting SQLite. .. change:: :tags: bug, schema :tickets: 5276 Fixed issue where ``dialect_options`` were omitted when a database object (e.g., :class:`.Table`) was copied using :func:`.tometadata`. .. change:: :tags: bug, sql :tickets: 5344 Correctly apply self_group in type_coerce element. The type coerce element did not correctly apply grouping rules when using in an expression .. change:: :tags: bug, oracle, reflection :tickets: 5421 Fixed bug in Oracle dialect where indexes that contain the full set of primary key columns would be mistaken as the primary key index itself, which is omitted, even if there were multiples. The check has been refined to compare the name of the primary key constraint against the index name itself, rather than trying to guess based on the columns present in the index. .. change:: :tags: change, sql, sybase :tickets: 5294 Added ``.offset`` support to sybase dialect. Pull request courtesy Alan D. Snow. .. change:: :tags: bug, engine :tickets: 5341 Fixed issue in :class:`.URL` object where stringifying the object would not URL encode special characters, preventing the URL from being re-consumable as a real URL. Pull request courtesy Miguel Grinberg. .. change:: :tags: usecase, mysql :tickets: 4860 Implemented row-level locking support for mysql. Pull request courtesy Quentin Somerville. .. change:: :tags: change, mssql :tickets: 5321 Moved the ``supports_sane_rowcount_returning = False`` requirement from the ``PyODBCConnector`` level to the ``MSDialect_pyodbc`` since pyodbc does work properly in some circumstances. .. change:: :tags: change, examples Added new option ``--raw`` to the examples.performance suite which will dump the raw profile test for consumption by any number of profiling visualizer tools. Removed the "runsnake" option as runsnake is very hard to build at this point; .. change:: :tags: bug, sql :tickets: 5353 Added :meth:`.Select.with_hint` output to the generic SQL string that is produced when calling ``str()`` on a statement. Previously, this clause would be omitted under the assumption that it was dialect specific. The hint text is presented within brackets to indicate the rendering of such hints varies among backends. .. change:: :tags: usecase, orm :tickets: 5198 Added a new parameter :paramref:`_orm.query_expression.default_expr` to the :func:`_orm.query_expression` construct, which will be appled to queries automatically if the :func:`_orm.with_expression` option is not used. Pull request courtesy Haoyu Sun. .. changelog:: ``` ### 1.3.17 ``` :released: May 13, 2020 .. change:: :tags: bug, oracle :tickets: 5246 Some modifications to how the cx_oracle dialect sets up per-column outputtype handlers for LOB and numeric datatypes to adjust for potential changes coming in cx_Oracle 8. .. change:: :tags: bug, orm :tickets: 5288 Fixed bug where using :func:`.with_polymorphic` as the target of a join via :meth:`.RelationshipComparator.of_type` on a mapper that already has a subquery-based with_polymorphic setting that's equivalent to the one requested would not correctly alias the ON clause in the join. .. change:: :tags: bug, oracle, performance :tickets: 5314 Changed the implementation of fetching CLOB and BLOB objects to use cx_Oracle's native implementation which fetches CLOB/BLOB objects inline with other result columns, rather than performing a separate fetch. As always, this can be disabled by setting auto_convert_lobs to False. As part of this change, the behavior of a CLOB that was given a blank string on INSERT now returns None on SELECT, which is now consistent with that of VARCHAR on Oracle. .. change:: :tags: usecase, postgresql :tickets: 5265 Added support for columns or type :class:`.ARRAY` of :class:`.Enum`, :class:`.JSON` or :class:`_postgresql.JSONB` in PostgreSQL. Previously a workaround was required in these use cases. .. change:: :tags: schema :tickets: 4138 Add ``comment`` attribute to :class:`_schema.Column` ``__repr__`` method. .. change:: :tags: bug, orm :tickets: 5303 Fixed issue in the area of where loader options such as selectinload() interact with the baked query system, such that the caching of a query is not supposed to occur if the loader options themselves have elements such as with_polymorphic() objects in them that currently are not cache-compatible. The baked loader could sometimes not fully invalidate itself in these some of these scenarios leading to missed eager loads. .. change:: :tags: bug, engine :tickets: 5326 Fixed fairly critical issue where the DBAPI connection could be returned to the connection pool while still in an un-rolled-back state. The reset agent responsible for rolling back the connection could be corrupted in the case that the transaction was "closed" without being rolled back or committed, which can occur in some scenarios when using ORM sessions and emitting .close() in a certain pattern involving savepoints. The fix ensures that the reset agent is always active. .. change:: :tags: bug, orm :tickets: 5304 Modified the internal "identity set" implementation, which is a set that hashes objects on their id() rather than their hash values, to not actually call the ``__hash__()`` method of the objects, which are typically user-mapped objects. Some methods were calling this method as a side effect of the implementation. .. change:: :tags: usecase, postgresql :tickets: 5266 Raise an explicit :class:`.exc.CompileError` when adding a table with a column of type :class:`.ARRAY` of :class:`.Enum` configured with :paramref:`.Enum.native_enum` set to ``False`` when :paramref:`.Enum.create_constraint` is not set to ``False`` .. change:: :tags: bug, schema :tickets: 5298 Fixed issue where an :class:`.Index` that is deferred in being associated with a table, such as as when it contains a :class:`.Column` that is not associated with any :class:`.Table` yet, would fail to attach correctly if it also contained a non table-oriented expression. .. change:: :tags: change, firebird :tickets: 5278 Adjusted dialect loading for ``firebird://`` URIs so the external sqlalchemy-firebird dialect will be used if it has been installed, otherwise fall back to the (now deprecated) internal Firebird dialect. .. change:: :tags: bug, mssql, reflection :tickets: 5255 Fix a regression introduced by the reflection of computed column in MSSQL when using the legacy TDS version 4.2. The dialect will try to detect the protocol version of first connect and run in compatibility mode if it cannot detect it. .. change:: :tags: bug, mssql, reflection :tickets: 5271 Fix a regression introduced by the reflection of computed column in MSSQL when using SQL server versions before 2012, which does not support the ``concat`` function. .. change:: :tags: bug, orm :tickets: 5269 An informative error message is raised when an ORM many-to-one comparison is attempted against an object that is not an actual mapped instance. Comparisons such as those to scalar subqueries aren't supported; generalized comparison with subqueries is better achieved using :meth:`~.RelationshipProperty.Comparator.has`. .. change:: :tags: usecase, orm :tickets: 5262 Added an accessor :attr:`.ColumnProperty.Comparator.expressions` which provides access to the group of columns mapped under a multi-column :class:`.ColumnProperty` attribute. .. change:: :tags: bug, schema :tickets: 5316 A warning is emitted when making use of the :attr:`.MetaData.sorted_tables` attribute as well as the :func:`_schema.sort_tables` function, and the given tables cannot be correctly sorted due to a cyclic dependency between foreign key constraints. In this case, the functions will no longer sort the involved tables by foreign key, and a warning will be emitted. Other tables that are not part of the cycle will still be returned in dependency order. Previously, the sorted_table routines would return a collection that would unconditionally omit all foreign keys when a cycle was detected, and no warning was emitted. .. change:: :tags: orm, usecase :tickets: 5237 Introduce :paramref:`_orm.relationship.sync_backref` flag in a relationship to control if the synchronization events that mutate the in-Python attributes are added. This supersedes the previous change :ticket:`5149`, which warned that ``viewonly=True`` relationship target of a back_populates or backref configuration would be disallowed. .. changelog:: ``` ### 1.3.16 ``` :released: April 7, 2020 .. change:: :tags: oracle, usecase :tickets: 5200 Implemented AUTOCOMMIT isolation level for Oracle when using cx_Oracle. Also added a fixed default isolation level of READ COMMITTED for Oracle. .. change:: :tags: bug, mysql :tickets: 5239 Fixed issue in MySQL dialect when connecting to a pseudo-MySQL database such as that provided by ProxySQL, the up front check for isolation level when it returns no row will not prevent the dialect from continuing to connect. A warning is emitted that the isolation level could not be detected. .. change:: :tags: bug, tests :tickets: 5201 Fixed an issue that prevented the test suite from running with the recently released py.test 5.4.0. .. change:: :tags: bug, oracle, reflection :tickets: 5146 Fixed regression / incorrect fix caused by fix for :ticket:`5146` where the Oracle dialect reads from the "all_tab_comments" view to get table comments but fails to accommodate for the current owner of the table being requested, causing it to read the wrong comment if multiple tables of the same name exist in multiple schemas. .. change:: :tags: types, enum :tickets: 5183 The :class:`.Enum` type now supports the parameter :paramref:`.Enum.length` to specify the length of the VARCHAR column to create when using non native enums by setting :paramref:`.Enum.native_enum` to ``False`` .. change:: :tags: bug, orm :tickets: 5228 Fixed bug in :func:`_orm.selectinload` loading option where two or more loaders that represent different relationships with the same string key name as referenced from a single :func:`_orm.with_polymorphic` construct with multiple subclass mappers would fail to invoke each subqueryload separately, instead making use of a single string-based slot that would prevent the other loaders from being invoked. .. change:: :tags: schema, reflection :tickets: 5063 Added support for reflection of "computed" columns, which are now returned as part of the structure returned by :meth:`_reflection.Inspector.get_columns`. When reflecting full :class:`_schema.Table` objects, computed columns will be represented using the :class:`.Computed` construct. .. change:: :tags: orm, performance :tickets: 5162 Modified the queries used by subqueryload and selectinload to no longer ORDER BY the primary key of the parent entity; this ordering was there to allow the rows as they come in to be copied into lists directly with a minimal level of Python-side collation. However, these ORDER BY clauses can negatively impact the performance of the query as in many scenarios these columns are derived from a subquery or are otherwise not actual primary key columns such that SQL planners cannot make use of indexes. The Python-side collation uses the native itertools.group_by() to collate the incoming rows, and has been modified to allow multiple row-groups-per-parent to be assembled together using list.extend(), which should still allow for relatively fast Python-side performance. There will still be an ORDER BY present for a relationship that includes an explicit order_by parameter, however this is the only ORDER BY that will be added to the query for both kinds of loading. .. change:: :tags: mssql, mysql, oracle, usecase :tickets: 5137 Added support for :meth:`.ColumnOperators.is_distinct_from` and :meth:`.ColumnOperators.isnot_distinct_from` to SQL Server, MySQL, and Oracle. .. change:: :tags: sqlite, usecase :tickets: 5164 Implemented AUTOCOMMIT isolation level for SQLite when using pysqlite. .. change:: :tags: bug, postgresql :tickets: 5205 Fixed issue where a "covering" index, e.g. those which have an INCLUDE clause, would be reflected including all the columns in INCLUDE clause as regular columns. A warning is now emitted if these additional columns are detected indicating that they are currently ignored. Note that full support for "covering" indexes is part of :ticket:`4458`. Pull request courtesy Marat Sharafutdinov. .. change:: :tags: sql, types :tickets: 5052 Add ability to literal compile a :class:`DateTime`, :class:`Date` or :class:`Time` when using the string dialect for debugging purposes. This change does not impact real dialect implementation that retain their current behavior. .. change:: :tags: installer :tickets: 5207 Ensured that the "pyproject.toml" file is not included in builds, as the presence of this file indicates to pip that a pep-517 installation process should be used. As this mode of operation appears to be not well supported by current tools / distros, these problems are avoided within the scope of SQLAlchemy installation by omitting the file. .. change:: :tags: bug, orm :tickets: 5210 Fixed issue where a lazyload that uses session-local "get" against a target many-to-one relationship where an object with the correct primary key is present, however it's an instance of a sibling class, does not correctly return None as is the case when the lazy loader actually emits a load for that row. .. change:: :tags: bug, orm, declarative :tickets: 5238 The string argument accepted as the first positional argument by the :func:`_orm.relationship` function when using the Declarative API is no longer interpreted using the Python ``eval()`` function; instead, the name is dot separated and the names are looked up directly in the name resolution dictionary without treating the value as a Python expression. However, passing a string argument to the other :func:`_orm.relationship` parameters that necessarily must accept Python expressions will still use ``eval()``; the documentation has been clarified to ensure that there is no ambiguity that this is in use. .. seealso:: :ref:`declarative_relationship_eval` - details on string evaluation .. changelog:: ``` ### 1.3.15 ``` :released: March 11, 2020 .. change:: :tags: bug, orm :tickets: 5194 Adjusted the error message emitted by :meth:`_query.Query.join` when a left hand side can't be located that the :meth:`_query.Query.select_from` method is the best way to resolve the issue. Also, within the 1.3 series, used a deterministic ordering when determining the FROM clause from a given column entity passed to :class:`_query.Query` so that the same expression is determined each time. .. change:: :tags: orm, bug :tickets: 5196 Fixed regression in 1.3.14 due to :ticket:`4849` where a sys.exc_info() call failed to be invoked correctly when a flush error would occur. Test coverage has been added for this exception case. .. changelog:: ``` ### 1.3.14 ``` :released: March 10, 2020 .. change:: :tags: bug, sql, postgresql :tickets: 5181 Fixed bug where a CTE of an INSERT/UPDATE/DELETE that also uses RETURNING could then not be SELECTed from directly, as the internal state of the compiler would try to treat the outer SELECT as a DELETE statement itself and access nonexistent state. .. change:: :tags: bug, orm :tickets: 5110 Fixed regression caused in 1.3.13 by :ticket:`5056` where a refactor of the ORM path registry system made it such that a path could no longer be compared to an empty tuple, which can occur in a particular kind of joined eager loading path. The "empty tuple" use case has been resolved so that the path registry is compared to a path registry in all cases; the :class:`.PathRegistry` object itself now implements ``__eq__()`` and ``__ne__()`` methods which will take place for all equality comparisons and continue to succeed in the not anticipated case that a non- :class:`.PathRegistry` object is compared, while emitting a warning that this object should not be the subject of the comparison. .. change:: :tags: bug, orm :tickets: 5149 Setting a relationship to viewonly=True which is also the target of a back_populates or backref configuration will now emit a warning and eventually be disallowed. back_populates refers specifically to mutation of an attribute or collection, which is disallowed when the attribute is subject to viewonly=True. The viewonly attribute is not subject to persistence behaviors which means it will not reflect correct results when it is locally mutated. .. change:: :tags: bug, oracle :tickets: 5146 Fixed a reflection bug where table comments could only be retrieved for tables actually owned by the user but not for tables visible to the user but owned by someone else. Pull request courtesy Dave Hirschfeld. .. change:: :tags: bug, performance :tickets: 5180 Revised an internal change to the test system added as a result of :ticket:`5085` where a testing-related module per dialect would be loaded unconditionally upon making use of that dialect, pulling in SQLAlchemy's testing framework as well as the ORM into the module import space. This would only impact initial startup time and memory to a modest extent, however it's best that these additional modules aren't reverse-dependent on straight Core usage. .. change:: :tags: bug, installation :tickets: 5138 Vendored the ``inspect.formatannotation`` function inside of ``sqlalchemy.util.compat``, which is needed for the vendored version of ``inspect.formatargspec``. The function is not documented in cPython and is not guaranteed to be available in future Python versions. .. change:: :tags: bug, mssql :tickets: 5132 Fixed issue where the :class:`_mssql.DATETIMEOFFSET` type would not accommodate for the ``None`` value, introduced as part of the series of fixes for this type first introduced in :ticket:`4983`, :ticket:`5045`. Additionally, added support for passing a backend-specific date formatted string through this type, as is typically allowed for date/time types on most other DBAPIs. .. change:: :tags: bug, engine :tickets: 5182 Expanded the scope of cursor/connection cleanup when a statement is executed to include when the result object fails to be constructed, or an after_cursor_execute() event raises an error, or autocommit / autoclose fails. This allows the DBAPI cursor to be cleaned up on failure and for connectionless execution allows the connection to be closed out and returned to the connection pool, where previously it waiting until garbage collection would trigger a pool return. .. change:: :tags: bug, postgresql :tickets: 5158 Fixed issue where the "schema_translate_map" feature would not work with a PostgreSQL native enumeration type (i.e. :class:`.Enum`, :class:`_postgresql.ENUM`) in that while the "CREATE TYPE" statement would be emitted with the correct schema, the schema would not be rendered in the CREATE TABLE statement at the point at which the enumeration was referenced. .. change:: :tags: usecase, ext :tickets: 5114 Added keyword arguments to the :meth:`.MutableList.sort` function so that a key function as well as the "reverse" keyword argument can be provided. .. change:: :tags: bug, general, py3k :tickets: 4849 Applied an explicit "cause" to most if not all internally raised exceptions that are raised from within an internal exception catch, to avoid misleading stacktraces that suggest an error within the handling of an exception. While it would be preferable to suppress the internally caught exception in the way that the ``__suppress_context__`` attribute would, there does not as yet seem to be a way to do this without suppressing an enclosing user constructed context, so for now it exposes the internally caught exception as the cause so that full information about the context of the error is maintained. .. change:: :tags: orm, bug :tickets: 5121 Fixed an additional regression in the same area as that of :ticket:`5080` introduced in 1.3.0b3 via :ticket:`4468` where the ability to create a joined option across a :func:`.with_polymorphic` into a relationship against the base class of that with_polymorphic, and then further into regular mapped relationships would fail as the base class component would not add itself to the load path in a way that could be located by the loader strategy. The changes applied in :ticket:`5080` have been further refined to also accommodate this scenario. .. change:: :tags: bug, postgresql, reflection :tickets: 5170 Fixed bug where PostgreSQL reflection of CHECK constraints would fail to parse the constraint if the SQL text contained newline characters. The regular expression has been adjusted to accommodate for this case. Pull request courtesy Eric Borczuk. .. change:: :tags: usecase, orm :tickets: 5129 Added a new flag :paramref:`.InstanceEvents.restore_load_context` and :paramref:`.SessionEvents.restore_load_context` which apply to the :meth:`.InstanceEvents.load`, :meth:`.InstanceEvents.refresh`, and :meth:`.SessionEvents.loaded_as_persistent` events, which when set will restore the "load context" of the object after the event hook has been called. This ensures that the object remains within the "loader context" of the load operation that is already ongoing, rather than the object being transferred to a new load context due to refresh operations which may have occurred in the event. A warning is now emitted when this condition occurs, which recommends use of the flag to resolve this case. The flag is "opt-in" so that there is no risk introduced to existing applications. The change additionally adds support for the ``raw=True`` flag to session lifecycle events. .. change:: :tags: bug, mysql :tickets: 5173 Fixed issue in MySQL :meth:`.mysql.Insert.on_duplicate_key_update` construct where using a SQL function or other composed expression for a column argument would not properly render the ``VALUES`` keyword surrounding the column itself. .. changelog:: ``` ### 1.3.13 ``` :released: January 22, 2020 .. change:: :tags: bug, postgresql :tickets: 5039 Fixed issue where the PostgreSQL dialect would fail to parse a reflected CHECK constraint that was a boolean-valued function (as opposed to a boolean-valued expression). .. change:: :tags: bug, ext :tickets: 5086 Fixed bug in sqlalchemy.ext.serializer where a unique :class:`.BindParameter` object could conflict with itself if it were present in the mapping itself, as well as the filter condition of the query, as one side would be used against the non-deserialized version and the other side would use the deserialized version. Logic is added to :class:`.BindParameter` similar to its "clone" method which will uniquify the parameter name upon deserialize so that it doesn't conflict with its original. .. change:: :tags: usecase, sql :tickets: 5079 A function created using :class:`.GenericFunction` can now specify that the name of the function should be rendered with or without quotes by assigning the :class:`.quoted_name` construct to the .name element of the object. Prior to 1.3.4, quoting was never applied to function names, and some quoting was introduced in :ticket:`4467` but no means to force quoting for a mixed case name was available. Additionally, the :class:`.quoted_name` construct when used as the name will properly register its lowercase name in the function registry so that the name continues to be available via the ``func.`` registry. .. seealso:: :class:`.GenericFunction` .. change:: :tags: bug, engine :tickets: 5048 Fixed issue where the collection of value processors on a :class:`.Compiled` object would be mutated when "expanding IN" parameters were used with a datatype that has bind value processors; in particular, this would mean that when using statement caching and/or baked queries, the same compiled._bind_processors collection would be mutated concurrently. Since these processors are the same function for a given bind parameter namespace every time, there was no actual negative effect of this issue, however, the execution of a :class:`.Compiled` object should never be causing any changes in its state, especially given that they are intended to be thread-safe and reusable once fully constructed. .. change:: :tags: tests, postgresql :tickets: 5057 Improved detection of two phase transactions requirement for the PostgreSQL database by testing that max_prepared_transactions is set to a value greater than 0. Pull request courtesy Federico Caselli. .. change:: :tags: bug, orm, engine :tickets: 5056, 5050, 5071 Added test support and repaired a wide variety of unnecessary reference cycles created for short-lived objects, mostly in the area of ORM queries. Thanks much to Carson Ip for the help on this. .. change:: :tags: orm, bug :tickets: 5107 Fixed regression in loader options introduced in 1.3.0b3 via :ticket:`4468` where the ability to create a loader option using :meth:`.PropComparator.of_type` targeting an aliased entity that is an inheriting subclass of the entity which the preceding relationship refers to would fail to produce a matching path. See also :ticket:`5082` fixed in this same release which involves a similar kind of issue. .. change:: :tags: bug, tests :tickets: 4946 Fixed a few test failures which would occur on Windows due to SQLite file locking issues, as well as some timing issues in connection pool related tests; pull request courtesy Federico Caselli. .. change:: :tags: orm, bug :tickets: 5082 Fixed regression in joined eager loading introduced in 1.3.0b3 via :ticket:`4468` where the ability to create a joined option across a :func:`.with_polymorphic` into a polymorphic subclass using :meth:`.RelationshipProperty.of_type` and then further along regular mapped relationships would fail as the polymorphic subclass would not add itself to the load path in a way that could be located by the loader strategy. A tweak has been made to resolve this scenario. .. change:: :tags: performance, orm Identified a performance issue in the system by which a join is constructed based on a mapped relationship. The clause adaption system would be used for the majority of join expressions including in the common case where no adaptation is needed. The conditions under which this adaptation occur have been refined so that average non-aliased joins along a simple relationship without a "secondary" table use about 70% less function calls. .. change:: :tags: usecase, postgresql :tickets: 5040 Added support for prefixes to the :class:`_expression.CTE` construct, to allow support for Postgresql 12 "MATERIALIZED" and "NOT MATERIALIZED" phrases. Pull request courtesy Marat Sharafutdinov. .. seealso:: :meth:`_expression.HasCTE.cte` .. change:: :tags: bug, mssql :tickets: 5045 Fixed issue where a timezone-aware ``datetime`` value being converted to string for use as a parameter value of a :class:`_mssql.DATETIMEOFFSET` column was omitting the fractional seconds. .. change:: :tags: bug, orm :tickets: 5068 Repaired a warning in the ORM flush process that was not covered by test coverage when deleting objects that use the "version_id" feature. This warning is generally unreachable unless using a dialect that sets the "supports_sane_rowcount" flag to False, which is not typically the case however is possible for some MySQL configurations as well as older Firebird drivers, and likely some third party dialects. .. change:: :tags: bug, orm :tickets: 5065 Fixed bug where usage of joined eager loading would not properly wrap the query inside of a subquery when :meth:`_query.Query.group_by` were used against the query. When any kind of result-limiting approach is used, such as DISTINCT, LIMIT, OFFSET, joined eager loading embeds the row-limited query inside of a subquery so that the collection results are not impacted. For some reason, the presence of GROUP BY was never included in this criterion, even though it has a similar effect as using DISTINCT. Additionally, the bug would prevent using GROUP BY at all for a joined eager load query for most database platforms which forbid non-aggregated, non-grouped columns from being in the query, as the additional columns for the joined eager load would not be accepted by the database. .. changelog:: ``` ### 1.3.12 ``` :released: December 16, 2019 .. change:: :tags: bug, sql :tickets: 5028 Fixed bug where "distinct" keyword passed to :func:`_expression.select` would not treat a string value as a "label reference" in the same way that the :meth:`_expression.select.distinct` does; it would instead raise unconditionally. This keyword argument and the others passed to :func:`_expression.select` will ultimately be deprecated for SQLAlchemy 2.0. .. change:: :tags: bug, orm :tickets: 4997 Fixed issue involving ``lazy="raise"`` strategy where an ORM delete of an object would raise for a simple "use-get" style many-to-one relationship that had lazy="raise" configured. This is inconsistent vs. the change introduced in 1.3 as part of :ticket:`4353`, where it was established that a history operation that does not expect emit SQL should bypass the ``lazy="raise"`` check, and instead effectively treat it as ``lazy="raise_on_sql"`` for this case. The fix adjusts the lazy loader strategy to not raise for the case where the lazy load was instructed that it should not emit SQL if the object were not present. .. change:: :tags: bug, sql Changed the text of the exception for "Can't resolve label reference" to include other kinds of label coercions, namely that "DISTINCT" is also in this category under the PostgreSQL dialect. .. change:: :tags: bug, orm :tickets: 5000 Fixed regression introduced in 1.3.0 related to the association proxy refactor in :ticket:`4351` that prevented :func:`.composite` attributes from working in terms of an association proxy that references them. .. change:: :tags: bug, mssql :tickets: 4983 Repaired support for the :class:`_mssql.DATETIMEOFFSET` datatype on PyODBC, by adding PyODBC-level result handlers as it does not include native support for this datatype. This includes usage of the Python 3 "timezone" tzinfo subclass in order to set up a timezone, which on Python 2 makes use of a minimal backport of "timezone" in sqlalchemy.util. .. change:: :tags: bug, orm :tickets: 4993 Setting persistence-related flags on :func:`_orm.relationship` while also setting viewonly=True will now emit a regular warning, as these flags do not make sense for a viewonly=True relationship. In particular, the "cascade" settings have their own warning that is generated based on the individual values, such as "delete, delete-orphan", that should not apply to a viewonly relationship. Note however that in the case of "cascade", these settings are still erroneously taking effect even though the relationship is set up as "viewonly". In 1.4, all persistence-related cascade settings will be disallowed on a viewonly=True relationship in order to resolve this issue. .. change:: :tags: bug, sqlite :tickets: 5014 Fixed issue to workaround SQLite's behavior of assigning "numeric" affinity to JSON datatypes, first described at :ref:`change_3850`, which returns scalar numeric JSON values as a number and not as a string that can be JSON deserialized. The SQLite-specific JSON deserializer now gracefully degrades for this case as an exception and bypasses deserialization for single numeric values, as from a JSON perspective they are already deserialized. .. change:: :tags: bug, orm, py3k :tickets: 4990 Fixed issue where when assigning a collection to itself as a slice, the mutation operation would fail as it would first erase the assigned collection inadvertently. As an assignment that does not change the contents should not generate events, the operation is now a no-op. Note that the fix only applies to Python 3; in Python 2, the ``__setitem__`` hook isn't called in this case; ``__setslice__`` is used instead which recreates the list item-by-item in all cases. .. change:: :tags: bug, orm :tickets: 5034 Fixed issue where by if the "begin" of a transaction failed at the Core engine/connection level, such as due to network error or database is locked for some transactional recipes, within the context of the :class:`.Session` procuring that connection from the connection pool and then immediately returning it, the ORM :class:`.Session` would not close the connection despite this connection not being stored within the state of that :class:`.Session`. This would lead to the connection being cleaned out by the connection pool weakref handler within garbage collection which is an unpreferred codepath that in some special configurations can emit errors in standard error. .. changelog:: ``` ### 1.3.11 ``` :released: November 11, 2019 .. change:: :tags: bug, mssql :tickets: 4973 Fixed issue in MSSQL dialect where an expression-based OFFSET value in a SELECT would be rejected, even though the dialect can render this expression inside of a ROW NUMBER-oriented LIMIT/OFFSET construct. .. change:: :tags: orm, usecase :tickets: 4934 Added accessor :meth:`_query.Query.is_single_entity` to :class:`_query.Query`, which will indicate if the results returned by this :class:`_query.Query` will be a list of ORM entities, or a tuple of entities or column expressions. SQLAlchemy hopes to improve upon the behavior of single entity / tuples in future releases such that the behavior would be explicit up front, however this attribute should be helpful with the current behavior. Pull request courtesy Patrick Hayes. .. change:: :tags: bug, mysql :tickets: 4945 Added "Connection was killed" message interpreted from the base pymysql.Error class in order to detect closed connection, based on reports that this message is arriving via a pymysql.InternalError() object which indicates pymysql is not handling it correctly. .. change:: :tags: bug, orm :tickets: 4954 The :paramref:`_orm.relationship.omit_join` flag was not intended to be manually set to True, and will now emit a warning when this occurs. The omit_join optimization is detected automatically, and the ``omit_join`` flag was only intended to disable the optimization in the hypothetical case that the optimization may have interfered with correct results, which has not been observed with the modern version of this feature. Setting the flag to True when it is not automatically detected may cause the selectin load feature to not work correctly when a non-default primary join condition is in use. .. change:: :tags: bug, orm :tickets: 4915 A warning is emitted if a primary key value is passed to :meth:`_query.Query.get` that consists of None for all primary key column positions. Previously, passing a single None outside of a tuple would raise a ``TypeError`` and passing a composite None (tuple of None values) would silently pass through. The fix now coerces the single None into a tuple where it is handled consistently with the other None conditions. Thanks to Lev Izraelit for the help with this. .. change:: :tags: bug, orm :tickets: 4947 The :class:`.BakedQuery` will not cache a query that was modified by a :meth:`.QueryEvents.before_compile` event, so that compilation hooks that may be applying ad-hoc modifications to queries will take effect on each run. In particular this is helpful for events that modify queries used in lazy loading as well as eager loading such as "select in" loading. In order to re-enable caching for a query modified by this event, a new flag ``bake_ok`` is added; see :ref:`baked_with_before_compile` for details. A longer term plan to provide a new form of SQL caching should solve this kind of issue more comprehensively. .. change:: :tags: bug, tests :tickets: 4920 Fixed test failures which would occur with newer SQLite as of version 3.30 or greater, due to their addition of nulls ordering syntax as well as new restrictions on aggregate functions. Pull request courtesy Nils Philippsen. .. change:: :tags: bug, installation, windows :tickets: 4967 Added a workaround for a setuptools-related failure that has been observed as occurring on Windows installations, where setuptools is not correctly reporting a build error when the MSVC build dependencies are not installed and therefore not allowing graceful degradation into non C extensions builds. .. change:: :tags: bug, sql, py3k :tickets: 4931 Changed the ``repr()`` of the :class:`.quoted_name` construct to use regular string repr() under Python 3, rather than running it through "backslashreplace" escaping, which can be misleading. .. change:: :tags: bug, oracle, firebird :tickets: 4931 Modified the approach of "name normalization" for the Oracle and Firebird dialects, which converts from the UPPERCASE-as-case-insensitive convention of these dialects into lowercase-as-case-insensitive for SQLAlchemy, to not automatically apply the :class:`.quoted_name` construct to a name that matches itself under upper or lower case conversion, as is the case for many non-european characters. All names used within metadata structures are converted to :class:`.quoted_name` objects in any case; the change here would only affect the output of some inspection functions. .. change:: :tags: bug, schema :tickets: 4911 Fixed bug where a table that would have a column label overlap with a plain column name, such as "foo.id AS foo_id" vs. "foo.foo_id", would prematurely generate the ``._label`` attribute for a column before this overlap could be detected due to the use of the ``index=True`` or ``unique=True`` flag on the column in conjunction with the default naming convention of ``"column_0_label"``. This would then lead to failures when ``._label`` were used later to generate a bound parameter name, in particular those used by the ORM when generating the WHERE clause for an UPDATE statement. The issue has been fixed by using an alternate ``._label`` accessor for DDL generation that does not affect the state of the :class:`_schema.Column`. The accessor also bypasses the key-deduplication step as it is not necessary for DDL, the naming is now consistently ``"<tablename>_<columnname>"`` without any subsequent numeric symbols when used in DDL. .. change:: :tags: bug, engine :tickets: 4902 Fixed bug where parameter repr as used in logging and error reporting needs additional context in order to distinguish between a list of parameters for a single statement and a list of parameter lists, as the "list of lists" structure could also indicate a single parameter list where the first parameter itself is a list, such as for an array parameter. The engine/connection now passes in an additional boolean indicating how the parameters should be considered. The only SQLAlchemy backend that expects arrays as parameters is that of psycopg2 which uses pyformat parameters, so this issue has not been too apparent, however as other drivers that use positional gain more features it is important that this be supported. It also eliminates the need for the parameter repr function to guess based on the parameter structure passed. .. change:: :tags: usecase, schema :tickets: 4894 Added DDL support for "computed columns"; these are DDL column specifications for columns that have a server-computed value, either upon SELECT (known as "virtual") or at the point of which they are INSERTed or UPDATEd (known as "stored"). Support is established for Postgresql, MySQL, Oracle SQL Server and Firebird. Thanks to Federico Caselli for lots of work on this one. .. seealso:: :ref:`computed_ddl` .. change:: :tags: bug, engine, postgresql :tickets: 4955 Fixed bug in :class:`_reflection.Inspector` where the cache key generation did not take into account arguments passed in the form of tuples, such as the tuple of view name styles to return for the PostgreSQL dialect. This would lead the inspector to cache too generally for a more specific set of criteria. The logic has been adjusted to include every keyword element in the cache, as every argument is expected to be appropriate for a cache else the caching decorator should be bypassed by the dialect. .. change:: :tags: bug, mssql :tickets: 4923 Fixed an issue in the :meth:`_engine.Engine.table_names` method where it would feed the dialect's default schema name back into the dialect level table function, which in the case of SQL Server would interpret it as a dot-tokenized schema name as viewed by the mssql dialect, which would cause the method to fail in the case where the database username actually had a dot inside of it. In 1.3, this method is still used by the :meth:`_schema.MetaData.reflect` function so is a prominent codepath. In 1.4, which is the current master development branch, this issue doesn't exist, both because :meth:`_schema.MetaData.reflect` isn't using this method nor does the method pass the default schema name explicitly. The fix nonetheless guards against the default server name value returned by the dialect from being interpreted as dot-tokenized name under any circumstances by wrapping it in quoted_name(). .. change:: :tags: bug, orm :tickets: 4974 Fixed ORM bug where a "secondary" table that referred to a selectable which in some way would refer to the local primary table would apply aliasing to both sides of the join condition when a relationship-related join, either via :meth:`_query.Query.join` or by :func:`_orm.joinedload`, were generated. The "local" side is now excluded. .. change:: :tags: usecase, sql :tickets: 4276 Added new accessors to expressions of type :class:`_types.JSON` to allow for specific datatype access and comparison, covering strings, integers, numeric, boolean elements. This revises the documented approach of CASTing to string when comparing values, instead adding specific functionality into the PostgreSQL, SQlite, MySQL dialects to reliably deliver these basic types in all cases. .. seealso:: :class:`_types.JSON` :meth:`.JSON.Comparator.as_string` :meth:`.JSON.Comparator.as_boolean` :meth:`.JSON.Comparator.as_float` :meth:`.JSON.Comparator.as_integer` .. change:: :tags: usecase, oracle :tickets: 4799 Added dialect-level flag ``encoding_errors`` to the cx_Oracle dialect, which can be specified as part of :func:`_sa.create_engine`. This is passed to SQLAlchemy's unicode decoding converter under Python 2, and to cx_Oracle's ``cursor.var()`` object as the ``encodingErrors`` parameter under Python 3, for the very unusual case that broken encodings are present in the target database which cannot be fetched unless error handling is relaxed. The value is ultimately one of the Python "encoding errors" parameters passed to ``decode()``. .. change:: :tags: usecase, sql :tickets: 4933 The :func:`_expression.text` construct now supports "unique" bound parameters, which will dynamically uniquify themselves on compilation thus allowing multiple :func:`_expression.text` constructs with the same bound parameter names to be combined together. .. change:: :tags: bug, oracle :tickets: 4913 The :class:`_types.NCHAR` datatype will now bind to the ``cx_Oracle.FIXED_NCHAR`` DBAPI data bindings when used in a bound parameter, which supplies proper comparison behavior against a variable-length string. Previously, the :class:`_types.NCHAR` datatype would bind to ``cx_oracle.NCHAR`` which is not fixed length; the :class:`_types.CHAR` datatype already binds to ``cx_Oracle.FIXED_CHAR`` so it is now consistent that :class:`_types.NCHAR` binds to ``cx_Oracle.FIXED_NCHAR``. .. change:: :tags: bug, firebird :tickets: 4903 Added additional "disconnect" message "Error writing data to the connection" to Firebird disconnection detection. Pull request courtesy lukens. .. changelog:: ``` ### 1.3.10 ``` :released: October 9, 2019 .. change:: :tags: bug, mssql :tickets: 4857 Fixed bug in SQL Server dialect with new "max_identifier_length" feature where the mssql dialect already featured this flag, and the implementation did not accommodate for the new initialization hook correctly. .. change:: :tags: bug, oracle :tickets: 4898, 4857 Fixed regression in Oracle dialect that was inadvertently using max identifier length of 128 characters on Oracle server 12.2 and greater even though the stated contract for the remainder of the 1.3 series is that this value stays at 30 until version SQLAlchemy 1.4. Also repaired issues with the retrieval of the "compatibility" version, and removed the warning emitted when the "v$parameter" view was not accessible as this was causing user confusion. .. changelog:: ``` ### 1.3.9 ``` :released: October 4, 2019 .. change:: :tags: usecase, engine :tickets: 4857 Added new :func:`_sa.create_engine` parameter :paramref:`_sa.create_engine.max_identifier_length`. This overrides the dialect-coded "max identifier length" in order to accommodate for databases that have recently changed this length and the SQLAlchemy dialect has not yet been adjusted to detect for that version. This parameter interacts with the existing :paramref:`_sa.create_engine.label_length` parameter in that it establishes the maximum (and default) value for anonymously generated labels. Additionally, post-connection detection of max identifier lengths has been added to the dialect system. This feature is first being used by the Oracle dialect. .. seealso:: :ref:`oracle_max_identifier_lengths` - in the Oracle dialect documentation .. change:: :tags: usecase, oracle :tickets: 4857 The Oracle dialect now emits a warning if Oracle version 12.2 or greater is used, and the :paramref:`_sa.create_engine.max_identifier_length` parameter is not set. The version in this specific case defaults to that of the "compatibility" version set in the Oracle server configuration, not the actual server version. In version 1.4, the default max_identifier_length for 12.2 or greater will move to 128 characters. In order to maintain forwards compatibility, applications should set :paramref:`_sa.create_engine.max_identifier_length` to 30 in order to maintain the same length behavior, or to 128 in order to test the upcoming behavior. This length determines among other things how generated constraint names are truncated for statements like ``CREATE CONSTRAINT`` and ``DROP CONSTRAINT``, which means a the new length may produce a name-mismatch against a name that was generated with the old length, impacting database migrations. .. seealso:: :ref:`oracle_max_identifier_lengths` - in the Oracle dialect documentation .. change:: :tags: usecase, sqlite :tickets: 4863 Added support for sqlite "URI" connections, which allow for sqlite-specific flags to be passed in the query string such as "read only" for Python sqlite3 drivers that support this. .. seealso:: :ref:`pysqlite_uri_connections` .. change:: :tags: bug, tests :tickets: 4285 Fixed unit test regression released in 1.3.8 that would cause failure for Oracle, SQL Server and other non-native ENUM platforms due to new enumeration tests added as part of :ticket:`4285` enum sortability in the unit of work; the enumerations created constraints that were duplicated on name. .. change:: :tags: bug, oracle :tickets: 4886 Restored adding cx_Oracle.DATETIME to the setinputsizes() call when a SQLAlchemy :class:`.Date`, :class:`.DateTime` or :class:`.Time` datatype is used, as some complex queries require this to be present. This was removed in the 1.2 series for arbitrary reasons. .. change:: :tags: bug, mssql :tickets: 4883 Added identifier quoting to the schema name applied to the "use" statement which is invoked when a SQL Server multipart schema name is used within a :class:`_schema.Table` that is being reflected, as well as for :class:`_reflection.Inspector` methods such as :meth:`_reflection.Inspector.get_table_names`; this accommodates for special characters or spaces in the database name. Additionally, the "use" statement is not emitted if the current database matches the target owner database name being passed. .. change:: :tags: bug, orm :tickets: 4872 Fixed regression in selectinload loader strategy caused by :ticket:`4775` (released in version 1.3.6) where a many-to-one attribute of None would no longer be populated by the loader. While this was usually not noticeable due to the lazyloader populating None upon get, it would lead to a detached instance error if the object were detached. .. change:: :tags: bug, orm :tickets: 4873 Passing a plain string expression to :meth:`.Session.query` is deprecated, as all string coercions were removed in :ticket:`4481` and this one should have been included. The :func:`_expression.literal_column` function may be used to produce a textual column expression. .. change:: :tags: usecase, sql :tickets: 4847 Added an explicit error message for the case when objects passed to :c