Changelog
### 1.14.1
```
==========================
This is a bugfix release for some problems reported following the 1.14.0 release. The major
problems fixed are the following.
* Problems with the new array printing, particularly the printing of complex
values, Please report any additional problems that may turn up.
* Problems with ``np.einsum`` due to the new ``optimized=True`` default. Some
fixes for optimization have been applied and ``optimize=False`` is now the
default.
* The sort order in ``np.unique`` when ``axis=<some-number>`` will now always
be lexicographic in the subarray elements. In previous NumPy versions there
was an optimization that could result in sorting the subarrays as unsigned
byte strings.
* The change in 1.14.0 that multi-field indexing of structured arrays returns a
view instead of a copy has been reverted but remains on track for NumPy 1.15.
Affected users should read the 1.14.1 Numpy User Guide section
"basics/structured arrays/accessing multiple fields" for advice on how to
manage this transition.
The Python versions supported in this release are 2.7 and 3.4 - 3.6. The Python
3.6 wheels available from PIP are built with Python 3.6.2 and should be
compatible with all previous versions of Python 3.6. The source releases were
cythonized with Cython 0.26.1, which is known to **not** support the upcoming
Python 3.7 release. People who wish to run Python 3.7 should check out the
NumPy repo and try building with the, as yet, unreleased master branch of
Cython.
Contributors
============
A total of 14 people contributed to this release. People with a "+" by their
names contributed a patch for the first time.
* Allan Haldane
* Charles Harris
* Daniel Smith
* Dennis Weyland +
* Eric Larson
* Eric Wieser
* Jarrod Millman
* Kenichi Maehashi +
* Marten van Kerkwijk
* Mathieu Lamarre
* Sebastian Berg
* Simon Conseil
* Simon Gibbons
* xoviat
Pull requests merged
====================
A total of 36 pull requests were merged for this release.
* `10339 <https://github.com/numpy/numpy/pull/10339>`__: BUG: restrict the __config__ modifications to win32
* `10368 <https://github.com/numpy/numpy/pull/10368>`__: MAINT: Adjust type promotion in linalg.norm
* `10375 <https://github.com/numpy/numpy/pull/10375>`__: BUG: add missing paren and remove quotes from repr of fieldless...
* `10395 <https://github.com/numpy/numpy/pull/10395>`__: MAINT: Update download URL in setup.py.
* `10396 <https://github.com/numpy/numpy/pull/10396>`__: BUG: fix einsum issue with unicode input and py2
* `10397 <https://github.com/numpy/numpy/pull/10397>`__: BUG: fix error message not formatted in einsum
* `10398 <https://github.com/numpy/numpy/pull/10398>`__: DOC: add documentation about how to handle new array printing
* `10403 <https://github.com/numpy/numpy/pull/10403>`__: BUG: Set einsum optimize parameter default to `False`.
* `10424 <https://github.com/numpy/numpy/pull/10424>`__: ENH: Fix repr of np.record objects to match np.void types 10412
* `10425 <https://github.com/numpy/numpy/pull/10425>`__: MAINT: Update zesty to artful for i386 testing
* `10431 <https://github.com/numpy/numpy/pull/10431>`__: REL: Add 1.14.1 release notes template
* `10435 <https://github.com/numpy/numpy/pull/10435>`__: MAINT: Use ValueError for duplicate field names in lookup (backport)
* `10534 <https://github.com/numpy/numpy/pull/10534>`__: BUG: Provide a better error message for out-of-order fields
* `10536 <https://github.com/numpy/numpy/pull/10536>`__: BUG: Resize bytes_ columns in genfromtxt (backport of 10401)
* `10537 <https://github.com/numpy/numpy/pull/10537>`__: BUG: multifield-indexing adds padding bytes: revert for 1.14.1
* `10539 <https://github.com/numpy/numpy/pull/10539>`__: BUG: fix np.save issue with python 2.7.5
* `10540 <https://github.com/numpy/numpy/pull/10540>`__: BUG: Add missing DECREF in Py2 int() cast
* `10541 <https://github.com/numpy/numpy/pull/10541>`__: TST: Add circleci document testing to maintenance/1.14.x
* `10542 <https://github.com/numpy/numpy/pull/10542>`__: BUG: complex repr has extra spaces, missing + (1.14 backport)
* `10550 <https://github.com/numpy/numpy/pull/10550>`__: BUG: Set missing exception after malloc
* `10557 <https://github.com/numpy/numpy/pull/10557>`__: BUG: In numpy.i, clear CARRAY flag if wrapped buffer is not C_CONTIGUOUS.
* `10558 <https://github.com/numpy/numpy/pull/10558>`__: DEP: Issue FutureWarning when malformed records detected.
* `10559 <https://github.com/numpy/numpy/pull/10559>`__: BUG: Fix einsum optimize logic for singleton dimensions
* `10560 <https://github.com/numpy/numpy/pull/10560>`__: BUG: Fix calling ufuncs with a positional output argument.
* `10561 <https://github.com/numpy/numpy/pull/10561>`__: BUG: Fix various Big-Endian test failures (ppc64)
* `10562 <https://github.com/numpy/numpy/pull/10562>`__: BUG: Make dtype.descr error for out-of-order fields.
* `10563 <https://github.com/numpy/numpy/pull/10563>`__: BUG: arrays not being flattened in `union1d`
* `10607 <https://github.com/numpy/numpy/pull/10607>`__: MAINT: Update sphinxext submodule hash.
* `10608 <https://github.com/numpy/numpy/pull/10608>`__: BUG: Revert sort optimization in np.unique.
* `10609 <https://github.com/numpy/numpy/pull/10609>`__: BUG: infinite recursion in str of 0d subclasses
* `10610 <https://github.com/numpy/numpy/pull/10610>`__: BUG: Align type definition with generated lapack
* `10612 <https://github.com/numpy/numpy/pull/10612>`__: BUG/ENH: Improve output for structured non-void types
* `10622 <https://github.com/numpy/numpy/pull/10622>`__: BUG: deallocate recursive closure in arrayprint.py (1.14 backport)
* `10624 <https://github.com/numpy/numpy/pull/10624>`__: BUG: Correctly identify comma separated dtype strings
* `10629 <https://github.com/numpy/numpy/pull/10629>`__: BUG: deallocate recursive closure in arrayprint.py (backport...
* `10630 <https://github.com/numpy/numpy/pull/10630>`__: REL: Prepare for 1.14.1 release.
=========================
```
### 1.14.0
```
==========================
Numpy 1.14.0 is the result of seven months of work and contains a large number
of bug fixes and new features, along with several changes with potential
compatibility issues. The major change that users will notice are the
stylistic changes in the way numpy arrays and scalars are printed, a change
that will affect doctests. See below for details on how to preserve the
old style printing when needed.
A major decision affecting future development concerns the schedule for
dropping Python 2.7 support in the runup to 2020. The decision has been made to
support 2.7 for all releases made in 2018, with the last release being
designated a long term release with support for bug fixes extending through
2019. In 2019 support for 2.7 will be dropped in all new releases. More details
can be found in the relevant NEP_.
This release supports Python 2.7 and 3.4 - 3.6.
.. _NEP: https://github.com/numpy/numpy/blob/master/doc/neps/dropping-python2.7-proposal.rst
Highlights
==========
* The `np.einsum` function uses BLAS when possible
* ``genfromtxt``, ``loadtxt``, ``fromregex`` and ``savetxt`` can now handle
files with arbitrary Python supported encoding.
* Major improvements to printing of NumPy arrays and scalars.
New functions
=============
* ``parametrize``: decorator added to numpy.testing
* ``chebinterpolate``: Interpolate function at Chebyshev points.
* ``format_float_positional`` and ``format_float_scientific`` : format
floating-point scalars unambiguously with control of rounding and padding.
* ``PyArray_ResolveWritebackIfCopy`` and ``PyArray_SetWritebackIfCopyBase``,
new C-API functions useful in achieving PyPy compatibity.
Deprecations
============
* Using ``np.bool_`` objects in place of integers is deprecated. Previously
``operator.index(np.bool_)`` was legal and allowed constructs such as
``[1, 2, 3][np.True_]``. That was misleading, as it behaved differently from
``np.array([1, 2, 3])[np.True_]``.
* Truth testing of an empty array is deprecated. To check if an array is not
empty, use ``array.size > 0``.
* Calling ``np.bincount`` with ``minlength=None`` is deprecated.
``minlength=0`` should be used instead.
* Calling ``np.fromstring`` with the default value of the ``sep`` argument is
deprecated. When that argument is not provided, a broken version of
``np.frombuffer`` is used that silently accepts unicode strings and -- after
encoding them as either utf-8 (python 3) or the default encoding
(python 2) -- treats them as binary data. If reading binary data is
desired, ``np.frombuffer`` should be used directly.
* The ``style`` option of array2string is deprecated in non-legacy printing mode.
* ``PyArray_SetUpdateIfCopyBase`` has been deprecated. For NumPy versions >= 1.14
use ``PyArray_SetWritebackIfCopyBase`` instead, see `C API changes` below for
more details.
* The use of ``UPDATEIFCOPY`` arrays is deprecated, see `C API changes` below
for details. We will not be dropping support for those arrays, but they are
not compatible with PyPy.
Future Changes
==============
* ``np.issubdtype`` will stop downcasting dtype-like arguments.
It might be expected that ``issubdtype(np.float32, 'float64')`` and
``issubdtype(np.float32, np.float64)`` mean the same thing - however, there
was an undocumented special case that translated the former into
``issubdtype(np.float32, np.floating)``, giving the surprising result of True.
This translation now gives a warning that explains what translation is
occurring. In the future, the translation will be disabled, and the first
example will be made equivalent to the second.
* ``np.linalg.lstsq`` default for ``rcond`` will be changed. The ``rcond``
parameter to ``np.linalg.lstsq`` will change its default to machine precision
times the largest of the input array dimensions. A FutureWarning is issued
when ``rcond`` is not passed explicitly.
* ``a.flat.__array__()`` will return a writeable copy of ``a`` when ``a`` is
non-contiguous. Previously it returned an UPDATEIFCOPY array when ``a`` was
writeable. Currently it returns a non-writeable copy. See gh-7054 for a
discussion of the issue.
* Unstructured void array's ``.item`` method will return a bytes object. In the
future, calling ``.item()`` on arrays or scalars of ``np.void`` datatype will
return a ``bytes`` object instead of a buffer or int array, the same as
returned by ``bytes(void_scalar)``. This may affect code which assumed the
return value was mutable, which will no longer be the case. A
``FutureWarning`` is now issued when this would occur.
Compatibility notes
===================
The mask of a masked array view is also a view rather than a copy
-----------------------------------------------------------------
There was a FutureWarning about this change in NumPy 1.11.x. In short, it is
now the case that, when changing a view of a masked array, changes to the mask
are propagated to the original. That was not previously the case. This change
affects slices in particular. Note that this does not yet work properly if the
mask of the original array is ``nomask`` and the mask of the view is changed.
See gh-5580 for an extended discussion. The original behavior of having a copy
of the mask can be obtained by calling the ``unshare_mask`` method of the view.
``np.ma.masked`` is no longer writeable
---------------------------------------
Attempts to mutate the ``masked`` constant now error, as the underlying arrays
are marked readonly. In the past, it was possible to get away with::
emulating a function that sometimes returns np.ma.masked
val = random.choice([np.ma.masked, 10])
var_arr = np.asarray(val)
val_arr += 1 now errors, previously changed np.ma.masked.data
``np.ma`` functions producing ``fill_value``s have changed
----------------------------------------------------------
Previously, ``np.ma.default_fill_value`` would return a 0d array, but
``np.ma.minimum_fill_value`` and ``np.ma.maximum_fill_value`` would return a
tuple of the fields. Instead, all three methods return a structured ``np.void``
object, which is what you would already find in the ``.fill_value`` attribute.
Additionally, the dtype guessing now matches that of ``np.array`` - so when
passing a python scalar ``x``, ``maximum_fill_value(x)`` is always the same as
``maximum_fill_value(np.array(x))``. Previously ``x = long(1)`` on Python 2
violated this assumption.
``a.flat.__array__()`` returns non-writeable arrays when ``a`` is non-contiguous
--------------------------------------------------------------------------------
The intent is that the UPDATEIFCOPY array previously returned when ``a`` was
non-contiguous will be replaced by a writeable copy in the future. This
temporary measure is aimed to notify folks who expect the underlying array be
modified in this situation that that will no longer be the case. The most
likely places for this to be noticed is when expressions of the form
``np.asarray(a.flat)`` are used, or when ``a.flat`` is passed as the out
parameter to a ufunc.
``np.tensordot`` now returns zero array when contracting over 0-length dimension
--------------------------------------------------------------------------------
Previously ``np.tensordot`` raised a ValueError when contracting over 0-length
dimension. Now it returns a zero array, which is consistent with the behaviour
of ``np.dot`` and ``np.einsum``.
``numpy.testing`` reorganized
-----------------------------
This is not expected to cause problems, but possibly something has been left
out. If you experience an unexpected import problem using ``numpy.testing``
let us know.
``np.asfarray`` no longer accepts non-dtypes through the ``dtype`` argument
---------------------------------------------------------------------------
This previously would accept ``dtype=some_array``, with the implied semantics
of ``dtype=some_array.dtype``. This was undocumented, unique across the numpy
functions, and if used would likely correspond to a typo.
1D ``np.linalg.norm`` preserves float input types, even for arbitrary orders
----------------------------------------------------------------------------
Previously, this would promote to ``float64`` when arbitrary orders were
passed, despite not doing so under the simple cases::
>>> f32 = np.float32([[1, 2]])
>>> np.linalg.norm(f32, 2.0, axis=-1).dtype
dtype('float32')
>>> np.linalg.norm(f32, 2.0001, axis=-1).dtype
dtype('float64') numpy 1.13
dtype('float32') numpy 1.14
This change affects only ``float32`` and ``float16`` arrays.
``count_nonzero(arr, axis=())`` now counts over no axes, not all axes
---------------------------------------------------------------------
Elsewhere, ``axis==()`` is always understood as "no axes", but
`count_nonzero` had a special case to treat this as "all axes". This was
inconsistent and surprising. The correct way to count over all axes has always
been to pass ``axis == None``.
``__init__.py`` files added to test directories
-----------------------------------------------
This is for pytest compatibility in the case of duplicate test file names in
the different directories. As a result, ``run_module_suite`` no longer works,
i.e., ``python <path-to-test-file>`` results in an error.
``.astype(bool)`` on unstructured void arrays now calls ``bool`` on each element
--------------------------------------------------------------------------------
On Python 2, ``void_array.astype(bool)`` would always return an array of
``True``, unless the dtype is ``V0``. On Python 3, this operation would usually
crash. Going forwards, `astype` matches the behavior of ``bool(np.void)``,
considering a buffer of all zeros as false, and anything else as true.
Checks for ``V0`` can still be done with ``arr.dtype.itemsize == 0``.
``MaskedArray.squeeze`` never returns ``np.ma.masked``
------------------------------------------------------
``np.squeeze`` is documented as returning a view, but the masked variant would
sometimes return ``masked``, which is not a view. This has been fixed, so that
the result is always a view on the original masked array.
This breaks any code that used ``masked_arr.squeeze() is np.ma.masked``, but
fixes code that writes to the result of `.squeeze()`.
Renamed first parameter of ``can_cast`` from ``from`` to ``from_``
------------------------------------------------------------------
The previous parameter name ``from`` is a reserved keyword in Python, which made
it difficult to pass the argument by name. This has been fixed by renaming
the parameter to ``from_``.
``isnat`` raises ``TypeError`` when passed wrong type
------------------------------------------------------
The ufunc ``isnat`` used to raise a ``ValueError`` when it was not passed
variables of type ``datetime`` or ``timedelta``. This has been changed to
raising a ``TypeError``.
``dtype.__getitem__`` raises ``TypeError`` when passed wrong type
-----------------------------------------------------------------
When indexed with a float, the dtype object used to raise ``ValueError``.
User-defined types now need to implement ``__str__`` and ``__repr__``
---------------------------------------------------------------------
Previously, user-defined types could fall back to a default implementation of
``__str__`` and ``__repr__`` implemented in numpy, but this has now been
removed. Now user-defined types will fall back to the python default
``object.__str__`` and ``object.__repr__``.
Many changes to array printing, disableable with the new "legacy" printing mode
-------------------------------------------------------------------------------
The ``str`` and ``repr`` of ndarrays and numpy scalars have been changed in
a variety of ways. These changes are likely to break downstream user's
doctests.
These new behaviors can be disabled to mostly reproduce numpy 1.13 behavior by
enabling the new 1.13 "legacy" printing mode. This is enabled by calling
``np.set_printoptions(legacy="1.13")``, or using the new ``legacy`` argument to
``np.array2string``, as ``np.array2string(arr, legacy='1.13')``.
In summary, the major changes are:
* For floating-point types:
* The ``repr`` of float arrays often omits a space previously printed
in the sign position. See the new ``sign`` option to ``np.set_printoptions``.
* Floating-point arrays and scalars use a new algorithm for decimal
representations, giving the shortest unique representation. This will
usually shorten ``float16`` fractional output, and sometimes ``float32`` and
``float128`` output. ``float64`` should be unaffected. See the new
``floatmode`` option to ``np.set_printoptions``.
* Float arrays printed in scientific notation no longer use fixed-precision,
and now instead show the shortest unique representation.
* The ``str`` of floating-point scalars is no longer truncated in python2.
* For other data types:
* Non-finite complex scalars print like ``nanj`` instead of ``nan*j``.
* ``NaT`` values in datetime arrays are now properly aligned.
* Arrays and scalars of ``np.void`` datatype are now printed using hex
notation.
* For line-wrapping:
* The "dtype" part of ndarray reprs will now be printed on the next line
if there isn't space on the last line of array output.
* The ``linewidth`` format option is now always respected.
The `repr` or `str` of an array will never exceed this, unless a single
element is too wide.
* The last line of an array string will never have more elements than earlier
lines.
* An extra space is no longer inserted on the first line if the elements are
too wide.
* For summarization (the use of ``...`` to shorten long arrays):
* A trailing comma is no longer inserted for ``str``.
Previously, ``str(np.arange(1001))`` gave
``'[ 0 1 2 ..., 998 999 1000]'``, which has an extra comma.
* For arrays of 2-D and beyond, when ``...`` is printed on its own line in
order to summarize any but the last axis, newlines are now appended to that
line to match its leading newlines and a trailing space character is
removed.
* ``MaskedArray`` arrays now separate printed elements with commas, always
print the dtype, and correctly wrap the elements of long arrays to multiple
lines. If there is more than 1 dimension, the array attributes are now
printed in a new "left-justified" printing style.
* ``recarray`` arrays no longer print a trailing space before their dtype, and
wrap to the right number of columns.
* 0d arrays no longer have their own idiosyncratic implementations of ``str``
and ``repr``. The ``style`` argument to ``np.array2string`` is deprecated.
* Arrays of ``bool`` datatype will omit the datatype in the ``repr``.
* User-defined ``dtypes`` (subclasses of ``np.generic``) now need to
implement ``__str__`` and ``__repr__``.
Some of these changes are described in more detail below. If you need to retain
the previous behavior for doctests or other reasons, you may want to do
something like::
FIXME: We need the str/repr formatting used in Numpy < 1.14.
try:
np.set_printoptions(legacy='1.13')
except TypeError:
pass
C API changes
=============
PyPy compatible alternative to ``UPDATEIFCOPY`` arrays
------------------------------------------------------
``UPDATEIFCOPY`` arrays are contiguous copies of existing arrays, possibly with
different dimensions, whose contents are copied back to the original array when
their refcount goes to zero and they are deallocated. Because PyPy does not use
refcounts, they do not function correctly with PyPy. NumPy is in the process of
eliminating their use internally and two new C-API functions,
* ``PyArray_SetWritebackIfCopyBase``
* ``PyArray_ResolveWritebackIfCopy``,
have been added together with a complimentary flag,
``NPY_ARRAY_WRITEBACKIFCOPY``. Using the new functionality also requires that
some flags be changed when new arrays are created, to wit:
``NPY_ARRAY_INOUT_ARRAY`` should be replaced by ``NPY_ARRAY_INOUT_ARRAY2`` and
``NPY_ARRAY_INOUT_FARRAY`` should be replaced by ``NPY_ARRAY_INOUT_FARRAY2``.
Arrays created with these new flags will then have the ``WRITEBACKIFCOPY``
semantics.
If PyPy compatibility is not a concern, these new functions can be ignored,
although there will be a ``DeprecationWarning``. If you do wish to pursue PyPy
compatibility, more information on these functions and their use may be found
in the c-api_ documentation and the example in how-to-extend_.
.. _c-api: https://github.com/numpy/numpy/blob/master/doc/source/reference/c-api.array.rst
.. _how-to-extend: https://github.com/numpy/numpy/blob/master/doc/source/user/c-info.how-to-extend.rst
New Features
============
Encoding argument for text IO functions
---------------------------------------
``genfromtxt``, ``loadtxt``, ``fromregex`` and ``savetxt`` can now handle files
with arbitrary encoding supported by Python via the encoding argument.
For backward compatibility the argument defaults to the special ``bytes`` value
which continues to treat text as raw byte values and continues to pass latin1
encoded bytes to custom converters.
Using any other value (including ``None`` for system default) will switch the
functions to real text IO so one receives unicode strings instead of bytes in
the resulting arrays.
External ``nose`` plugins are usable by ``numpy.testing.Tester``
----------------------------------------------------------------
``numpy.testing.Tester`` is now aware of ``nose`` plugins that are outside the
``nose`` built-in ones. This allows using, for example, ``nose-timer`` like
so: ``np.test(extra_argv=['--with-timer', '--timer-top-n', '20'])`` to
obtain the runtime of the 20 slowest tests. An extra keyword ``timer`` was
also added to ``Tester.test``, so ``np.test(timer=20)`` will also report the 20
slowest tests.
``parametrize`` decorator added to ``numpy.testing``
----------------------------------------------------
A basic ``parametrize`` decorator is now available in ``numpy.testing``. It is
intended to allow rewriting yield based tests that have been deprecated in
pytest so as to facilitate the transition to pytest in the future. The nose
testing framework has not been supported for several years and looks like
abandonware.
The new ``parametrize`` decorator does not have the full functionality of the
one in pytest. It doesn't work for classes, doesn't support nesting, and does
not substitute variable names. Even so, it should be adequate to rewrite the
NumPy tests.
``chebinterpolate`` function added to ``numpy.polynomial.chebyshev``
--------------------------------------------------------------------
The new ``chebinterpolate`` function interpolates a given function at the
Chebyshev points of the first kind. A new ``Chebyshev.interpolate`` class
method adds support for interpolation over arbitrary intervals using the scaled
and shifted Chebyshev points of the first kind.
Support for reading lzma compressed text files in Python 3
----------------------------------------------------------
With Python versions containing the ``lzma`` module the text IO functions can
now transparently read from files with ``xz`` or ``lzma`` extension.
``sign`` option added to ``np.setprintoptions`` and ``np.array2string``
-----------------------------------------------------------------------
This option controls printing of the sign of floating-point types, and may be
one of the characters '-', '+' or ' '. With '+' numpy always prints the sign of
positive values, with ' ' it always prints a space (whitespace character) in
the sign position of positive values, and with '-' it will omit the sign
character for positive values. The new default is '-'.
This new default changes the float output relative to numpy 1.13. The old
behavior can be obtained in 1.13 "legacy" printing mode, see compatibility
notes above.
``hermitian`` option added to``np.linalg.matrix_rank``
------------------------------------------------------
The new ``hermitian`` option allows choosing between standard SVD based matrix
rank calculation and the more efficient eigenvalue based method for
symmetric/hermitian matrices.
``threshold`` and ``edgeitems`` options added to ``np.array2string``
--------------------------------------------------------------------
These options could previously be controlled using ``np.set_printoptions``, but
now can be changed on a per-call basis as arguments to ``np.array2string``.
``concatenate`` and ``stack`` gained an ``out`` argument
--------------------------------------------------------
A preallocated buffer of the desired dtype can now be used for the output of
these functions.
Support for PGI flang compiler on Windows
-----------------------------------------
The PGI flang compiler is a Fortran front end for LLVM released by NVIDIA under
the Apache 2 license. It can be invoked by ::
python setup.py config --compiler=clang --fcompiler=flang install
There is little experience with this new compiler, so any feedback from people
using it will be appreciated.
Improvements
============
Numerator degrees of freedom in ``random.noncentral_f`` need only be positive.
------------------------------------------------------------------------------
Prior to NumPy 1.14.0, the numerator degrees of freedom needed to be > 1, but
the distribution is valid for values > 0, which is the new requirement.
The GIL is released for all ``np.einsum`` variations
----------------------------------------------------
Some specific loop structures which have an accelerated loop version
did not release the GIL prior to NumPy 1.14.0. This oversight has been
fixed.
The `np.einsum` function will use BLAS when possible and optimize by default
----------------------------------------------------------------------------
The ``np.einsum`` function will now call ``np.tensordot`` when appropriate.
Because ``np.tensordot`` uses BLAS when possible, that will speed up execution.
By default, ``np.einsum`` will also attempt optimization as the overhead is
small relative to the potential improvement in speed.
``f2py`` now handles arrays of dimension 0
------------------------------------------
``f2py`` now allows for the allocation of arrays of dimension 0. This allows
for more consistent handling of corner cases downstream.
``numpy.distutils`` supports using MSVC and mingw64-gfortran together
---------------------------------------------------------------------
Numpy distutils now supports using Mingw64 gfortran and MSVC compilers
together. This enables the production of Python extension modules on Windows
containing Fortran code while retaining compatibility with the
binaries distributed by Python.org. Not all use cases are supported,
but most common ways to wrap Fortran for Python are functional.
Compilation in this mode is usually enabled automatically, and can be
selected via the ``--fcompiler`` and ``--compiler`` options to
``setup.py``. Moreover, linking Fortran codes to static OpenBLAS is
supported; by default a gfortran compatible static archive
``openblas.a`` is looked for.
``np.linalg.pinv`` now works on stacked matrices
------------------------------------------------
Previously it was limited to a single 2d array.
``numpy.save`` aligns data to 64 bytes instead of 16
----------------------------------------------------
Saving NumPy arrays in the ``npy`` format with ``numpy.save`` inserts
padding before the array data to align it at 64 bytes. Previously
this was only 16 bytes (and sometimes less due to a bug in the code
for version 2). Now the alignment is 64 bytes, which matches the
widest SIMD instruction set commonly available, and is also the most
common cache line size. This makes ``npy`` files easier to use in
programs which open them with ``mmap``, especially on Linux where an
``mmap`` offset must be a multiple of the page size.
NPZ files now can be written without using temporary files
----------------------------------------------------------
In Python 3.6+ ``numpy.savez`` and ``numpy.savez_compressed`` now write
directly to a ZIP file, without creating intermediate temporary files.
Better support for empty structured and string types
----------------------------------------------------
Structured types can contain zero fields, and string dtypes can contain zero
characters. Zero-length strings still cannot be created directly, and must be
constructed through structured dtypes::
str0 = np.empty(10, np.dtype([('v', str, N)]))['v']
void0 = np.empty(10, np.void)
It was always possible to work with these, but the following operations are
now supported for these arrays:
* `arr.sort()`
* `arr.view(bytes)`
* `arr.resize(...)`
* `pickle.dumps(arr)`
Support for ``decimal.Decimal`` in ``np.lib.financial``
-------------------------------------------------------
Unless otherwise stated all functions within the ``financial`` package now
support using the ``decimal.Decimal`` built-in type.
Float printing now uses "dragon4" algorithm for shortest decimal representation
-------------------------------------------------------------------------------
The ``str`` and ``repr`` of floating-point values (16, 32, 64 and 128 bit) are
now printed to give the shortest decimal representation which uniquely
identifies the value from others of the same type. Previously this was only
true for ``float64`` values. The remaining float types will now often be shorter
than in numpy 1.13. Arrays printed in scientific notation now also use the
shortest scientific representation, instead of fixed precision as before.
Additionally, the `str` of float scalars scalars will no longer be truncated
in python2, unlike python2 `float`s. `np.double` scalars now have a ``str``
and ``repr`` identical to that of a python3 float.
New functions ``np.format_float_scientific`` and ``np.format_float_positional``
are provided to generate these decimal representations.
A new option ``floatmode`` has been added to ``np.set_printoptions`` and
``np.array2string``, which gives control over uniqueness and rounding of
printed elements in an array. The new default is ``floatmode='maxprec'`` with
``precision=8``, which will print at most 8 fractional digits, or fewer if an
element can be uniquely represented with fewer. A useful new mode is
``floatmode="unique"``, which will output enough digits to specify the array
elements uniquely.
Numpy complex-floating-scalars with values like ``inf*j`` or ``nan*j`` now
print as ``infj`` and ``nanj``, like the pure-python ``complex`` type.
The ``FloatFormat`` and ``LongFloatFormat`` classes are deprecated and should
both be replaced by ``FloatingFormat``. Similarly ``ComplexFormat`` and
``LongComplexFormat`` should be replaced by ``ComplexFloatingFormat``.
``void`` datatype elements are now printed in hex notation
----------------------------------------------------------
A hex representation compatible with the python ``bytes`` type is now printed
for unstructured ``np.void`` elements, e.g., ``V4`` datatype. Previously, in
python2 the raw void data of the element was printed to stdout, or in python3
the integer byte values were shown.
printing style for ``void`` datatypes is now independently customizable
-----------------------------------------------------------------------
The printing style of ``np.void`` arrays is now independently customizable
using the ``formatter`` argument to ``np.set_printoptions``, using the
``'void'`` key, instead of the catch-all ``numpystr`` key as before.
Reduced memory usage of ``np.loadtxt``
--------------------------------------
``np.loadtxt`` now reads files in chunks instead of all at once which decreases
its memory usage significantly for large files.
Changes
=======
Multiple-field indexing/assignment of structured arrays
-------------------------------------------------------
The indexing and assignment of structured arrays with multiple fields has
changed in a number of ways, as warned about in previous releases.
First, indexing a structured array with multiple fields, e.g.,
``arr[['f1', 'f3']]``, returns a view into the original array instead of a
copy. The returned view will have extra padding bytes corresponding to
intervening fields in the original array, unlike the copy in 1.13, which will
affect code such as ``arr[['f1', 'f3']].view(newdtype)``.
Second, assignment between structured arrays will now occur "by position"
instead of "by field name". The Nth field of the destination will be set to the
Nth field of the source regardless of field name, unlike in numpy versions 1.6
to 1.13 in which fields in the destination array were set to the
identically-named field in the source array or to 0 if the source did not have
a field.
Correspondingly, the order of fields in a structured dtypes now matters when
computing dtype equality. For example, with the dtypes ::
x = dtype({'names': ['A', 'B'], 'formats': ['i4', 'f4'], 'offsets': [0, 4]})
y = dtype({'names': ['B', 'A'], 'formats': ['f4', 'i4'], 'offsets': [4, 0]})
the expression ``x == y`` will now return ``False``, unlike before.
This makes dictionary based dtype specifications like
``dtype({'a': ('i4', 0), 'b': ('f4', 4)})`` dangerous in python < 3.6
since dict key order is not preserved in those versions.
Assignment from a structured array to a boolean array now raises a ValueError,
unlike in 1.13, where it always set the destination elements to ``True``.
Assignment from structured array with more than one field to a non-structured
array now raises a ValueError. In 1.13 this copied just the first field of the
source to the destination.
Using field "titles" in multiple-field indexing is now disallowed, as is
repeating a field name in a multiple-field index.
The documentation for structured arrays in the user guide has been
significantly updated to reflect these changes.
Integer and Void scalars are now unaffected by ``np.set_string_function``
-------------------------------------------------------------------------
Previously, unlike most other numpy scalars, the ``str`` and ``repr`` of
integer and void scalars could be controlled by ``np.set_string_function``.
This is no longer possible.
0d array printing changed, ``style`` arg of array2string deprecated
-------------------------------------------------------------------
Previously the ``str`` and ``repr`` of 0d arrays had idiosyncratic
implementations which returned ``str(a.item())`` and ``'array(' +
repr(a.item()) + ')'`` respectively for 0d array ``a``, unlike both numpy
scalars and higher dimension ndarrays.
Now, the ``str`` of a 0d array acts like a numpy scalar using ``str(a[()])``
and the ``repr`` acts like higher dimension arrays using ``formatter(a[()])``,
where ``formatter`` can be specified using ``np.set_printoptions``. The
``style`` argument of ``np.array2string`` is deprecated.
This new behavior is disabled in 1.13 legacy printing mode, see compatibility
notes above.
Seeding ``RandomState`` using an array requires a 1-d array
-----------------------------------------------------------
``RandomState`` previously would accept empty arrays or arrays with 2 or more
dimensions, which resulted in either a failure to seed (empty arrays) or for
some of the passed values to be ignored when setting the seed.
``MaskedArray`` objects show a more useful ``repr``
---------------------------------------------------
The ``repr`` of a ``MaskedArray`` is now closer to the python code that would
produce it, with arrays now being shown with commas and dtypes. Like the other
formatting changes, this can be disabled with the 1.13 legacy printing mode in
order to help transition doctests.
The ``repr`` of ``np.polynomial`` classes is more explicit
----------------------------------------------------------
It now shows the domain and window parameters as keyword arguments to make
them more clear::
>>> np.polynomial.Polynomial(range(4))
Polynomial([0., 1., 2., 3.], domain=[-1, 1], window=[-1, 1])
==========================
```
### 1.13.3
```
==========================
This is a bugfix release for some problems found since 1.13.1. The most
important fixes are for CVE-2017-12852 and temporary elision. Users of earlier
versions of 1.13 should upgrade.
The Python versions supported are 2.7 and 3.4 - 3.6. The Python 3.6 wheels
available from PIP are built with Python 3.6.2 and should be compatible with
all previous versions of Python 3.6. It was cythonized with Cython 0.26.1,
which should be free of the bugs found in 0.27 while also being compatible with
Python 3.7-dev. The Windows wheels were built with OpenBlas instead ATLAS,
which should improve the performance of the linear algebra functions.
The NumPy 1.13.3 release is a re-release of 1.13.2, which suffered from a
bug in Cython 0.27.0.
Contributors
============
A total of 12 people contributed to this release. People with a "+" by their
names contributed a patch for the first time.
* Allan Haldane
* Brandon Carter
* Charles Harris
* Eric Wieser
* Iryna Shcherbina +
* James Bourbeau +
* Jonathan Helmus
* Julian Taylor
* Matti Picus
* Michael Lamparski +
* Michael Seifert
* Ralf Gommers
Pull requests merged
====================
A total of 22 pull requests were merged for this release.
* 9390 BUG: Return the poly1d coefficients array directly
* 9555 BUG: Fix regression in 1.13.x in distutils.mingw32ccompiler.
* 9556 BUG: Fix true_divide when dtype=np.float64 specified.
* 9557 DOC: Fix some rst markup in numpy/doc/basics.py.
* 9558 BLD: Remove -xhost flag from IntelFCompiler.
* 9559 DOC: Removes broken docstring example (source code, png, pdf)...
* 9580 BUG: Add hypot and cabs functions to WIN32 blacklist.
* 9732 BUG: Make scalar function elision check if temp is writeable.
* 9736 BUG: Various fixes to np.gradient
* 9742 BUG: Fix np.pad for CVE-2017-12852
* 9744 BUG: Check for exception in sort functions, add tests
* 9745 DOC: Add whitespace after "versionadded::" directive so it actually...
* 9746 BUG: Memory leak in np.dot of size 0
* 9747 BUG: Adjust gfortran version search regex
* 9757 BUG: Cython 0.27 breaks NumPy on Python 3.
* 9764 BUG: Ensure `_npy_scaled_cexp{,f,l}` is defined when needed.
* 9765 BUG: PyArray_CountNonzero does not check for exceptions
* 9766 BUG: Fixes histogram monotonicity check for unsigned bin values
* 9767 BUG: Ensure consistent result dtype of count_nonzero
* 9771 BUG: MAINT: Fix mtrand for Cython 0.27.
* 9772 DOC: Create the 1.13.2 release notes.
* 9794 DOC: Create 1.13.3 release notes.
==========================
```
### 1.13.1
```
==========================
This is a bugfix release for problems found in 1.13.0. The major changes are
fixes for the new memory overlap detection and temporary elision as well as
reversion of the removal of the boolean binary ``-`` operator. Users of 1.13.0
should upgrade.
Thr Python versions supported are 2.7 and 3.4 - 3.6. Note that the Python 3.6
wheels available from PIP are built against 3.6.1, hence will not work when
used with 3.6.0 due to Python bug 29943_. NumPy 1.13.2 will be released shortly
after Python 3.6.2 is out to fix that problem. If you are using 3.6.0 the
workaround is to upgrade to 3.6.1 or use an earlier Python version.
.. _29943: https://bugs.python.org/issue29943
Pull requests merged
====================
A total of 19 pull requests were merged for this release.
* 9240 DOC: BLD: fix lots of Sphinx warnings/errors.
* 9255 Revert "DEP: Raise TypeError for subtract(bool_, bool_)."
* 9261 BUG: don't elide into readonly and updateifcopy temporaries for...
* 9262 BUG: fix missing keyword rename for common block in numpy.f2py
* 9263 BUG: handle resize of 0d array
* 9267 DOC: update f2py front page and some doc build metadata.
* 9299 BUG: Fix Intel compilation on Unix.
* 9317 BUG: fix wrong ndim used in empty where check
* 9319 BUG: Make extensions compilable with MinGW on Py2.7
* 9339 BUG: Prevent crash if ufunc doc string is null
* 9340 BUG: umath: un-break ufunc where= when no out= is given
* 9371 DOC: Add isnat/positive ufunc to documentation
* 9372 BUG: Fix error in fromstring function from numpy.core.records...
* 9373 BUG: ')' is printed at the end pointer of the buffer in numpy.f2py.
* 9374 DOC: Create NumPy 1.13.1 release notes.
* 9376 BUG: Prevent hang traversing ufunc userloop linked list
* 9377 DOC: Use x1 and x2 in the heaviside docstring.
* 9378 DOC: Add $PARAMS to the isnat docstring
* 9379 DOC: Update the 1.13.1 release notes
Contributors
============
A total of 12 people contributed to this release. People with a "+" by their
names contributed a patch for the first time.
* Andras Deak +
* Bob Eldering +
* Charles Harris
* Daniel Hrisca +
* Eric Wieser
* Joshua Leahy +
* Julian Taylor
* Michael Seifert
* Pauli Virtanen
* Ralf Gommers
* Roland Kaufmann
* Warren Weckesser
==========================
```
### 1.13.0
```
==========================
This release supports Python 2.7 and 3.4 - 3.6.
Highlights
==========
* Operations like ``a + b + c`` will reuse temporaries on some platforms,
resulting in less memory use and faster execution.
* Inplace operations check if inputs overlap outputs and create temporaries
to avoid problems.
* New ``__array_ufunc__`` attribute provides improved ability for classes to
override default ufunc behavior.
* New ``np.block`` function for creating blocked arrays.
New functions
=============
* New ``np.positive`` ufunc.
* New ``np.divmod`` ufunc provides more efficient divmod.
* New ``np.isnat`` ufunc tests for NaT special values.
* New ``np.heaviside`` ufunc computes the Heaviside function.
* New ``np.isin`` function, improves on ``in1d``.
* New ``np.block`` function for creating blocked arrays.
* New ``PyArray_MapIterArrayCopyIfOverlap`` added to NumPy C-API.
See below for details.
Deprecations
============
* Calling ``np.fix``, ``np.isposinf``, and ``np.isneginf`` with ``f(x, y=out)``
is deprecated - the argument should be passed as ``f(x, out=out)``, which
matches other ufunc-like interfaces.
* Use of the C-API ``NPY_CHAR`` type number deprecated since version 1.7 will
now raise deprecation warnings at runtime. Extensions built with older f2py
versions need to be recompiled to remove the warning.
* ``np.ma.argsort``, ``np.ma.minimum.reduce``, and ``np.ma.maximum.reduce``
should be called with an explicit `axis` argument when applied to arrays with
more than 2 dimensions, as the default value of this argument (``None``) is
inconsistent with the rest of numpy (``-1``, ``0``, and ``0``, respectively).
* ``np.ma.MaskedArray.mini`` is deprecated, as it almost duplicates the
functionality of ``np.MaskedArray.min``. Exactly equivalent behaviour
can be obtained with ``np.ma.minimum.reduce``.
* The single-argument form of ``np.ma.minimum`` and ``np.ma.maximum`` is
deprecated. ``np.maximum``. ``np.ma.minimum(x)`` should now be spelt
``np.ma.minimum.reduce(x)``, which is consistent with how this would be done
with ``np.minimum``.
* Calling ``ndarray.conjugate`` on non-numeric dtypes is deprecated (it
should match the behavior of ``np.conjugate``, which throws an error).
* Calling ``expand_dims`` when the ``axis`` keyword does not satisfy
``-a.ndim - 1 <= axis <= a.ndim``, where ``a`` is the array being reshaped,
is deprecated.
Future Changes
==============
* Assignment between structured arrays with different field names will change
in NumPy 1.14. Previously, fields in the dst would be set to the value of the
identically-named field in the src. In numpy 1.14 fields will instead be
assigned 'by position': The n-th field of the dst will be set to the n-th
field of the src array. Note that the ``FutureWarning`` raised in NumPy 1.12
incorrectly reported this change as scheduled for NumPy 1.13 rather than
NumPy 1.14.
Build System Changes
====================
* ``numpy.distutils`` now automatically determines C-file dependencies with
GCC compatible compilers.
Compatibility notes
===================
Error type changes
------------------
* ``numpy.hstack()`` now throws ``ValueError`` instead of ``IndexError`` when
input is empty.
* Functions taking an axis argument, when that argument is out of range, now
throw ``np.AxisError`` instead of a mixture of ``IndexError`` and
``ValueError``. For backwards compatibility, ``AxisError`` subclasses both of
these.
Tuple object dtypes
-------------------
Support has been removed for certain obscure dtypes that were unintentionally
allowed, of the form ``(old_dtype, new_dtype)``, where either of the dtypes
is or contains the ``object`` dtype. As an exception, dtypes of the form
``(object, [('name', object)])`` are still supported due to evidence of
existing use.
DeprecationWarning to error
---------------------------
See Changes section for more detail.
* ``partition``, TypeError when non-integer partition index is used.
* ``NpyIter_AdvancedNew``, ValueError when ``oa_ndim == 0`` and ``op_axes`` is NULL
* ``negative(bool_)``, TypeError when negative applied to booleans.
* ``subtract(bool_, bool_)``, TypeError when subtracting boolean from boolean.
* ``np.equal, np.not_equal``, object identity doesn't override failed comparison.
* ``np.equal, np.not_equal``, object identity doesn't override non-boolean comparison.
* Deprecated boolean indexing behavior dropped. See Changes below for details.
* Deprecated ``np.alterdot()`` and ``np.restoredot()`` removed.
FutureWarning to changed behavior
---------------------------------
See Changes section for more detail.
* ``numpy.average`` preserves subclasses
* ``array == None`` and ``array != None`` do element-wise comparison.
* ``np.equal, np.not_equal``, object identity doesn't override comparison result.
dtypes are now always true
--------------------------
Previously ``bool(dtype)`` would fall back to the default python
implementation, which checked if ``len(dtype) > 0``. Since ``dtype`` objects
implement ``__len__`` as the number of record fields, ``bool`` of scalar dtypes
would evaluate to ``False``, which was unintuitive. Now ``bool(dtype) == True``
for all dtypes.
``__getslice__`` and ``__setslice__`` are no longer needed in ``ndarray`` subclasses
------------------------------------------------------------------------------------
When subclassing np.ndarray in Python 2.7, it is no longer _necessary_ to
implement ``__*slice__`` on the derived class, as ``__*item__`` will intercept
these calls correctly.
Any code that did implement these will work exactly as before. Code that
invokes``ndarray.__getslice__`` (e.g. through ``super(...).__getslice__``) will
now issue a DeprecationWarning - ``.__getitem__(slice(start, end))`` should be
used instead.
Indexing MaskedArrays/Constants with ``...`` (ellipsis) now returns MaskedArray
-------------------------------------------------------------------------------
This behavior mirrors that of np.ndarray, and accounts for nested arrays in
MaskedArrays of object dtype, and ellipsis combined with other forms of
indexing.
C API changes
=============
GUfuncs on empty arrays and NpyIter axis removal
------------------------------------------------
It is now allowed to remove a zero-sized axis from NpyIter. Which may mean
that code removing axes from NpyIter has to add an additional check when
accessing the removed dimensions later on.
The largest followup change is that gufuncs are now allowed to have zero-sized
inner dimensions. This means that a gufunc now has to anticipate an empty inner
dimension, while this was never possible and an error raised instead.
For most gufuncs no change should be necessary. However, it is now possible
for gufuncs with a signature such as ``(..., N, M) -> (..., M)`` to return
a valid result if ``N=0`` without further wrapping code.
``PyArray_MapIterArrayCopyIfOverlap`` added to NumPy C-API
----------------------------------------------------------
Similar to ``PyArray_MapIterArray`` but with an additional ``copy_if_overlap``
argument. If ``copy_if_overlap != 0``, checks if input has memory overlap with
any of the other arrays and make copies as appropriate to avoid problems if the
input is modified during the iteration. See the documentation for more complete
documentation.
New Features
============
``__array_ufunc__`` added
-------------------------
This is the renamed and redesigned ``__numpy_ufunc__``. Any class, ndarray
subclass or not, can define this method or set it to ``None`` in order to
override the behavior of NumPy's ufuncs. This works quite similarly to Python's
``__mul__`` and other binary operation routines. See the documentation for a
more detailed description of the implementation and behavior of this new
option. The API is provisional, we do not yet guarantee backward compatibility
as modifications may be made pending feedback. See the NEP_ and
documentation_ for more details.
.. _NEP: https://github.com/numpy/numpy/blob/master/doc/neps/ufunc-overrides.rst
.. _documentation: https://github.com/charris/numpy/blob/master/doc/source/reference/arrays.classes.rst
New ``positive`` ufunc
----------------------
This ufunc corresponds to unary `+`, but unlike `+` on an ndarray it will raise
an error if array values do not support numeric operations.
New ``divmod`` ufunc
--------------------
This ufunc corresponds to the Python builtin `divmod`, and is used to implement
`divmod` when called on numpy arrays. ``np.divmod(x, y)`` calculates a result
equivalent to ``(np.floor_divide(x, y), np.remainder(x, y))`` but is
approximately twice as fast as calling the functions separately.
``np.isnat`` ufunc tests for NaT special datetime and timedelta values
----------------------------------------------------------------------
The new ufunc ``np.isnat`` finds the positions of special NaT values
within datetime and timedelta arrays. This is analogous to ``np.isnan``.
``np.heaviside`` ufunc computes the Heaviside function
------------------------------------------------------
The new function ``np.heaviside(x, h0)`` (a ufunc) computes the Heaviside
function:
.. code::
{ 0 if x < 0,
heaviside(x, h0) = { h0 if x == 0,
{ 1 if x > 0.
``np.block`` function for creating blocked arrays
-------------------------------------------------
Add a new ``block`` function to the current stacking functions ``vstack``,
``hstack``, and ``stack``. This allows concatenation across multiple axes
simultaneously, with a similar syntax to array creation, but where elements
can themselves be arrays. For instance::
>>> A = np.eye(2) * 2
>>> B = np.eye(3) * 3
>>> np.block([
... [A, np.zeros((2, 3))],
... [np.ones((3, 2)), B ]
... ])
array([[ 2., 0., 0., 0., 0.],
[ 0., 2., 0., 0., 0.],
[ 1., 1., 3., 0., 0.],
[ 1., 1., 0., 3., 0.],
[ 1., 1., 0., 0., 3.]])
While primarily useful for block matrices, this works for arbitrary dimensions
of arrays.
It is similar to Matlab's square bracket notation for creating block matrices.
``isin`` function, improving on ``in1d``
----------------------------------------
The new function ``isin`` tests whether each element of an N-dimensonal
array is present anywhere within a second array. It is an enhancement
of ``in1d`` that preserves the shape of the first array.
Temporary elision
-----------------
On platforms providing the ``backtrace`` function NumPy will try to avoid
creating temporaries in expression involving basic numeric types.
For example ``d = a + b + c`` is transformed to ``d = a + b; d += c`` which can
improve performance for large arrays as less memory bandwidth is required to
perform the operation.
``axes`` argument for ``unique``
--------------------------------
In an N-dimensional array, the user can now choose the axis along which to look
for duplicate N-1-dimensional elements using ``numpy.unique``. The original
behaviour is recovered if ``axis=None`` (default).
``np.gradient`` now supports unevenly spaced data
-------------------------------------------------
Users can now specify a not-constant spacing for data.
In particular ``np.gradient`` can now take:
1. A single scalar to specify a sample distance for all dimensions.
2. N scalars to specify a constant sample distance for each dimension.
i.e. ``dx``, ``dy``, ``dz``, ...
3. N arrays to specify the coordinates of the values along each dimension of F.
The length of the array must match the size of the corresponding dimension
4. Any combination of N scalars/arrays with the meaning of 2. and 3.
This means that, e.g., it is now possible to do the following::
>>> f = np.array([[1, 2, 6], [3, 4, 5]], dtype=np.float)
>>> dx = 2.
>>> y = [1., 1.5, 3.5]
>>> np.gradient(f, dx, y)
[array([[ 1. , 1. , -0.5], [ 1. , 1. , -0.5]]),
array([[ 2. , 2. , 2. ], [ 2. , 1.7, 0.5]])]
Support for returning arrays of arbitrary dimensions in ``apply_along_axis``
----------------------------------------------------------------------------
Previously, only scalars or 1D arrays could be returned by the function passed
to ``apply_along_axis``. Now, it can return an array of any dimensionality
(including 0D), and the shape of this array replaces the axis of the array
being iterated over.
``.ndim`` property added to ``dtype`` to complement ``.shape``
--------------------------------------------------------------
For consistency with ``ndarray`` and ``broadcast``, ``d.ndim`` is a shorthand
for ``len(d.shape)``.
Support for tracemalloc in Python 3.6
-------------------------------------
NumPy now supports memory tracing with tracemalloc_ module of Python 3.6 or
newer. Memory allocations from NumPy are placed into the domain defined by
``numpy.lib.tracemalloc_domain``.
Note that NumPy allocation will not show up in tracemalloc_ of earlier Python
versions.
.. _tracemalloc: https://docs.python.org/3/library/tracemalloc.html
NumPy may be built with relaxed stride checking debugging
---------------------------------------------------------
Setting NPY_RELAXED_STRIDES_DEBUG=1 in the environment when relaxed stride
checking is enabled will cause NumPy to be compiled with the affected strides
set to the maximum value of npy_intp in order to help detect invalid usage of
the strides in downstream projects. When enabled, invalid usage often results
in an error being raised, but the exact type of error depends on the details of
the code. TypeError and OverflowError have been observed in the wild.
It was previously the case that this option was disabled for releases and
enabled in master and changing between the two required editing the code. It is
now disabled by default but can be enabled for test builds.
Improvements
============
Ufunc behavior for overlapping inputs
-------------------------------------
Operations where ufunc input and output operands have memory overlap
produced undefined results in previous NumPy versions, due to data
dependency issues. In NumPy 1.13.0, results from such operations are
now defined to be the same as for equivalent operations where there is
no memory overlap.
Operations affected now make temporary copies, as needed to eliminate
data dependency. As detecting these cases is computationally
expensive, a heuristic is used, which may in rare cases result to
needless temporary copies. For operations where the data dependency
is simple enough for the heuristic to analyze, temporary copies will
not be made even if the arrays overlap, if it can be deduced copies
are not necessary. As an example,``np.add(a, b, out=a)`` will not
involve copies.
To illustrate a previously undefined operation::
>>> x = np.arange(16).astype(float)
>>> np.add(x[1:], x[:-1], out=x[1:])
In NumPy 1.13.0 the last line is guaranteed to be equivalent to::
>>> np.add(x[1:].copy(), x[:-1].copy(), out=x[1:])
A similar operation with simple non-problematic data dependence is::
>>> x = np.arange(16).astype(float)
>>> np.add(x[1:], x[:-1], out=x[:-1])
It will continue to produce the same results as in previous NumPy
versions, and will not involve unnecessary temporary copies.
The change applies also to in-place binary operations, for example::
>>> x = np.random.rand(500, 500)
>>> x += x.T
This statement is now guaranteed to be equivalent to ``x[...] = x + x.T``,
whereas in previous NumPy versions the results were undefined.
Partial support for 64-bit f2py extensions with MinGW
-----------------------------------------------------
Extensions that incorporate Fortran libraries can now be built using the free
MinGW_ toolset, also under Python 3.5. This works best for extensions that only
do calculations and uses the runtime modestly (reading and writing from files,
for instance). Note that this does not remove the need for Mingwpy; if you make
extensive use of the runtime, you will most likely run into issues_. Instead,
it should be regarded as a band-aid until Mingwpy is fully functional.
Extensions can also be compiled using the MinGW toolset using the runtime
library from the (moveable) WinPython 3.4 distribution, which can be useful for
programs with a PySide1/Qt4 front-end.
.. _MinGW: https://sf.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/6.2.0/threads-win32/seh/
.. _issues: https://mingwpy.github.io/issues.html
Performance improvements for ``packbits`` and ``unpackbits``
------------------------------------------------------------
The functions ``numpy.packbits`` with boolean input and ``numpy.unpackbits`` have
been optimized to be a significantly faster for contiguous data.
Fix for PPC long double floating point information
--------------------------------------------------
In previous versions of NumPy, the ``finfo`` function returned invalid
information about the `double double`_ format of the ``longdouble`` float type
on Power PC (PPC). The invalid values resulted from the failure of the NumPy
algorithm to deal with the variable number of digits in the significand
that are a feature of `PPC long doubles`. This release by-passes the failing
algorithm by using heuristics to detect the presence of the PPC double double
format. A side-effect of using these heuristics is that the ``finfo``
function is faster than previous releases.
.. _PPC long doubles: https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.genprogc/128bit_long_double_floating-point_datatype.htm
.. _double double: https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_formatDouble-double_arithmetic
Better default repr for ``ndarray`` subclasses
----------------------------------------------
Subclasses of ndarray with no ``repr`` specialization now correctly indent
their data and type lines.
More reliable comparisons of masked arrays
------------------------------------------
Comparisons of masked arrays were buggy for masked scalars and failed for
structured arrays with dimension higher than one. Both problems are now
solved. In the process, it was ensured that in getting the result for a
structured array, masked fields are properly ignored, i.e., the result is equal
if all fields that are non-masked in both are equal, thus making the behaviour
identical to what one gets by comparing an unstructured masked array and then
doing ``.all()`` over some axis.
np.matrix with booleans elements can now be created using the string syntax
---------------------------------------------------------------------------
``np.matrix`` failed whenever one attempts to use it with booleans, e.g.,
``np.matrix('True')``. Now, this works as expected.
More ``linalg`` operations now accept empty vectors and matrices
----------------------------------------------------------------
All of the following functions in ``np.linalg`` now work when given input
arrays with a 0 in the last two dimensions: ``det``, ``slogdet``, ``pinv``,
``eigvals``, ``eigvalsh``, ``eig``, ``eigh``.
Bundled version of LAPACK is now 3.2.2
--------------------------------------
NumPy comes bundled with a minimal implementation of lapack for systems without
a lapack library installed, under the name of ``lapack_lite``. This has been
upgraded from LAPACK 3.0.0 (June 30, 1999) to LAPACK 3.2.2 (June 30, 2010). See
the `LAPACK changelogs`_ for details on the all the changes this entails.
While no new features are exposed through ``numpy``, this fixes some bugs
regarding "workspace" sizes, and in some places may use faster algorithms.
.. _`LAPACK changelogs`: http://www.netlib.org/lapack/release_notes.html_4_history_of_lapack_releases
``reduce`` of ``np.hypot.reduce`` and ``np.logical_xor`` allowed in more cases
------------------------------------------------------------------------------
This now works on empty arrays, returning 0, and can reduce over multiple axes.
Previously, a ``ValueError`` was thrown in these cases.
Better ``repr`` of object arrays
--------------------------------
Object arrays that contain themselves no longer cause a recursion error.
Object arrays that contain ``list`` objects are now printed in a way that makes
clear the difference between a 2d object array, and a 1d object array of lists.
Changes
=======
``argsort`` on masked arrays takes the same default arguments as ``sort``
-------------------------------------------------------------------------
By default, ``argsort`` now places the masked values at the end of the sorted
array, in the same way that ``sort`` already did. Additionally, the
``end_with`` argument is added to ``argsort``, for consistency with ``sort``.
Note that this argument is not added at the end, so breaks any code that
passed ``fill_value`` as a positional argument.
``average`` now preserves subclasses
------------------------------------
For ndarray subclasses, ``numpy.average`` will now return an instance of the
subclass, matching the behavior of most other NumPy functions such as ``mean``.
As a consequence, also calls that returned a scalar may now return a subclass
array scalar.
``array == None`` and ``array != None`` do element-wise comparison
----------------------------------------
pyup-bot
commented
6 years ago
Update numpy from 1.12.1 to 1.14.2.
Changelog
### 1.14.1 ``` ========================== This is a bugfix release for some problems reported following the 1.14.0 release. The major problems fixed are the following. * Problems with the new array printing, particularly the printing of complex values, Please report any additional problems that may turn up. * Problems with ``np.einsum`` due to the new ``optimized=True`` default. Some fixes for optimization have been applied and ``optimize=False`` is now the default. * The sort order in ``np.unique`` when ``axis=<some-number>`` will now always be lexicographic in the subarray elements. In previous NumPy versions there was an optimization that could result in sorting the subarrays as unsigned byte strings. * The change in 1.14.0 that multi-field indexing of structured arrays returns a view instead of a copy has been reverted but remains on track for NumPy 1.15. Affected users should read the 1.14.1 Numpy User Guide section "basics/structured arrays/accessing multiple fields" for advice on how to manage this transition. The Python versions supported in this release are 2.7 and 3.4 - 3.6. The Python 3.6 wheels available from PIP are built with Python 3.6.2 and should be compatible with all previous versions of Python 3.6. The source releases were cythonized with Cython 0.26.1, which is known to **not** support the upcoming Python 3.7 release. People who wish to run Python 3.7 should check out the NumPy repo and try building with the, as yet, unreleased master branch of Cython. Contributors ============ A total of 14 people contributed to this release. People with a "+" by their names contributed a patch for the first time. * Allan Haldane * Charles Harris * Daniel Smith * Dennis Weyland + * Eric Larson * Eric Wieser * Jarrod Millman * Kenichi Maehashi + * Marten van Kerkwijk * Mathieu Lamarre * Sebastian Berg * Simon Conseil * Simon Gibbons * xoviat Pull requests merged ==================== A total of 36 pull requests were merged for this release. * `10339 <https://github.com/numpy/numpy/pull/10339>`__: BUG: restrict the __config__ modifications to win32 * `10368 <https://github.com/numpy/numpy/pull/10368>`__: MAINT: Adjust type promotion in linalg.norm * `10375 <https://github.com/numpy/numpy/pull/10375>`__: BUG: add missing paren and remove quotes from repr of fieldless... * `10395 <https://github.com/numpy/numpy/pull/10395>`__: MAINT: Update download URL in setup.py. * `10396 <https://github.com/numpy/numpy/pull/10396>`__: BUG: fix einsum issue with unicode input and py2 * `10397 <https://github.com/numpy/numpy/pull/10397>`__: BUG: fix error message not formatted in einsum * `10398 <https://github.com/numpy/numpy/pull/10398>`__: DOC: add documentation about how to handle new array printing * `10403 <https://github.com/numpy/numpy/pull/10403>`__: BUG: Set einsum optimize parameter default to `False`. * `10424 <https://github.com/numpy/numpy/pull/10424>`__: ENH: Fix repr of np.record objects to match np.void types 10412 * `10425 <https://github.com/numpy/numpy/pull/10425>`__: MAINT: Update zesty to artful for i386 testing * `10431 <https://github.com/numpy/numpy/pull/10431>`__: REL: Add 1.14.1 release notes template * `10435 <https://github.com/numpy/numpy/pull/10435>`__: MAINT: Use ValueError for duplicate field names in lookup (backport) * `10534 <https://github.com/numpy/numpy/pull/10534>`__: BUG: Provide a better error message for out-of-order fields * `10536 <https://github.com/numpy/numpy/pull/10536>`__: BUG: Resize bytes_ columns in genfromtxt (backport of 10401) * `10537 <https://github.com/numpy/numpy/pull/10537>`__: BUG: multifield-indexing adds padding bytes: revert for 1.14.1 * `10539 <https://github.com/numpy/numpy/pull/10539>`__: BUG: fix np.save issue with python 2.7.5 * `10540 <https://github.com/numpy/numpy/pull/10540>`__: BUG: Add missing DECREF in Py2 int() cast * `10541 <https://github.com/numpy/numpy/pull/10541>`__: TST: Add circleci document testing to maintenance/1.14.x * `10542 <https://github.com/numpy/numpy/pull/10542>`__: BUG: complex repr has extra spaces, missing + (1.14 backport) * `10550 <https://github.com/numpy/numpy/pull/10550>`__: BUG: Set missing exception after malloc * `10557 <https://github.com/numpy/numpy/pull/10557>`__: BUG: In numpy.i, clear CARRAY flag if wrapped buffer is not C_CONTIGUOUS. * `10558 <https://github.com/numpy/numpy/pull/10558>`__: DEP: Issue FutureWarning when malformed records detected. * `10559 <https://github.com/numpy/numpy/pull/10559>`__: BUG: Fix einsum optimize logic for singleton dimensions * `10560 <https://github.com/numpy/numpy/pull/10560>`__: BUG: Fix calling ufuncs with a positional output argument. * `10561 <https://github.com/numpy/numpy/pull/10561>`__: BUG: Fix various Big-Endian test failures (ppc64) * `10562 <https://github.com/numpy/numpy/pull/10562>`__: BUG: Make dtype.descr error for out-of-order fields. * `10563 <https://github.com/numpy/numpy/pull/10563>`__: BUG: arrays not being flattened in `union1d` * `10607 <https://github.com/numpy/numpy/pull/10607>`__: MAINT: Update sphinxext submodule hash. * `10608 <https://github.com/numpy/numpy/pull/10608>`__: BUG: Revert sort optimization in np.unique. * `10609 <https://github.com/numpy/numpy/pull/10609>`__: BUG: infinite recursion in str of 0d subclasses * `10610 <https://github.com/numpy/numpy/pull/10610>`__: BUG: Align type definition with generated lapack * `10612 <https://github.com/numpy/numpy/pull/10612>`__: BUG/ENH: Improve output for structured non-void types * `10622 <https://github.com/numpy/numpy/pull/10622>`__: BUG: deallocate recursive closure in arrayprint.py (1.14 backport) * `10624 <https://github.com/numpy/numpy/pull/10624>`__: BUG: Correctly identify comma separated dtype strings * `10629 <https://github.com/numpy/numpy/pull/10629>`__: BUG: deallocate recursive closure in arrayprint.py (backport... * `10630 <https://github.com/numpy/numpy/pull/10630>`__: REL: Prepare for 1.14.1 release. ========================= ``` ### 1.14.0 ``` ========================== Numpy 1.14.0 is the result of seven months of work and contains a large number of bug fixes and new features, along with several changes with potential compatibility issues. The major change that users will notice are the stylistic changes in the way numpy arrays and scalars are printed, a change that will affect doctests. See below for details on how to preserve the old style printing when needed. A major decision affecting future development concerns the schedule for dropping Python 2.7 support in the runup to 2020. The decision has been made to support 2.7 for all releases made in 2018, with the last release being designated a long term release with support for bug fixes extending through 2019. In 2019 support for 2.7 will be dropped in all new releases. More details can be found in the relevant NEP_. This release supports Python 2.7 and 3.4 - 3.6. .. _NEP: https://github.com/numpy/numpy/blob/master/doc/neps/dropping-python2.7-proposal.rst Highlights ========== * The `np.einsum` function uses BLAS when possible * ``genfromtxt``, ``loadtxt``, ``fromregex`` and ``savetxt`` can now handle files with arbitrary Python supported encoding. * Major improvements to printing of NumPy arrays and scalars. New functions ============= * ``parametrize``: decorator added to numpy.testing * ``chebinterpolate``: Interpolate function at Chebyshev points. * ``format_float_positional`` and ``format_float_scientific`` : format floating-point scalars unambiguously with control of rounding and padding. * ``PyArray_ResolveWritebackIfCopy`` and ``PyArray_SetWritebackIfCopyBase``, new C-API functions useful in achieving PyPy compatibity. Deprecations ============ * Using ``np.bool_`` objects in place of integers is deprecated. Previously ``operator.index(np.bool_)`` was legal and allowed constructs such as ``[1, 2, 3][np.True_]``. That was misleading, as it behaved differently from ``np.array([1, 2, 3])[np.True_]``. * Truth testing of an empty array is deprecated. To check if an array is not empty, use ``array.size > 0``. * Calling ``np.bincount`` with ``minlength=None`` is deprecated. ``minlength=0`` should be used instead. * Calling ``np.fromstring`` with the default value of the ``sep`` argument is deprecated. When that argument is not provided, a broken version of ``np.frombuffer`` is used that silently accepts unicode strings and -- after encoding them as either utf-8 (python 3) or the default encoding (python 2) -- treats them as binary data. If reading binary data is desired, ``np.frombuffer`` should be used directly. * The ``style`` option of array2string is deprecated in non-legacy printing mode. * ``PyArray_SetUpdateIfCopyBase`` has been deprecated. For NumPy versions >= 1.14 use ``PyArray_SetWritebackIfCopyBase`` instead, see `C API changes` below for more details. * The use of ``UPDATEIFCOPY`` arrays is deprecated, see `C API changes` below for details. We will not be dropping support for those arrays, but they are not compatible with PyPy. Future Changes ============== * ``np.issubdtype`` will stop downcasting dtype-like arguments. It might be expected that ``issubdtype(np.float32, 'float64')`` and ``issubdtype(np.float32, np.float64)`` mean the same thing - however, there was an undocumented special case that translated the former into ``issubdtype(np.float32, np.floating)``, giving the surprising result of True. This translation now gives a warning that explains what translation is occurring. In the future, the translation will be disabled, and the first example will be made equivalent to the second. * ``np.linalg.lstsq`` default for ``rcond`` will be changed. The ``rcond`` parameter to ``np.linalg.lstsq`` will change its default to machine precision times the largest of the input array dimensions. A FutureWarning is issued when ``rcond`` is not passed explicitly. * ``a.flat.__array__()`` will return a writeable copy of ``a`` when ``a`` is non-contiguous. Previously it returned an UPDATEIFCOPY array when ``a`` was writeable. Currently it returns a non-writeable copy. See gh-7054 for a discussion of the issue. * Unstructured void array's ``.item`` method will return a bytes object. In the future, calling ``.item()`` on arrays or scalars of ``np.void`` datatype will return a ``bytes`` object instead of a buffer or int array, the same as returned by ``bytes(void_scalar)``. This may affect code which assumed the return value was mutable, which will no longer be the case. A ``FutureWarning`` is now issued when this would occur. Compatibility notes =================== The mask of a masked array view is also a view rather than a copy ----------------------------------------------------------------- There was a FutureWarning about this change in NumPy 1.11.x. In short, it is now the case that, when changing a view of a masked array, changes to the mask are propagated to the original. That was not previously the case. This change affects slices in particular. Note that this does not yet work properly if the mask of the original array is ``nomask`` and the mask of the view is changed. See gh-5580 for an extended discussion. The original behavior of having a copy of the mask can be obtained by calling the ``unshare_mask`` method of the view. ``np.ma.masked`` is no longer writeable --------------------------------------- Attempts to mutate the ``masked`` constant now error, as the underlying arrays are marked readonly. In the past, it was possible to get away with:: emulating a function that sometimes returns np.ma.masked val = random.choice([np.ma.masked, 10]) var_arr = np.asarray(val) val_arr += 1 now errors, previously changed np.ma.masked.data ``np.ma`` functions producing ``fill_value``s have changed ---------------------------------------------------------- Previously, ``np.ma.default_fill_value`` would return a 0d array, but ``np.ma.minimum_fill_value`` and ``np.ma.maximum_fill_value`` would return a tuple of the fields. Instead, all three methods return a structured ``np.void`` object, which is what you would already find in the ``.fill_value`` attribute. Additionally, the dtype guessing now matches that of ``np.array`` - so when passing a python scalar ``x``, ``maximum_fill_value(x)`` is always the same as ``maximum_fill_value(np.array(x))``. Previously ``x = long(1)`` on Python 2 violated this assumption. ``a.flat.__array__()`` returns non-writeable arrays when ``a`` is non-contiguous -------------------------------------------------------------------------------- The intent is that the UPDATEIFCOPY array previously returned when ``a`` was non-contiguous will be replaced by a writeable copy in the future. This temporary measure is aimed to notify folks who expect the underlying array be modified in this situation that that will no longer be the case. The most likely places for this to be noticed is when expressions of the form ``np.asarray(a.flat)`` are used, or when ``a.flat`` is passed as the out parameter to a ufunc. ``np.tensordot`` now returns zero array when contracting over 0-length dimension -------------------------------------------------------------------------------- Previously ``np.tensordot`` raised a ValueError when contracting over 0-length dimension. Now it returns a zero array, which is consistent with the behaviour of ``np.dot`` and ``np.einsum``. ``numpy.testing`` reorganized ----------------------------- This is not expected to cause problems, but possibly something has been left out. If you experience an unexpected import problem using ``numpy.testing`` let us know. ``np.asfarray`` no longer accepts non-dtypes through the ``dtype`` argument --------------------------------------------------------------------------- This previously would accept ``dtype=some_array``, with the implied semantics of ``dtype=some_array.dtype``. This was undocumented, unique across the numpy functions, and if used would likely correspond to a typo. 1D ``np.linalg.norm`` preserves float input types, even for arbitrary orders ---------------------------------------------------------------------------- Previously, this would promote to ``float64`` when arbitrary orders were passed, despite not doing so under the simple cases:: >>> f32 = np.float32([[1, 2]]) >>> np.linalg.norm(f32, 2.0, axis=-1).dtype dtype('float32') >>> np.linalg.norm(f32, 2.0001, axis=-1).dtype dtype('float64') numpy 1.13 dtype('float32') numpy 1.14 This change affects only ``float32`` and ``float16`` arrays. ``count_nonzero(arr, axis=())`` now counts over no axes, not all axes --------------------------------------------------------------------- Elsewhere, ``axis==()`` is always understood as "no axes", but `count_nonzero` had a special case to treat this as "all axes". This was inconsistent and surprising. The correct way to count over all axes has always been to pass ``axis == None``. ``__init__.py`` files added to test directories ----------------------------------------------- This is for pytest compatibility in the case of duplicate test file names in the different directories. As a result, ``run_module_suite`` no longer works, i.e., ``python <path-to-test-file>`` results in an error. ``.astype(bool)`` on unstructured void arrays now calls ``bool`` on each element -------------------------------------------------------------------------------- On Python 2, ``void_array.astype(bool)`` would always return an array of ``True``, unless the dtype is ``V0``. On Python 3, this operation would usually crash. Going forwards, `astype` matches the behavior of ``bool(np.void)``, considering a buffer of all zeros as false, and anything else as true. Checks for ``V0`` can still be done with ``arr.dtype.itemsize == 0``. ``MaskedArray.squeeze`` never returns ``np.ma.masked`` ------------------------------------------------------ ``np.squeeze`` is documented as returning a view, but the masked variant would sometimes return ``masked``, which is not a view. This has been fixed, so that the result is always a view on the original masked array. This breaks any code that used ``masked_arr.squeeze() is np.ma.masked``, but fixes code that writes to the result of `.squeeze()`. Renamed first parameter of ``can_cast`` from ``from`` to ``from_`` ------------------------------------------------------------------ The previous parameter name ``from`` is a reserved keyword in Python, which made it difficult to pass the argument by name. This has been fixed by renaming the parameter to ``from_``. ``isnat`` raises ``TypeError`` when passed wrong type ------------------------------------------------------ The ufunc ``isnat`` used to raise a ``ValueError`` when it was not passed variables of type ``datetime`` or ``timedelta``. This has been changed to raising a ``TypeError``. ``dtype.__getitem__`` raises ``TypeError`` when passed wrong type ----------------------------------------------------------------- When indexed with a float, the dtype object used to raise ``ValueError``. User-defined types now need to implement ``__str__`` and ``__repr__`` --------------------------------------------------------------------- Previously, user-defined types could fall back to a default implementation of ``__str__`` and ``__repr__`` implemented in numpy, but this has now been removed. Now user-defined types will fall back to the python default ``object.__str__`` and ``object.__repr__``. Many changes to array printing, disableable with the new "legacy" printing mode ------------------------------------------------------------------------------- The ``str`` and ``repr`` of ndarrays and numpy scalars have been changed in a variety of ways. These changes are likely to break downstream user's doctests. These new behaviors can be disabled to mostly reproduce numpy 1.13 behavior by enabling the new 1.13 "legacy" printing mode. This is enabled by calling ``np.set_printoptions(legacy="1.13")``, or using the new ``legacy`` argument to ``np.array2string``, as ``np.array2string(arr, legacy='1.13')``. In summary, the major changes are: * For floating-point types: * The ``repr`` of float arrays often omits a space previously printed in the sign position. See the new ``sign`` option to ``np.set_printoptions``. * Floating-point arrays and scalars use a new algorithm for decimal representations, giving the shortest unique representation. This will usually shorten ``float16`` fractional output, and sometimes ``float32`` and ``float128`` output. ``float64`` should be unaffected. See the new ``floatmode`` option to ``np.set_printoptions``. * Float arrays printed in scientific notation no longer use fixed-precision, and now instead show the shortest unique representation. * The ``str`` of floating-point scalars is no longer truncated in python2. * For other data types: * Non-finite complex scalars print like ``nanj`` instead of ``nan*j``. * ``NaT`` values in datetime arrays are now properly aligned. * Arrays and scalars of ``np.void`` datatype are now printed using hex notation. * For line-wrapping: * The "dtype" part of ndarray reprs will now be printed on the next line if there isn't space on the last line of array output. * The ``linewidth`` format option is now always respected. The `repr` or `str` of an array will never exceed this, unless a single element is too wide. * The last line of an array string will never have more elements than earlier lines. * An extra space is no longer inserted on the first line if the elements are too wide. * For summarization (the use of ``...`` to shorten long arrays): * A trailing comma is no longer inserted for ``str``. Previously, ``str(np.arange(1001))`` gave ``'[ 0 1 2 ..., 998 999 1000]'``, which has an extra comma. * For arrays of 2-D and beyond, when ``...`` is printed on its own line in order to summarize any but the last axis, newlines are now appended to that line to match its leading newlines and a trailing space character is removed. * ``MaskedArray`` arrays now separate printed elements with commas, always print the dtype, and correctly wrap the elements of long arrays to multiple lines. If there is more than 1 dimension, the array attributes are now printed in a new "left-justified" printing style. * ``recarray`` arrays no longer print a trailing space before their dtype, and wrap to the right number of columns. * 0d arrays no longer have their own idiosyncratic implementations of ``str`` and ``repr``. The ``style`` argument to ``np.array2string`` is deprecated. * Arrays of ``bool`` datatype will omit the datatype in the ``repr``. * User-defined ``dtypes`` (subclasses of ``np.generic``) now need to implement ``__str__`` and ``__repr__``. Some of these changes are described in more detail below. If you need to retain the previous behavior for doctests or other reasons, you may want to do something like:: FIXME: We need the str/repr formatting used in Numpy < 1.14. try: np.set_printoptions(legacy='1.13') except TypeError: pass C API changes ============= PyPy compatible alternative to ``UPDATEIFCOPY`` arrays ------------------------------------------------------ ``UPDATEIFCOPY`` arrays are contiguous copies of existing arrays, possibly with different dimensions, whose contents are copied back to the original array when their refcount goes to zero and they are deallocated. Because PyPy does not use refcounts, they do not function correctly with PyPy. NumPy is in the process of eliminating their use internally and two new C-API functions, * ``PyArray_SetWritebackIfCopyBase`` * ``PyArray_ResolveWritebackIfCopy``, have been added together with a complimentary flag, ``NPY_ARRAY_WRITEBACKIFCOPY``. Using the new functionality also requires that some flags be changed when new arrays are created, to wit: ``NPY_ARRAY_INOUT_ARRAY`` should be replaced by ``NPY_ARRAY_INOUT_ARRAY2`` and ``NPY_ARRAY_INOUT_FARRAY`` should be replaced by ``NPY_ARRAY_INOUT_FARRAY2``. Arrays created with these new flags will then have the ``WRITEBACKIFCOPY`` semantics. If PyPy compatibility is not a concern, these new functions can be ignored, although there will be a ``DeprecationWarning``. If you do wish to pursue PyPy compatibility, more information on these functions and their use may be found in the c-api_ documentation and the example in how-to-extend_. .. _c-api: https://github.com/numpy/numpy/blob/master/doc/source/reference/c-api.array.rst .. _how-to-extend: https://github.com/numpy/numpy/blob/master/doc/source/user/c-info.how-to-extend.rst New Features ============ Encoding argument for text IO functions --------------------------------------- ``genfromtxt``, ``loadtxt``, ``fromregex`` and ``savetxt`` can now handle files with arbitrary encoding supported by Python via the encoding argument. For backward compatibility the argument defaults to the special ``bytes`` value which continues to treat text as raw byte values and continues to pass latin1 encoded bytes to custom converters. Using any other value (including ``None`` for system default) will switch the functions to real text IO so one receives unicode strings instead of bytes in the resulting arrays. External ``nose`` plugins are usable by ``numpy.testing.Tester`` ---------------------------------------------------------------- ``numpy.testing.Tester`` is now aware of ``nose`` plugins that are outside the ``nose`` built-in ones. This allows using, for example, ``nose-timer`` like so: ``np.test(extra_argv=['--with-timer', '--timer-top-n', '20'])`` to obtain the runtime of the 20 slowest tests. An extra keyword ``timer`` was also added to ``Tester.test``, so ``np.test(timer=20)`` will also report the 20 slowest tests. ``parametrize`` decorator added to ``numpy.testing`` ---------------------------------------------------- A basic ``parametrize`` decorator is now available in ``numpy.testing``. It is intended to allow rewriting yield based tests that have been deprecated in pytest so as to facilitate the transition to pytest in the future. The nose testing framework has not been supported for several years and looks like abandonware. The new ``parametrize`` decorator does not have the full functionality of the one in pytest. It doesn't work for classes, doesn't support nesting, and does not substitute variable names. Even so, it should be adequate to rewrite the NumPy tests. ``chebinterpolate`` function added to ``numpy.polynomial.chebyshev`` -------------------------------------------------------------------- The new ``chebinterpolate`` function interpolates a given function at the Chebyshev points of the first kind. A new ``Chebyshev.interpolate`` class method adds support for interpolation over arbitrary intervals using the scaled and shifted Chebyshev points of the first kind. Support for reading lzma compressed text files in Python 3 ---------------------------------------------------------- With Python versions containing the ``lzma`` module the text IO functions can now transparently read from files with ``xz`` or ``lzma`` extension. ``sign`` option added to ``np.setprintoptions`` and ``np.array2string`` ----------------------------------------------------------------------- This option controls printing of the sign of floating-point types, and may be one of the characters '-', '+' or ' '. With '+' numpy always prints the sign of positive values, with ' ' it always prints a space (whitespace character) in the sign position of positive values, and with '-' it will omit the sign character for positive values. The new default is '-'. This new default changes the float output relative to numpy 1.13. The old behavior can be obtained in 1.13 "legacy" printing mode, see compatibility notes above. ``hermitian`` option added to``np.linalg.matrix_rank`` ------------------------------------------------------ The new ``hermitian`` option allows choosing between standard SVD based matrix rank calculation and the more efficient eigenvalue based method for symmetric/hermitian matrices. ``threshold`` and ``edgeitems`` options added to ``np.array2string`` -------------------------------------------------------------------- These options could previously be controlled using ``np.set_printoptions``, but now can be changed on a per-call basis as arguments to ``np.array2string``. ``concatenate`` and ``stack`` gained an ``out`` argument -------------------------------------------------------- A preallocated buffer of the desired dtype can now be used for the output of these functions. Support for PGI flang compiler on Windows ----------------------------------------- The PGI flang compiler is a Fortran front end for LLVM released by NVIDIA under the Apache 2 license. It can be invoked by :: python setup.py config --compiler=clang --fcompiler=flang install There is little experience with this new compiler, so any feedback from people using it will be appreciated. Improvements ============ Numerator degrees of freedom in ``random.noncentral_f`` need only be positive. ------------------------------------------------------------------------------ Prior to NumPy 1.14.0, the numerator degrees of freedom needed to be > 1, but the distribution is valid for values > 0, which is the new requirement. The GIL is released for all ``np.einsum`` variations ---------------------------------------------------- Some specific loop structures which have an accelerated loop version did not release the GIL prior to NumPy 1.14.0. This oversight has been fixed. The `np.einsum` function will use BLAS when possible and optimize by default ---------------------------------------------------------------------------- The ``np.einsum`` function will now call ``np.tensordot`` when appropriate. Because ``np.tensordot`` uses BLAS when possible, that will speed up execution. By default, ``np.einsum`` will also attempt optimization as the overhead is small relative to the potential improvement in speed. ``f2py`` now handles arrays of dimension 0 ------------------------------------------ ``f2py`` now allows for the allocation of arrays of dimension 0. This allows for more consistent handling of corner cases downstream. ``numpy.distutils`` supports using MSVC and mingw64-gfortran together --------------------------------------------------------------------- Numpy distutils now supports using Mingw64 gfortran and MSVC compilers together. This enables the production of Python extension modules on Windows containing Fortran code while retaining compatibility with the binaries distributed by Python.org. Not all use cases are supported, but most common ways to wrap Fortran for Python are functional. Compilation in this mode is usually enabled automatically, and can be selected via the ``--fcompiler`` and ``--compiler`` options to ``setup.py``. Moreover, linking Fortran codes to static OpenBLAS is supported; by default a gfortran compatible static archive ``openblas.a`` is looked for. ``np.linalg.pinv`` now works on stacked matrices ------------------------------------------------ Previously it was limited to a single 2d array. ``numpy.save`` aligns data to 64 bytes instead of 16 ---------------------------------------------------- Saving NumPy arrays in the ``npy`` format with ``numpy.save`` inserts padding before the array data to align it at 64 bytes. Previously this was only 16 bytes (and sometimes less due to a bug in the code for version 2). Now the alignment is 64 bytes, which matches the widest SIMD instruction set commonly available, and is also the most common cache line size. This makes ``npy`` files easier to use in programs which open them with ``mmap``, especially on Linux where an ``mmap`` offset must be a multiple of the page size. NPZ files now can be written without using temporary files ---------------------------------------------------------- In Python 3.6+ ``numpy.savez`` and ``numpy.savez_compressed`` now write directly to a ZIP file, without creating intermediate temporary files. Better support for empty structured and string types ---------------------------------------------------- Structured types can contain zero fields, and string dtypes can contain zero characters. Zero-length strings still cannot be created directly, and must be constructed through structured dtypes:: str0 = np.empty(10, np.dtype([('v', str, N)]))['v'] void0 = np.empty(10, np.void) It was always possible to work with these, but the following operations are now supported for these arrays: * `arr.sort()` * `arr.view(bytes)` * `arr.resize(...)` * `pickle.dumps(arr)` Support for ``decimal.Decimal`` in ``np.lib.financial`` ------------------------------------------------------- Unless otherwise stated all functions within the ``financial`` package now support using the ``decimal.Decimal`` built-in type. Float printing now uses "dragon4" algorithm for shortest decimal representation ------------------------------------------------------------------------------- The ``str`` and ``repr`` of floating-point values (16, 32, 64 and 128 bit) are now printed to give the shortest decimal representation which uniquely identifies the value from others of the same type. Previously this was only true for ``float64`` values. The remaining float types will now often be shorter than in numpy 1.13. Arrays printed in scientific notation now also use the shortest scientific representation, instead of fixed precision as before. Additionally, the `str` of float scalars scalars will no longer be truncated in python2, unlike python2 `float`s. `np.double` scalars now have a ``str`` and ``repr`` identical to that of a python3 float. New functions ``np.format_float_scientific`` and ``np.format_float_positional`` are provided to generate these decimal representations. A new option ``floatmode`` has been added to ``np.set_printoptions`` and ``np.array2string``, which gives control over uniqueness and rounding of printed elements in an array. The new default is ``floatmode='maxprec'`` with ``precision=8``, which will print at most 8 fractional digits, or fewer if an element can be uniquely represented with fewer. A useful new mode is ``floatmode="unique"``, which will output enough digits to specify the array elements uniquely. Numpy complex-floating-scalars with values like ``inf*j`` or ``nan*j`` now print as ``infj`` and ``nanj``, like the pure-python ``complex`` type. The ``FloatFormat`` and ``LongFloatFormat`` classes are deprecated and should both be replaced by ``FloatingFormat``. Similarly ``ComplexFormat`` and ``LongComplexFormat`` should be replaced by ``ComplexFloatingFormat``. ``void`` datatype elements are now printed in hex notation ---------------------------------------------------------- A hex representation compatible with the python ``bytes`` type is now printed for unstructured ``np.void`` elements, e.g., ``V4`` datatype. Previously, in python2 the raw void data of the element was printed to stdout, or in python3 the integer byte values were shown. printing style for ``void`` datatypes is now independently customizable ----------------------------------------------------------------------- The printing style of ``np.void`` arrays is now independently customizable using the ``formatter`` argument to ``np.set_printoptions``, using the ``'void'`` key, instead of the catch-all ``numpystr`` key as before. Reduced memory usage of ``np.loadtxt`` -------------------------------------- ``np.loadtxt`` now reads files in chunks instead of all at once which decreases its memory usage significantly for large files. Changes ======= Multiple-field indexing/assignment of structured arrays ------------------------------------------------------- The indexing and assignment of structured arrays with multiple fields has changed in a number of ways, as warned about in previous releases. First, indexing a structured array with multiple fields, e.g., ``arr[['f1', 'f3']]``, returns a view into the original array instead of a copy. The returned view will have extra padding bytes corresponding to intervening fields in the original array, unlike the copy in 1.13, which will affect code such as ``arr[['f1', 'f3']].view(newdtype)``. Second, assignment between structured arrays will now occur "by position" instead of "by field name". The Nth field of the destination will be set to the Nth field of the source regardless of field name, unlike in numpy versions 1.6 to 1.13 in which fields in the destination array were set to the identically-named field in the source array or to 0 if the source did not have a field. Correspondingly, the order of fields in a structured dtypes now matters when computing dtype equality. For example, with the dtypes :: x = dtype({'names': ['A', 'B'], 'formats': ['i4', 'f4'], 'offsets': [0, 4]}) y = dtype({'names': ['B', 'A'], 'formats': ['f4', 'i4'], 'offsets': [4, 0]}) the expression ``x == y`` will now return ``False``, unlike before. This makes dictionary based dtype specifications like ``dtype({'a': ('i4', 0), 'b': ('f4', 4)})`` dangerous in python < 3.6 since dict key order is not preserved in those versions. Assignment from a structured array to a boolean array now raises a ValueError, unlike in 1.13, where it always set the destination elements to ``True``. Assignment from structured array with more than one field to a non-structured array now raises a ValueError. In 1.13 this copied just the first field of the source to the destination. Using field "titles" in multiple-field indexing is now disallowed, as is repeating a field name in a multiple-field index. The documentation for structured arrays in the user guide has been significantly updated to reflect these changes. Integer and Void scalars are now unaffected by ``np.set_string_function`` ------------------------------------------------------------------------- Previously, unlike most other numpy scalars, the ``str`` and ``repr`` of integer and void scalars could be controlled by ``np.set_string_function``. This is no longer possible. 0d array printing changed, ``style`` arg of array2string deprecated ------------------------------------------------------------------- Previously the ``str`` and ``repr`` of 0d arrays had idiosyncratic implementations which returned ``str(a.item())`` and ``'array(' + repr(a.item()) + ')'`` respectively for 0d array ``a``, unlike both numpy scalars and higher dimension ndarrays. Now, the ``str`` of a 0d array acts like a numpy scalar using ``str(a[()])`` and the ``repr`` acts like higher dimension arrays using ``formatter(a[()])``, where ``formatter`` can be specified using ``np.set_printoptions``. The ``style`` argument of ``np.array2string`` is deprecated. This new behavior is disabled in 1.13 legacy printing mode, see compatibility notes above. Seeding ``RandomState`` using an array requires a 1-d array ----------------------------------------------------------- ``RandomState`` previously would accept empty arrays or arrays with 2 or more dimensions, which resulted in either a failure to seed (empty arrays) or for some of the passed values to be ignored when setting the seed. ``MaskedArray`` objects show a more useful ``repr`` --------------------------------------------------- The ``repr`` of a ``MaskedArray`` is now closer to the python code that would produce it, with arrays now being shown with commas and dtypes. Like the other formatting changes, this can be disabled with the 1.13 legacy printing mode in order to help transition doctests. The ``repr`` of ``np.polynomial`` classes is more explicit ---------------------------------------------------------- It now shows the domain and window parameters as keyword arguments to make them more clear:: >>> np.polynomial.Polynomial(range(4)) Polynomial([0., 1., 2., 3.], domain=[-1, 1], window=[-1, 1]) ========================== ``` ### 1.13.3 ``` ========================== This is a bugfix release for some problems found since 1.13.1. The most important fixes are for CVE-2017-12852 and temporary elision. Users of earlier versions of 1.13 should upgrade. The Python versions supported are 2.7 and 3.4 - 3.6. The Python 3.6 wheels available from PIP are built with Python 3.6.2 and should be compatible with all previous versions of Python 3.6. It was cythonized with Cython 0.26.1, which should be free of the bugs found in 0.27 while also being compatible with Python 3.7-dev. The Windows wheels were built with OpenBlas instead ATLAS, which should improve the performance of the linear algebra functions. The NumPy 1.13.3 release is a re-release of 1.13.2, which suffered from a bug in Cython 0.27.0. Contributors ============ A total of 12 people contributed to this release. People with a "+" by their names contributed a patch for the first time. * Allan Haldane * Brandon Carter * Charles Harris * Eric Wieser * Iryna Shcherbina + * James Bourbeau + * Jonathan Helmus * Julian Taylor * Matti Picus * Michael Lamparski + * Michael Seifert * Ralf Gommers Pull requests merged ==================== A total of 22 pull requests were merged for this release. * 9390 BUG: Return the poly1d coefficients array directly * 9555 BUG: Fix regression in 1.13.x in distutils.mingw32ccompiler. * 9556 BUG: Fix true_divide when dtype=np.float64 specified. * 9557 DOC: Fix some rst markup in numpy/doc/basics.py. * 9558 BLD: Remove -xhost flag from IntelFCompiler. * 9559 DOC: Removes broken docstring example (source code, png, pdf)... * 9580 BUG: Add hypot and cabs functions to WIN32 blacklist. * 9732 BUG: Make scalar function elision check if temp is writeable. * 9736 BUG: Various fixes to np.gradient * 9742 BUG: Fix np.pad for CVE-2017-12852 * 9744 BUG: Check for exception in sort functions, add tests * 9745 DOC: Add whitespace after "versionadded::" directive so it actually... * 9746 BUG: Memory leak in np.dot of size 0 * 9747 BUG: Adjust gfortran version search regex * 9757 BUG: Cython 0.27 breaks NumPy on Python 3. * 9764 BUG: Ensure `_npy_scaled_cexp{,f,l}` is defined when needed. * 9765 BUG: PyArray_CountNonzero does not check for exceptions * 9766 BUG: Fixes histogram monotonicity check for unsigned bin values * 9767 BUG: Ensure consistent result dtype of count_nonzero * 9771 BUG: MAINT: Fix mtrand for Cython 0.27. * 9772 DOC: Create the 1.13.2 release notes. * 9794 DOC: Create 1.13.3 release notes. ========================== ``` ### 1.13.1 ``` ========================== This is a bugfix release for problems found in 1.13.0. The major changes are fixes for the new memory overlap detection and temporary elision as well as reversion of the removal of the boolean binary ``-`` operator. Users of 1.13.0 should upgrade. Thr Python versions supported are 2.7 and 3.4 - 3.6. Note that the Python 3.6 wheels available from PIP are built against 3.6.1, hence will not work when used with 3.6.0 due to Python bug 29943_. NumPy 1.13.2 will be released shortly after Python 3.6.2 is out to fix that problem. If you are using 3.6.0 the workaround is to upgrade to 3.6.1 or use an earlier Python version. .. _29943: https://bugs.python.org/issue29943 Pull requests merged ==================== A total of 19 pull requests were merged for this release. * 9240 DOC: BLD: fix lots of Sphinx warnings/errors. * 9255 Revert "DEP: Raise TypeError for subtract(bool_, bool_)." * 9261 BUG: don't elide into readonly and updateifcopy temporaries for... * 9262 BUG: fix missing keyword rename for common block in numpy.f2py * 9263 BUG: handle resize of 0d array * 9267 DOC: update f2py front page and some doc build metadata. * 9299 BUG: Fix Intel compilation on Unix. * 9317 BUG: fix wrong ndim used in empty where check * 9319 BUG: Make extensions compilable with MinGW on Py2.7 * 9339 BUG: Prevent crash if ufunc doc string is null * 9340 BUG: umath: un-break ufunc where= when no out= is given * 9371 DOC: Add isnat/positive ufunc to documentation * 9372 BUG: Fix error in fromstring function from numpy.core.records... * 9373 BUG: ')' is printed at the end pointer of the buffer in numpy.f2py. * 9374 DOC: Create NumPy 1.13.1 release notes. * 9376 BUG: Prevent hang traversing ufunc userloop linked list * 9377 DOC: Use x1 and x2 in the heaviside docstring. * 9378 DOC: Add $PARAMS to the isnat docstring * 9379 DOC: Update the 1.13.1 release notes Contributors ============ A total of 12 people contributed to this release. People with a "+" by their names contributed a patch for the first time. * Andras Deak + * Bob Eldering + * Charles Harris * Daniel Hrisca + * Eric Wieser * Joshua Leahy + * Julian Taylor * Michael Seifert * Pauli Virtanen * Ralf Gommers * Roland Kaufmann * Warren Weckesser ========================== ``` ### 1.13.0 ``` ========================== This release supports Python 2.7 and 3.4 - 3.6. Highlights ========== * Operations like ``a + b + c`` will reuse temporaries on some platforms, resulting in less memory use and faster execution. * Inplace operations check if inputs overlap outputs and create temporaries to avoid problems. * New ``__array_ufunc__`` attribute provides improved ability for classes to override default ufunc behavior. * New ``np.block`` function for creating blocked arrays. New functions ============= * New ``np.positive`` ufunc. * New ``np.divmod`` ufunc provides more efficient divmod. * New ``np.isnat`` ufunc tests for NaT special values. * New ``np.heaviside`` ufunc computes the Heaviside function. * New ``np.isin`` function, improves on ``in1d``. * New ``np.block`` function for creating blocked arrays. * New ``PyArray_MapIterArrayCopyIfOverlap`` added to NumPy C-API. See below for details. Deprecations ============ * Calling ``np.fix``, ``np.isposinf``, and ``np.isneginf`` with ``f(x, y=out)`` is deprecated - the argument should be passed as ``f(x, out=out)``, which matches other ufunc-like interfaces. * Use of the C-API ``NPY_CHAR`` type number deprecated since version 1.7 will now raise deprecation warnings at runtime. Extensions built with older f2py versions need to be recompiled to remove the warning. * ``np.ma.argsort``, ``np.ma.minimum.reduce``, and ``np.ma.maximum.reduce`` should be called with an explicit `axis` argument when applied to arrays with more than 2 dimensions, as the default value of this argument (``None``) is inconsistent with the rest of numpy (``-1``, ``0``, and ``0``, respectively). * ``np.ma.MaskedArray.mini`` is deprecated, as it almost duplicates the functionality of ``np.MaskedArray.min``. Exactly equivalent behaviour can be obtained with ``np.ma.minimum.reduce``. * The single-argument form of ``np.ma.minimum`` and ``np.ma.maximum`` is deprecated. ``np.maximum``. ``np.ma.minimum(x)`` should now be spelt ``np.ma.minimum.reduce(x)``, which is consistent with how this would be done with ``np.minimum``. * Calling ``ndarray.conjugate`` on non-numeric dtypes is deprecated (it should match the behavior of ``np.conjugate``, which throws an error). * Calling ``expand_dims`` when the ``axis`` keyword does not satisfy ``-a.ndim - 1 <= axis <= a.ndim``, where ``a`` is the array being reshaped, is deprecated. Future Changes ============== * Assignment between structured arrays with different field names will change in NumPy 1.14. Previously, fields in the dst would be set to the value of the identically-named field in the src. In numpy 1.14 fields will instead be assigned 'by position': The n-th field of the dst will be set to the n-th field of the src array. Note that the ``FutureWarning`` raised in NumPy 1.12 incorrectly reported this change as scheduled for NumPy 1.13 rather than NumPy 1.14. Build System Changes ==================== * ``numpy.distutils`` now automatically determines C-file dependencies with GCC compatible compilers. Compatibility notes =================== Error type changes ------------------ * ``numpy.hstack()`` now throws ``ValueError`` instead of ``IndexError`` when input is empty. * Functions taking an axis argument, when that argument is out of range, now throw ``np.AxisError`` instead of a mixture of ``IndexError`` and ``ValueError``. For backwards compatibility, ``AxisError`` subclasses both of these. Tuple object dtypes ------------------- Support has been removed for certain obscure dtypes that were unintentionally allowed, of the form ``(old_dtype, new_dtype)``, where either of the dtypes is or contains the ``object`` dtype. As an exception, dtypes of the form ``(object, [('name', object)])`` are still supported due to evidence of existing use. DeprecationWarning to error --------------------------- See Changes section for more detail. * ``partition``, TypeError when non-integer partition index is used. * ``NpyIter_AdvancedNew``, ValueError when ``oa_ndim == 0`` and ``op_axes`` is NULL * ``negative(bool_)``, TypeError when negative applied to booleans. * ``subtract(bool_, bool_)``, TypeError when subtracting boolean from boolean. * ``np.equal, np.not_equal``, object identity doesn't override failed comparison. * ``np.equal, np.not_equal``, object identity doesn't override non-boolean comparison. * Deprecated boolean indexing behavior dropped. See Changes below for details. * Deprecated ``np.alterdot()`` and ``np.restoredot()`` removed. FutureWarning to changed behavior --------------------------------- See Changes section for more detail. * ``numpy.average`` preserves subclasses * ``array == None`` and ``array != None`` do element-wise comparison. * ``np.equal, np.not_equal``, object identity doesn't override comparison result. dtypes are now always true -------------------------- Previously ``bool(dtype)`` would fall back to the default python implementation, which checked if ``len(dtype) > 0``. Since ``dtype`` objects implement ``__len__`` as the number of record fields, ``bool`` of scalar dtypes would evaluate to ``False``, which was unintuitive. Now ``bool(dtype) == True`` for all dtypes. ``__getslice__`` and ``__setslice__`` are no longer needed in ``ndarray`` subclasses ------------------------------------------------------------------------------------ When subclassing np.ndarray in Python 2.7, it is no longer _necessary_ to implement ``__*slice__`` on the derived class, as ``__*item__`` will intercept these calls correctly. Any code that did implement these will work exactly as before. Code that invokes``ndarray.__getslice__`` (e.g. through ``super(...).__getslice__``) will now issue a DeprecationWarning - ``.__getitem__(slice(start, end))`` should be used instead. Indexing MaskedArrays/Constants with ``...`` (ellipsis) now returns MaskedArray ------------------------------------------------------------------------------- This behavior mirrors that of np.ndarray, and accounts for nested arrays in MaskedArrays of object dtype, and ellipsis combined with other forms of indexing. C API changes ============= GUfuncs on empty arrays and NpyIter axis removal ------------------------------------------------ It is now allowed to remove a zero-sized axis from NpyIter. Which may mean that code removing axes from NpyIter has to add an additional check when accessing the removed dimensions later on. The largest followup change is that gufuncs are now allowed to have zero-sized inner dimensions. This means that a gufunc now has to anticipate an empty inner dimension, while this was never possible and an error raised instead. For most gufuncs no change should be necessary. However, it is now possible for gufuncs with a signature such as ``(..., N, M) -> (..., M)`` to return a valid result if ``N=0`` without further wrapping code. ``PyArray_MapIterArrayCopyIfOverlap`` added to NumPy C-API ---------------------------------------------------------- Similar to ``PyArray_MapIterArray`` but with an additional ``copy_if_overlap`` argument. If ``copy_if_overlap != 0``, checks if input has memory overlap with any of the other arrays and make copies as appropriate to avoid problems if the input is modified during the iteration. See the documentation for more complete documentation. New Features ============ ``__array_ufunc__`` added ------------------------- This is the renamed and redesigned ``__numpy_ufunc__``. Any class, ndarray subclass or not, can define this method or set it to ``None`` in order to override the behavior of NumPy's ufuncs. This works quite similarly to Python's ``__mul__`` and other binary operation routines. See the documentation for a more detailed description of the implementation and behavior of this new option. The API is provisional, we do not yet guarantee backward compatibility as modifications may be made pending feedback. See the NEP_ and documentation_ for more details. .. _NEP: https://github.com/numpy/numpy/blob/master/doc/neps/ufunc-overrides.rst .. _documentation: https://github.com/charris/numpy/blob/master/doc/source/reference/arrays.classes.rst New ``positive`` ufunc ---------------------- This ufunc corresponds to unary `+`, but unlike `+` on an ndarray it will raise an error if array values do not support numeric operations. New ``divmod`` ufunc -------------------- This ufunc corresponds to the Python builtin `divmod`, and is used to implement `divmod` when called on numpy arrays. ``np.divmod(x, y)`` calculates a result equivalent to ``(np.floor_divide(x, y), np.remainder(x, y))`` but is approximately twice as fast as calling the functions separately. ``np.isnat`` ufunc tests for NaT special datetime and timedelta values ---------------------------------------------------------------------- The new ufunc ``np.isnat`` finds the positions of special NaT values within datetime and timedelta arrays. This is analogous to ``np.isnan``. ``np.heaviside`` ufunc computes the Heaviside function ------------------------------------------------------ The new function ``np.heaviside(x, h0)`` (a ufunc) computes the Heaviside function: .. code:: { 0 if x < 0, heaviside(x, h0) = { h0 if x == 0, { 1 if x > 0. ``np.block`` function for creating blocked arrays ------------------------------------------------- Add a new ``block`` function to the current stacking functions ``vstack``, ``hstack``, and ``stack``. This allows concatenation across multiple axes simultaneously, with a similar syntax to array creation, but where elements can themselves be arrays. For instance:: >>> A = np.eye(2) * 2 >>> B = np.eye(3) * 3 >>> np.block([ ... [A, np.zeros((2, 3))], ... [np.ones((3, 2)), B ] ... ]) array([[ 2., 0., 0., 0., 0.], [ 0., 2., 0., 0., 0.], [ 1., 1., 3., 0., 0.], [ 1., 1., 0., 3., 0.], [ 1., 1., 0., 0., 3.]]) While primarily useful for block matrices, this works for arbitrary dimensions of arrays. It is similar to Matlab's square bracket notation for creating block matrices. ``isin`` function, improving on ``in1d`` ---------------------------------------- The new function ``isin`` tests whether each element of an N-dimensonal array is present anywhere within a second array. It is an enhancement of ``in1d`` that preserves the shape of the first array. Temporary elision ----------------- On platforms providing the ``backtrace`` function NumPy will try to avoid creating temporaries in expression involving basic numeric types. For example ``d = a + b + c`` is transformed to ``d = a + b; d += c`` which can improve performance for large arrays as less memory bandwidth is required to perform the operation. ``axes`` argument for ``unique`` -------------------------------- In an N-dimensional array, the user can now choose the axis along which to look for duplicate N-1-dimensional elements using ``numpy.unique``. The original behaviour is recovered if ``axis=None`` (default). ``np.gradient`` now supports unevenly spaced data ------------------------------------------------- Users can now specify a not-constant spacing for data. In particular ``np.gradient`` can now take: 1. A single scalar to specify a sample distance for all dimensions. 2. N scalars to specify a constant sample distance for each dimension. i.e. ``dx``, ``dy``, ``dz``, ... 3. N arrays to specify the coordinates of the values along each dimension of F. The length of the array must match the size of the corresponding dimension 4. Any combination of N scalars/arrays with the meaning of 2. and 3. This means that, e.g., it is now possible to do the following:: >>> f = np.array([[1, 2, 6], [3, 4, 5]], dtype=np.float) >>> dx = 2. >>> y = [1., 1.5, 3.5] >>> np.gradient(f, dx, y) [array([[ 1. , 1. , -0.5], [ 1. , 1. , -0.5]]), array([[ 2. , 2. , 2. ], [ 2. , 1.7, 0.5]])] Support for returning arrays of arbitrary dimensions in ``apply_along_axis`` ---------------------------------------------------------------------------- Previously, only scalars or 1D arrays could be returned by the function passed to ``apply_along_axis``. Now, it can return an array of any dimensionality (including 0D), and the shape of this array replaces the axis of the array being iterated over. ``.ndim`` property added to ``dtype`` to complement ``.shape`` -------------------------------------------------------------- For consistency with ``ndarray`` and ``broadcast``, ``d.ndim`` is a shorthand for ``len(d.shape)``. Support for tracemalloc in Python 3.6 ------------------------------------- NumPy now supports memory tracing with tracemalloc_ module of Python 3.6 or newer. Memory allocations from NumPy are placed into the domain defined by ``numpy.lib.tracemalloc_domain``. Note that NumPy allocation will not show up in tracemalloc_ of earlier Python versions. .. _tracemalloc: https://docs.python.org/3/library/tracemalloc.html NumPy may be built with relaxed stride checking debugging --------------------------------------------------------- Setting NPY_RELAXED_STRIDES_DEBUG=1 in the environment when relaxed stride checking is enabled will cause NumPy to be compiled with the affected strides set to the maximum value of npy_intp in order to help detect invalid usage of the strides in downstream projects. When enabled, invalid usage often results in an error being raised, but the exact type of error depends on the details of the code. TypeError and OverflowError have been observed in the wild. It was previously the case that this option was disabled for releases and enabled in master and changing between the two required editing the code. It is now disabled by default but can be enabled for test builds. Improvements ============ Ufunc behavior for overlapping inputs ------------------------------------- Operations where ufunc input and output operands have memory overlap produced undefined results in previous NumPy versions, due to data dependency issues. In NumPy 1.13.0, results from such operations are now defined to be the same as for equivalent operations where there is no memory overlap. Operations affected now make temporary copies, as needed to eliminate data dependency. As detecting these cases is computationally expensive, a heuristic is used, which may in rare cases result to needless temporary copies. For operations where the data dependency is simple enough for the heuristic to analyze, temporary copies will not be made even if the arrays overlap, if it can be deduced copies are not necessary. As an example,``np.add(a, b, out=a)`` will not involve copies. To illustrate a previously undefined operation:: >>> x = np.arange(16).astype(float) >>> np.add(x[1:], x[:-1], out=x[1:]) In NumPy 1.13.0 the last line is guaranteed to be equivalent to:: >>> np.add(x[1:].copy(), x[:-1].copy(), out=x[1:]) A similar operation with simple non-problematic data dependence is:: >>> x = np.arange(16).astype(float) >>> np.add(x[1:], x[:-1], out=x[:-1]) It will continue to produce the same results as in previous NumPy versions, and will not involve unnecessary temporary copies. The change applies also to in-place binary operations, for example:: >>> x = np.random.rand(500, 500) >>> x += x.T This statement is now guaranteed to be equivalent to ``x[...] = x + x.T``, whereas in previous NumPy versions the results were undefined. Partial support for 64-bit f2py extensions with MinGW ----------------------------------------------------- Extensions that incorporate Fortran libraries can now be built using the free MinGW_ toolset, also under Python 3.5. This works best for extensions that only do calculations and uses the runtime modestly (reading and writing from files, for instance). Note that this does not remove the need for Mingwpy; if you make extensive use of the runtime, you will most likely run into issues_. Instead, it should be regarded as a band-aid until Mingwpy is fully functional. Extensions can also be compiled using the MinGW toolset using the runtime library from the (moveable) WinPython 3.4 distribution, which can be useful for programs with a PySide1/Qt4 front-end. .. _MinGW: https://sf.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/6.2.0/threads-win32/seh/ .. _issues: https://mingwpy.github.io/issues.html Performance improvements for ``packbits`` and ``unpackbits`` ------------------------------------------------------------ The functions ``numpy.packbits`` with boolean input and ``numpy.unpackbits`` have been optimized to be a significantly faster for contiguous data. Fix for PPC long double floating point information -------------------------------------------------- In previous versions of NumPy, the ``finfo`` function returned invalid information about the `double double`_ format of the ``longdouble`` float type on Power PC (PPC). The invalid values resulted from the failure of the NumPy algorithm to deal with the variable number of digits in the significand that are a feature of `PPC long doubles`. This release by-passes the failing algorithm by using heuristics to detect the presence of the PPC double double format. A side-effect of using these heuristics is that the ``finfo`` function is faster than previous releases. .. _PPC long doubles: https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.genprogc/128bit_long_double_floating-point_datatype.htm .. _double double: https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_formatDouble-double_arithmetic Better default repr for ``ndarray`` subclasses ---------------------------------------------- Subclasses of ndarray with no ``repr`` specialization now correctly indent their data and type lines. More reliable comparisons of masked arrays ------------------------------------------ Comparisons of masked arrays were buggy for masked scalars and failed for structured arrays with dimension higher than one. Both problems are now solved. In the process, it was ensured that in getting the result for a structured array, masked fields are properly ignored, i.e., the result is equal if all fields that are non-masked in both are equal, thus making the behaviour identical to what one gets by comparing an unstructured masked array and then doing ``.all()`` over some axis. np.matrix with booleans elements can now be created using the string syntax --------------------------------------------------------------------------- ``np.matrix`` failed whenever one attempts to use it with booleans, e.g., ``np.matrix('True')``. Now, this works as expected. More ``linalg`` operations now accept empty vectors and matrices ---------------------------------------------------------------- All of the following functions in ``np.linalg`` now work when given input arrays with a 0 in the last two dimensions: ``det``, ``slogdet``, ``pinv``, ``eigvals``, ``eigvalsh``, ``eig``, ``eigh``. Bundled version of LAPACK is now 3.2.2 -------------------------------------- NumPy comes bundled with a minimal implementation of lapack for systems without a lapack library installed, under the name of ``lapack_lite``. This has been upgraded from LAPACK 3.0.0 (June 30, 1999) to LAPACK 3.2.2 (June 30, 2010). See the `LAPACK changelogs`_ for details on the all the changes this entails. While no new features are exposed through ``numpy``, this fixes some bugs regarding "workspace" sizes, and in some places may use faster algorithms. .. _`LAPACK changelogs`: http://www.netlib.org/lapack/release_notes.html_4_history_of_lapack_releases ``reduce`` of ``np.hypot.reduce`` and ``np.logical_xor`` allowed in more cases ------------------------------------------------------------------------------ This now works on empty arrays, returning 0, and can reduce over multiple axes. Previously, a ``ValueError`` was thrown in these cases. Better ``repr`` of object arrays -------------------------------- Object arrays that contain themselves no longer cause a recursion error. Object arrays that contain ``list`` objects are now printed in a way that makes clear the difference between a 2d object array, and a 1d object array of lists. Changes ======= ``argsort`` on masked arrays takes the same default arguments as ``sort`` ------------------------------------------------------------------------- By default, ``argsort`` now places the masked values at the end of the sorted array, in the same way that ``sort`` already did. Additionally, the ``end_with`` argument is added to ``argsort``, for consistency with ``sort``. Note that this argument is not added at the end, so breaks any code that passed ``fill_value`` as a positional argument. ``average`` now preserves subclasses ------------------------------------ For ndarray subclasses, ``numpy.average`` will now return an instance of the subclass, matching the behavior of most other NumPy functions such as ``mean``. As a consequence, also calls that returned a scalar may now return a subclass array scalar. ``array == None`` and ``array != None`` do element-wise comparison ----------------------------------------