Changelog
### 1.4.0
```
---------------------------
- Allow caching co-routines with `Memory.cache`.
https://github.com/joblib/joblib/pull/894
- Try to cast ``n_jobs`` to int in parallel and raise an error if
it fails. This means that ``n_jobs=2.3`` will now result in
``effective_n_jobs=2`` instead of failing.
https://github.com/joblib/joblib/pull/1539
- Ensure that errors in the task generator given to Parallel's call
are raised in the results consumming thread.
https://github.com/joblib/joblib/pull/1491
- Adjust codebase to NumPy 2.0 by changing ``np.NaN`` to ``np.nan``
and importing ``byte_bounds`` from ``np.lib.array_utils``.
https://github.com/joblib/joblib/pull/1501
- The parameter ``return_as`` in ``joblib.Parallel`` can now be set to
``generator_unordered``. In this case the results will be returned in the
order of task completion rather than the order of submission.
https://github.com/joblib/joblib/pull/1463
- dask backend now supports ``return_as=generator`` and
``return_as=generator_unordered``.
https://github.com/joblib/joblib/pull/1520
- Vendor cloudpickle 3.0.0 and end support for Python 3.7 which has
reached end of life.
https://github.com/joblib/joblib/pull/1487
https://github.com/joblib/joblib/pull/1515
```
### 1.3.2
```
---------------------------
- Fix a regression in ``joblib.Parallel`` introduced in 1.3.0 where
explicitly setting ``n_jobs=None`` was not interpreted as "unset".
https://github.com/joblib/joblib/pull/1475
- Fix a regression in ``joblib.Parallel`` introduced in 1.3.0 where
``joblib.Parallel`` logging methods exposed from inheritance to
``joblib.Logger`` didn't work because of missing logger
initialization.
https://github.com/joblib/joblib/pull/1494
- Various maintenance updates to the doc, the ci and the test.
https://github.com/joblib/joblib/pull/1480,
https://github.com/joblib/joblib/pull/1481,
https://github.com/joblib/joblib/pull/1476,
https://github.com/joblib/joblib/pull/1492
```
### 1.3.1
```
---------------------------
- Fix compatibility with python 3.7 by vendor loky 3.4.1
which is compatible with this version.
https://github.com/joblib/joblib/pull/1472
```
### 1.3.0
```
---------------------------
- Ensure native byte order for memmap arrays in ``joblib.load``.
https://github.com/joblib/joblib/issues/1353
- Add ability to change default Parallel backend in tests by setting the
``JOBLIB_TESTS_DEFAULT_PARALLEL_BACKEND`` environment variable.
https://github.com/joblib/joblib/pull/1356
- Fix temporary folder creation in `joblib.Parallel` on Linux subsystems on Windows
which do have `/dev/shm` but don't have the `os.statvfs` function
https://github.com/joblib/joblib/issues/1353
- Drop runtime dependency on ``distutils``. ``distutils`` is going away
in Python 3.12 and is deprecated from Python 3.10 onwards. This import
was kept around to avoid breaking scikit-learn, however it's now been
long enough since scikit-learn deployed a fixed (verion 1.1 was released
in May 2022) that it should be safe to remove this.
https://github.com/joblib/joblib/pull/1361
- A warning is raised when a pickling error occurs during caching operations.
In version 1.5, this warning will be turned into an error. For all other
errors, a new warning has been introduced: ``joblib.memory.CacheWarning``.
https://github.com/joblib/joblib/pull/1359
- Avoid (module, name) collisions when caching nested functions. This fix
changes the module name of nested functions, invalidating caches from
previous versions of Joblib.
https://github.com/joblib/joblib/pull/1374
- Add ``cache_validation_callback`` in :meth:`joblib.Memory.cache`, to allow
custom cache invalidation based on the metadata of the function call.
https://github.com/joblib/joblib/pull/1149
- Add a ``return_as`` parameter for ``Parallel``, that enables consuming
results asynchronously.
https://github.com/joblib/joblib/pull/1393,
https://github.com/joblib/joblib/pull/1458
- Improve the behavior of ``joblib`` for ``n_jobs=1``, with simplified
tracebacks and more efficient running time.
https://github.com/joblib/joblib/pull/1393
- Add the ``parallel_config`` context manager to allow for more fine-grained
control over the backend configuration. It should be used in place of the
``parallel_backend`` context manager. In particular, it has the advantage
of not requiring to set a specific backend in the context manager.
https://github.com/joblib/joblib/pull/1392,
https://github.com/joblib/joblib/pull/1457
- Add ``items_limit`` and ``age_limit`` in :meth:`joblib.Memory.reduce_size`
to make it easy to limit the number of items and remove items that have
not been accessed for a long time in the cache.
https://github.com/joblib/joblib/pull/1200
- Deprecate ``bytes_limit`` in ``Memory`` as this is not automatically enforced,
the limit can be directly passed to :meth:`joblib.Memory.reduce_size` which
needs to be called to actually enforce the limit.
https://github.com/joblib/joblib/pull/1447
- Vendor ``loky`` 3.4.0 which includes various fixes.
https://github.com/joblib/joblib/pull/1422
- Various updates to the documentation and to benchmarking tools.
https://github.com/joblib/joblib/pull/1343,
https://github.com/joblib/joblib/pull/1348,
https://github.com/joblib/joblib/pull/1411,
https://github.com/joblib/joblib/pull/1451,
https://github.com/joblib/joblib/pull/1427,
https://github.com/joblib/joblib/pull/1400
- Move project metadata to ``pyproject.toml``.
https://github.com/joblib/joblib/pull/1382,
https://github.com/joblib/joblib/pull/1433
- Add more tests to improve python ``nogil`` support.
https://github.com/joblib/joblib/pull/1394,
https://github.com/joblib/joblib/pull/1395
```
### 1.2.0
```
-------------
- Fix a security issue where ``eval(pre_dispatch)`` could potentially run
arbitrary code. Now only basic numerics are supported.
https://github.com/joblib/joblib/pull/1327
- Make sure that joblib works even when multiprocessing is not available,
for instance with Pyodide
https://github.com/joblib/joblib/pull/1256
- Avoid unnecessary warnings when workers and main process delete
the temporary memmap folder contents concurrently.
https://github.com/joblib/joblib/pull/1263
- Fix memory alignment bug for pickles containing numpy arrays.
This is especially important when loading the pickle with
``mmap_mode != None`` as the resulting ``numpy.memmap`` object
would not be able to correct the misalignment without performing
a memory copy.
This bug would cause invalid computation and segmentation faults
with native code that would directly access the underlying data
buffer of a numpy array, for instance C/C++/Cython code compiled
with older GCC versions or some old OpenBLAS written in platform
specific assembly.
https://github.com/joblib/joblib/pull/1254
- Vendor cloudpickle 2.2.0 which adds support for PyPy 3.8+.
- Vendor loky 3.3.0 which fixes several bugs including:
- robustly forcibly terminating worker processes in case of a crash
(https://github.com/joblib/joblib/pull/1269);
- avoiding leaking worker processes in case of nested loky parallel
calls;
- reliability spawn the correct number of reusable workers.
```
### 1.1.1
```
-------------
- Fix a security issue where ``eval(pre_dispatch)`` could potentially run
arbitrary code. Now only basic numerics are supported.
https://github.com/joblib/joblib/pull/1327
```
### 1.1.0
```
--------------
- Fix byte order inconsistency issue during deserialization using joblib.load
in cross-endian environment: the numpy arrays are now always loaded to
use the system byte order, independently of the byte order of the system
that serialized the pickle.
https://github.com/joblib/joblib/pull/1181
- Fix joblib.Memory bug with the ``ignore`` parameter when the cached function
is a decorated function.
https://github.com/joblib/joblib/pull/1165
- Fix `joblib.Memory` to properly handle caching for functions defined
interactively in a IPython session or in Jupyter notebook cell.
https://github.com/joblib/joblib/pull/1214
- Update vendored loky (from version 2.9 to 3.0) and cloudpickle (from
version 1.6 to 2.0)
https://github.com/joblib/joblib/pull/1218
```
Links
- PyPI: https://pypi.org/project/joblib
- Changelog: https://data.safetycli.com/changelogs/joblib/
This PR updates joblib from 1.0.1 to 1.4.0.
Changelog
### 1.4.0 ``` --------------------------- - Allow caching co-routines with `Memory.cache`. https://github.com/joblib/joblib/pull/894 - Try to cast ``n_jobs`` to int in parallel and raise an error if it fails. This means that ``n_jobs=2.3`` will now result in ``effective_n_jobs=2`` instead of failing. https://github.com/joblib/joblib/pull/1539 - Ensure that errors in the task generator given to Parallel's call are raised in the results consumming thread. https://github.com/joblib/joblib/pull/1491 - Adjust codebase to NumPy 2.0 by changing ``np.NaN`` to ``np.nan`` and importing ``byte_bounds`` from ``np.lib.array_utils``. https://github.com/joblib/joblib/pull/1501 - The parameter ``return_as`` in ``joblib.Parallel`` can now be set to ``generator_unordered``. In this case the results will be returned in the order of task completion rather than the order of submission. https://github.com/joblib/joblib/pull/1463 - dask backend now supports ``return_as=generator`` and ``return_as=generator_unordered``. https://github.com/joblib/joblib/pull/1520 - Vendor cloudpickle 3.0.0 and end support for Python 3.7 which has reached end of life. https://github.com/joblib/joblib/pull/1487 https://github.com/joblib/joblib/pull/1515 ``` ### 1.3.2 ``` --------------------------- - Fix a regression in ``joblib.Parallel`` introduced in 1.3.0 where explicitly setting ``n_jobs=None`` was not interpreted as "unset". https://github.com/joblib/joblib/pull/1475 - Fix a regression in ``joblib.Parallel`` introduced in 1.3.0 where ``joblib.Parallel`` logging methods exposed from inheritance to ``joblib.Logger`` didn't work because of missing logger initialization. https://github.com/joblib/joblib/pull/1494 - Various maintenance updates to the doc, the ci and the test. https://github.com/joblib/joblib/pull/1480, https://github.com/joblib/joblib/pull/1481, https://github.com/joblib/joblib/pull/1476, https://github.com/joblib/joblib/pull/1492 ``` ### 1.3.1 ``` --------------------------- - Fix compatibility with python 3.7 by vendor loky 3.4.1 which is compatible with this version. https://github.com/joblib/joblib/pull/1472 ``` ### 1.3.0 ``` --------------------------- - Ensure native byte order for memmap arrays in ``joblib.load``. https://github.com/joblib/joblib/issues/1353 - Add ability to change default Parallel backend in tests by setting the ``JOBLIB_TESTS_DEFAULT_PARALLEL_BACKEND`` environment variable. https://github.com/joblib/joblib/pull/1356 - Fix temporary folder creation in `joblib.Parallel` on Linux subsystems on Windows which do have `/dev/shm` but don't have the `os.statvfs` function https://github.com/joblib/joblib/issues/1353 - Drop runtime dependency on ``distutils``. ``distutils`` is going away in Python 3.12 and is deprecated from Python 3.10 onwards. This import was kept around to avoid breaking scikit-learn, however it's now been long enough since scikit-learn deployed a fixed (verion 1.1 was released in May 2022) that it should be safe to remove this. https://github.com/joblib/joblib/pull/1361 - A warning is raised when a pickling error occurs during caching operations. In version 1.5, this warning will be turned into an error. For all other errors, a new warning has been introduced: ``joblib.memory.CacheWarning``. https://github.com/joblib/joblib/pull/1359 - Avoid (module, name) collisions when caching nested functions. This fix changes the module name of nested functions, invalidating caches from previous versions of Joblib. https://github.com/joblib/joblib/pull/1374 - Add ``cache_validation_callback`` in :meth:`joblib.Memory.cache`, to allow custom cache invalidation based on the metadata of the function call. https://github.com/joblib/joblib/pull/1149 - Add a ``return_as`` parameter for ``Parallel``, that enables consuming results asynchronously. https://github.com/joblib/joblib/pull/1393, https://github.com/joblib/joblib/pull/1458 - Improve the behavior of ``joblib`` for ``n_jobs=1``, with simplified tracebacks and more efficient running time. https://github.com/joblib/joblib/pull/1393 - Add the ``parallel_config`` context manager to allow for more fine-grained control over the backend configuration. It should be used in place of the ``parallel_backend`` context manager. In particular, it has the advantage of not requiring to set a specific backend in the context manager. https://github.com/joblib/joblib/pull/1392, https://github.com/joblib/joblib/pull/1457 - Add ``items_limit`` and ``age_limit`` in :meth:`joblib.Memory.reduce_size` to make it easy to limit the number of items and remove items that have not been accessed for a long time in the cache. https://github.com/joblib/joblib/pull/1200 - Deprecate ``bytes_limit`` in ``Memory`` as this is not automatically enforced, the limit can be directly passed to :meth:`joblib.Memory.reduce_size` which needs to be called to actually enforce the limit. https://github.com/joblib/joblib/pull/1447 - Vendor ``loky`` 3.4.0 which includes various fixes. https://github.com/joblib/joblib/pull/1422 - Various updates to the documentation and to benchmarking tools. https://github.com/joblib/joblib/pull/1343, https://github.com/joblib/joblib/pull/1348, https://github.com/joblib/joblib/pull/1411, https://github.com/joblib/joblib/pull/1451, https://github.com/joblib/joblib/pull/1427, https://github.com/joblib/joblib/pull/1400 - Move project metadata to ``pyproject.toml``. https://github.com/joblib/joblib/pull/1382, https://github.com/joblib/joblib/pull/1433 - Add more tests to improve python ``nogil`` support. https://github.com/joblib/joblib/pull/1394, https://github.com/joblib/joblib/pull/1395 ``` ### 1.2.0 ``` ------------- - Fix a security issue where ``eval(pre_dispatch)`` could potentially run arbitrary code. Now only basic numerics are supported. https://github.com/joblib/joblib/pull/1327 - Make sure that joblib works even when multiprocessing is not available, for instance with Pyodide https://github.com/joblib/joblib/pull/1256 - Avoid unnecessary warnings when workers and main process delete the temporary memmap folder contents concurrently. https://github.com/joblib/joblib/pull/1263 - Fix memory alignment bug for pickles containing numpy arrays. This is especially important when loading the pickle with ``mmap_mode != None`` as the resulting ``numpy.memmap`` object would not be able to correct the misalignment without performing a memory copy. This bug would cause invalid computation and segmentation faults with native code that would directly access the underlying data buffer of a numpy array, for instance C/C++/Cython code compiled with older GCC versions or some old OpenBLAS written in platform specific assembly. https://github.com/joblib/joblib/pull/1254 - Vendor cloudpickle 2.2.0 which adds support for PyPy 3.8+. - Vendor loky 3.3.0 which fixes several bugs including: - robustly forcibly terminating worker processes in case of a crash (https://github.com/joblib/joblib/pull/1269); - avoiding leaking worker processes in case of nested loky parallel calls; - reliability spawn the correct number of reusable workers. ``` ### 1.1.1 ``` ------------- - Fix a security issue where ``eval(pre_dispatch)`` could potentially run arbitrary code. Now only basic numerics are supported. https://github.com/joblib/joblib/pull/1327 ``` ### 1.1.0 ``` -------------- - Fix byte order inconsistency issue during deserialization using joblib.load in cross-endian environment: the numpy arrays are now always loaded to use the system byte order, independently of the byte order of the system that serialized the pickle. https://github.com/joblib/joblib/pull/1181 - Fix joblib.Memory bug with the ``ignore`` parameter when the cached function is a decorated function. https://github.com/joblib/joblib/pull/1165 - Fix `joblib.Memory` to properly handle caching for functions defined interactively in a IPython session or in Jupyter notebook cell. https://github.com/joblib/joblib/pull/1214 - Update vendored loky (from version 2.9 to 3.0) and cloudpickle (from version 1.6 to 2.0) https://github.com/joblib/joblib/pull/1218 ```Links
- PyPI: https://pypi.org/project/joblib - Changelog: https://data.safetycli.com/changelogs/joblib/