We are excited to announce the release of PyTorch® 2.5! This release features a new CuDNN backend for SDPA, enabling speedups by default for users of SDPA on H100s or newer GPUs. As well, regional compilation of torch.compile offers a way to reduce the cold start up time for torch.compile by allowing users to compile a repeated nn.Module (e.g. a transformer layer in LLM) without recompilations. Finally, TorchInductor CPP backend offers solid performance speedup with numerous enhancements like FP16 support, CPP wrapper, AOT-Inductor mode, and max-autotune mode.
This release is composed of 4095 commits from 504 contributors since PyTorch 2.4. We want to sincerely thank our dedicated community for your contributions. As always, we encourage you to try these out and report any issues as we improve 2.5. More information about how to get started with the PyTorch 2-series can be found at our Getting Started page.
As well, please check out our new ecosystem projects releases with TorchRec and TorchFix.
Beta
Prototype
CuDNN backend for SDPA
FlexAttention
torch.compile regional compilation without recompilations
Compiled Autograd
TorchDynamo added support for exception handling & MutableMapping types
Flight Recorder
TorchInductor CPU backend optimization
Max-autotune Support on CPU with GEMM Template
TorchInductor on Windows
FP16 support on CPU path for both eager mode and TorchInductor CPP backend
Autoload Device Extension
Enhanced Intel GPU support
*To see a full list of public feature submissions click here.
BETA FEATURES
[Beta] CuDNN backend for SDPA
The cuDNN "Fused Flash Attention" backend was landed for torch.nn.functional.scaled_dot_product_attention. On NVIDIA H100 GPUs this can provide up to 75% speed-up over FlashAttentionV2. This speedup is enabled by default for all users of SDPA on H100 or newer GPUs.
[Beta] torch.compile regional compilation without recompilations
Regional compilation without recompilations, via torch._dynamo.config.inline_inbuilt_nn_modules which default to True in 2.5+. This option allows users to compile a repeated nn.Module (e.g. a transformer layer in LLM) without recompilations. Compared to compiling the full model, this option can result in smaller compilation latencies with 1%-5% performance degradation compared to full model compilation.
This feature advances Inductor’s CPU backend optimization, including CPP backend code generation and FX fusions with customized CPU kernels. The Inductor CPU backend supports vectorization of common data types and all Inductor IR operations, along with the static and symbolic shapes. It is compatible with both Linux and Windows OS and supports the default Python wrapper, the CPP wrapper, and AOT-Inductor mode.
Additionally, it extends the max-autotune mode of the GEMM template (prototyped in 2.5), offering further performance gains. The backend supports various FX fusions, lowering to customized kernels such as oneDNN for Linear/Conv operations and SDPA. The Inductor CPU backend consistently achieves performance speedups across three benchmark suites—TorchBench, Hugging Face, and timms—outperforming eager mode in 97.5% of the 193 models tested.
PROTOTYPE FEATURES
[Prototype] FlexAttention
We've introduced a flexible API that enables implementing various attention mechanisms such as Sliding Window, Causal Mask, and PrefixLM with just a few lines of idiomatic PyTorch code. This API leverages torch.compile to generate a fused FlashAttention kernel, which eliminates extra memory allocation and achieves performance comparable to handwritten implementations. Additionally, we automatically generate the backwards pass using PyTorch's autograd machinery. Furthermore, our API can take advantage of sparsity in the attention mask, resulting in significant improvements over standard attention implementations.
We’ve just uploaded mypy 1.12 to the Python Package Index (PyPI). Mypy is a static type
checker for Python. This release includes new features, performance improvements and bug fixes.
You can install it as follows:
python3 -m pip install -U mypy
You can read the full documentation for this release on Read the Docs.
Support Python 3.12 Syntax for Generics (PEP 695)
Support for the new type parameter syntax introduced in Python 3.12 is now enabled by default,
documented, and no longer experimental. It was available through a feature flag in
mypy 1.11 as an experimental feature.
This example demonstrates the new syntax:
# Generic function
def f[T](https://github.com/python/mypy/blob/master/x: T) -> T: ...
reveal_type(f(1)) # Revealed type is 'int'
Generic class
class C[T]:
def init(self, x: T) -> None:
self.x = x
c = C('a')
reveal_type(c.x) # Revealed type is 'str'
Black is now officially tested with Python 3.13 and provides Python 3.13
mypyc-compiled wheels. (#4436) (#4449)
Black will issue an error when used with Python 3.12.5, due to an upstream memory
safety issue in Python 3.12.5 that can cause Black's AST safety checks to fail. Please
use Python 3.12.6 or Python 3.12.4 instead. (#4447)
Black no longer supports running with Python 3.8 (#4452)
Stable style
Fix crashes involving comments in parenthesised return types or X | Y style unions.
(#4453)
Fix skipping Jupyter cells with unknown %% magic (#4462)
Preview style
Fix type annotation spacing between * and more complex type variable tuple (i.e. def fn(*args: *tuple[*Ts, T]) -> None: pass) (#4440)
Caching
Fix bug where the cache was shared between runs with and without --unstable (#4466)
Packaging
Upgrade version of mypyc used to 1.12 beta (#4450) (#4449)
blackd now requires a newer version of aiohttp. (#4451)
Output
Added Python target version information on parse error (#4378)
Add information about Black version to internal error messages (#4457)
Black is now officially tested with Python 3.13 and provides Python 3.13
mypyc-compiled wheels. (#4436) (#4449)
Black will issue an error when used with Python 3.12.5, due to an upstream memory
safety issue in Python 3.12.5 that can cause Black's AST safety checks to fail. Please
use Python 3.12.6 or Python 3.12.4 instead. (#4447)
Black no longer supports running with Python 3.8 (#4452)
Stable style
Fix crashes involving comments in parenthesised return types or X | Y style unions.
(#4453)
Fix skipping Jupyter cells with unknown %% magic (#4462)
Preview style
Fix type annotation spacing between * and more complex type variable tuple (i.e. def fn(*args: *tuple[*Ts, T]) -> None: pass) (#4440)
Caching
Fix bug where the cache was shared between runs with and without --unstable (#4466)
Packaging
Upgrade version of mypyc used to 1.12 beta (#4450) (#4449)
blackd now requires a newer version of aiohttp. (#4451)
Output
Added Python target version information on parse error (#4378)
Add information about Black version to internal error messages (#4457)
Check out the blog post for a migration guide and overview of the changes!
Breaking changes
The pytest rules PT001 and PT023 now default to omitting the decorator parentheses when there are no arguments
(#12838, #13292).
This was a change that we attempted to make in Ruff v0.6.0, but only partially made due to an error on our part.
See the blog post for more details.
The useless-try-except rule (in our tryceratops category) has been recoded from TRY302 to
TRY203 (#13502). This ensures Ruff's code is consistent with
the same rule in the tryceratops linter.
[flake8-simplify] Stabilize preview behavior for SIM115 so that the rule can detect files
being opened from a wider range of standard-library functions (#12959).
CLI
Add explanation of fixable in --statistics command (#13774)
Check out the blog post for a migration guide and overview of the changes!
Breaking changes
The pytest rules PT001 and PT023 now default to omitting the decorator parentheses when there are no arguments
(#12838, #13292).
This was a change that we attempted to make in Ruff v0.6.0, but only partially made due to an error on our part.
See the blog post for more details.
The useless-try-except rule (in our tryceratops category) has been recoded from TRY302 to
TRY203 (#13502). This ensures Ruff's code is consistent with
the same rule in the tryceratops linter.
[flake8-simplify] Stabilize preview behavior for SIM115 so that the rule can detect files
being opened from a wider range of standard-library functions (#12959).
CLI
Add explanation of fixable in --statistics command (#13774)
- Add ``NETRC`` to the list of environment variables always passed through. (:issue:`3410`)
Improved Documentation - 4.23.0
replace [tool.pyproject] and [tool.tox.pyproject] with [tool.tox] in config.rst (:issue:3411)
v4.22.0 (2024-10-15)
Features - 4.22.0
- Implement dependency group support as defined in :pep:`735` - see :ref:`dependency_groups` - by :user:`gaborbernat`. (:issue:`3408`)
v4.21.2 (2024-10-03)
Bugfixes - 4.21.2
Include tox.toml in sdist archives to fix test failures resulting from its lack.
by :user:mgorny (:issue:3389)
v4.21.1 (2024-10-02)
Bugfixes - 4.21.1
- Fix error when using ``requires`` within a TOML configuration file - by :user:`gaborbernat`. (:issue:`3386`)
- Fix error when using ``deps`` within a TOML configuration file - by :user:`gaborbernat`. (:issue:`3387`)
- Multiple fixes for the TOML configuration by :user:`gaborbernat`.:
Do not fail when there is an empty command within commands.
Allow references for set_env by accepting list of dictionaries for it.
Do not try to be smart about reference unrolling, instead allow the user to control it via the extend flag,
available both for posargs and ref replacements.
The ref replacements raw key has been renamed to of. (:issue:3388)
v4.21.0 (2024-09-30)
Features - 4.21.0
Native TOML configuration support - by :user:gaborbernat. (:issue:999)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore major ver...
_Description has been truncated_
dependabot[bot]
commented
1 month ago
Updates the requirements on numpy, torch, qrcode, horde-sdk, horde-engine, horde-model-reference, mypy, black, ruff, tox and pre-commit to permit the latest version. Updates
numpyfrom 1.26.4 to 2.1.2Release notes
Sourced from numpy's releases.
... (truncated)
Commits
f5afe3dMerge pull request #27507 from charris/prepare-2.1.26b9ef48REL: Prepare for the NumPy 2.1.2 release [wheel build]6d85a24MAINT: Pin setuptools for Python < 3.12 [wheel build]7f327d8Merge pull request #27506 from charris/backport-2750309e9cd9BUG: avoid segfault on bad arguments in ndarray.array_function06b4793Merge pull request #27501 from charris/backport-27495364efb5MAINT: Bump pypa/cibuildwheel from 2.21.1 to 2.21.2df0e261Merge pull request #27484 from ngoldbaum/use-weak-defaultfa61bc6Merge pull request #27481 from charris/backport-27472d6d06aeMerge pull request #27480 from charris/backport-27475Updates
torchfrom 2.3.1 to 2.5.0Release notes
Sourced from torch's releases.
... (truncated)
Commits
32f585d[Release only] use triton 3.1.x from pypi (#137895)417a076[split build] move periodic split builds into own concurrency group (#135510)...119e734[RELEASE-ONLY CHANGES] Fix dependency on filesystem on Linux (#137242)783a6a4[MPS] Add regression test forfft.fftfreq(#137215)5375201[MPS] Add missing dispatch to rshift.Tensor (#137212)1de132e[MPS] Fix 5D+ reductions over negative dimentions (#137211)0b1b609[NCCL] Don't overridewaitUntilInitialized's setting of `comm->initialized_...0b45af9Fix addmm silent correctness on aarch64 (#137208)1a0b166[ONNX] Add assertion nodes to ignoring list (#137214)3a541efClarify thatlibtorchAPI is C++17 compatible (#137206)Updates
qrcodefrom 7.4.2 to 8.0Changelog
Sourced from qrcode's changelog.
Commits
456b01dPreparing release 8.0909b791Merge pull request #370 from lincolnloop/poetrydc783edUpdate CHANGELOG and README68f3b29Raise error if embedded image is provided and error correction is lower than Hffa2b4aMerge branch 'poetry' into fix/embedded-and-error-correction39cf502Fix test dependency skipd991b67Update poetry dependency groups to allow having pillow and png together11b0f95Merge branch 'poetry' into decompat5760eecRun ruf and fix formatting137946aFix import errors in testsUpdates
horde-sdkto 0.15.1Release notes
Sourced from horde-sdk's releases.
Commits
eb20833Merge pull request #272 from Haidra-Org/main7912776fix: use new default ssl context in all aiohttp requests (#271)f9df079Merge pull request #270 from Haidra-Org/main63a3917style: fix19dae45feat: addssl_contextarge726c55docs: add missing ssl_context docstringa9260d9ci: remove pre-commit job steps; replaced by dedi. actione80451ffix: use a defaultcertifibased ssl contextUpdates
horde-engineto 2.16.0Release notes
Sourced from horde-engine's releases.
Commits
2da5186Merge pull request #349 from Haidra-Org/maincd9497btests/fix: include all models required for tests as fixtures1404626fix: prevent comfyui internals error with layer diffusion7aa80cdfeat: use latest horde depsf8f671efix: pin depqrcodeto 7.4.2b4f87adtests/fix: check flux n_iter same as other n_iter tests8b7eea1fix: longer timeout for long running civitai requests4197530feat: usetorch>=2.4.1andcu124by defaultddfce30feat: comfyui3bb4dec3b6e220ci: update changelogUpdates
horde-model-referenceto 0.9.1Release notes
Sourced from horde-model-reference's releases.
Commits
d1a64a9Merge pull request #148 from Haidra-Org/main825cf0eci: remove pre-commit github action (replaced by https://pre-commit.ci/)a954f5dchore: update black pre-commit hook64c5681[pre-commit.ci] pre-commit autoupdate2bac968build(deps-dev): bump the python-packages group across 1 directory with 6 upd...a3e2377fix: set new type hint for otherrequirementsfields8bd5607fix: allow floats for model requirement valuese6dfd02ci:pre-commit.ciconfigUpdates
mypyfrom 1.11.2 to 1.12.0Changelog
Sourced from mypy's changelog.
... (truncated)
Commits
f2a39b1Update version to 1.12.0b4ec37aAdd one more 1.12 changelog item (#17936)cc1c679Better handling of generic functions in partial plugin (#17925)d65a013Add latest 1.12 changes to changelog (#17921)2e38965Fix union callees with functools.partial (#17903)c5d3673Document ReadOnly (PEP 705) (#17905)964a7a5Make ReadOnly TypedDict items covariant (#17904)24bfb34Include CHANGELOG.md in sdists (#17882)21d46eddocumentation for TypeIs (#17821)c692943Improvements to functools.partial of types (#17898)Updates
blackfrom 24.8.0 to 24.10.0Release notes
Sourced from black's releases.
Changelog
Sourced from black's changelog.
Commits
1b2427aPrepare release 24.10.0 (#4471)a22b1ebAdd mypyc 3.13 wheel build (#4449)b7d0e72Bump AndreMiras/coveralls-python-action from 65c1672f0b8a201702d86c81b79187df...f1a2f92Include --unstable in cache key (#4466)8d9d18cFix skipping Jupyter cells with unknown %% magic (#4462)bbfdba3Fix docs CI: use venv for uv to fix 'failed to create directory' (#4460)8fb2addUse builtin generics (#4458)2a45cecFix crashes with comments in parentheses (#4453)b4d6d86Drop Python 3.8 support (#4452)ac018c1Require newer aiohttp for blackd (#4451)Updates
rufffrom 0.6.5 to 0.7.0Release notes
Sourced from ruff's releases.
... (truncated)
Changelog
Sourced from ruff's changelog.
... (truncated)
Commits
5e6de4eChangelog for Ruff v0.7 (#13794)70e5c4aRecodeTRY302toTRY203(#13502)9218d6bRemoveallow-unused-importssetting from the common lint options (#13677)1b79ae9[ruff-0.7] Stabilise the expansion ofopen-file-with-context-handlerto wor...2b87587[flake8-pytest-style] Fix defaults whenlint.flake8-pytest-styleconfig s...d1e15f6Remove tab-size setting (#12835)89a8215Remove error messages for removed CLI aliases (#12833)202c6a6Removeoutput-format=textsetting (#12836)5c3c0c4[red-knot] Inference for comparison of union types (#13781)6b7a738Add explanation of fixable in--statisticscommand (#13774)Updates
toxto 4.23.0Release notes
Sourced from tox's releases.
Changelog
Sourced from tox's changelog.
... (truncated)
Commits
7696a8erelease 4.23.0c01a023Add NETRC to the default_pass_env list (#3410)971e7dareplace tool.pyproject and tool.tox.pyproject with tool.tox in config… (#3411)81a83bdFix typo in docs for dependency_groupsb912cb2release 4.22.0568b90fAdd dependency-groups support (PEP-735) (#3409)f919d0dExpose type checking dependencies into an extra (#3404)7c0e3ed[pre-commit.ci] pre-commit autoupdate (#3406)6db3730Fix the fix environment definition (#3407)2bf065aBump pypa/gh-action-pypi-publish from 1.10.2 to 1.10.3 (#3392)Updates
pre-committo 4.0.1Release notes
Sourced from pre-commit's releases.
Changelog
Sourced from pre-commit's changelog.
... (truncated)
Commits
cc4a522v4.0.1772d7d4Merge pull request #3324 from pre-commit/migrate-config-purelib222c62bfix migrate-config for purelib yaml3d5548bMerge pull request #3323 from pre-commit/pre-commit-ci-update-config4235a87[pre-commit.ci] pre-commit autoupdatedbccd57v4.0.0d07e529Merge pull request #3320 from pre-commit/remove-python-venv801b956remove deprecated python_venv aliasa2f7b80Merge pull request #3315 from pre-commit/warn-deprecated-stage-names-on-initd317223add warning for deprecates stages for remote repos on initDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show