search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.96k stars 4.65k forks source link

.NET 9 per-preview performance overview #98209

Open kotlarmilos opened 7 months ago

kotlarmilos commented 7 months ago

Description

This issue provides an overview of the all performance regressions from microbenchmark automated runs in WASM, Mono AOT, and Interpreter during the timeframe of .NET 9 preview releases. WASM regressions are tracked separately at https://github.com/dotnet/runtime/issues/96444.

List of queries for autofilling issues:

Mono AOT

Summary of the most severe unresolved regressions from .NET 9

Details ### RC1 Issue | Status | Comment -- | -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/39525 | **Resolved** | `ConcurrentDictionary`, affecting CoreCLR as well, x64/arm64 https://github.com/dotnet/perf-autofiling-issues/issues/39828 | Resolving | `Regex_Cache.IsMatch` regression, CoreCLR as well (https://github.com/dotnet/runtime/issues/106346) ### Preview 7 Issue | Status | Comment -- | -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/39238 | **Closed** | Likely caused LLVM 19 bump, no action https://github.com/dotnet/runtime/issues/105335 | **Resolved** | `ConcurrentBag` (threading) https://github.com/dotnet/runtime/issues/106475 | Investigation | `Perf_FloatingPointTensorPrimitives.MaxMagnitude` https://github.com/dotnet/perf-autofiling-issues/issues/38390 | **Closed** | `Base64EncodeDecodeInPlaceTests` https://github.com/dotnet/perf-autofiling-issues/issues/37939 | **Resolved** | Tagged causing PR, Tanner says by design. https://github.com/dotnet/runtime/issues/104302 | **Resolved** | `CredentialCacheTests`, no action expected regression https://github.com/dotnet/perf-autofiling-issues/issues/37022 | **Closed** | JSON and Buffer regressions, tagged author https://github.com/dotnet/runtime/issues/104022, https://github.com/dotnet/perf-autofiling-issues/issues/37021 | **Resolved** | Vector SIMD intrinsics (both x64 and arm64), tagged Tanner ### Preview 6 Issue | Status | Comment -- | -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/37020 | **Closed** | `System.Collections.IndexerSetReverse` https://github.com/dotnet/runtime/issues/103653 | Investigation | `Perf_Base64` https://github.com/dotnet/runtime/issues/103652 | Investigation | Vector SIMD intrinsics https://github.com/dotnet/runtime/issues/104076 | Resolving | `Perf_Double/Single/NumberCultureInfo`, tagged author, looks like Mono perf measurements weren't accurate before merging https://github.com/dotnet/runtime/issues/103460, https://github.com/dotnet/perf-autofiling-issues/issues/35612 | Resolving | `BigInteger`, tagged author ### Preview 5 Issue | Status | Comment -- | -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/35132 | **false pos.** | JSON regressions, tagged author https://github.com/dotnet/runtime/issues/102557 | **Backlog** | `VectorConvert`, x64 only https://github.com/dotnet/runtime/issues/102281 | **Resolved** | `CredentialCacheTests`, Assigned https://github.com/dotnet/perf-autofiling-issues/issues/33792 | **Resolved** | `UnboundedChannelPerfTests`, Tagged author, PR reverted https://github.com/dotnet/runtime/issues/106402 | Investigation | `GetType_Resolvers`, Tagged author ### Preview 4 Issue | Status | Comment -- | -- | -- https://github.com/dotnet/runtime/issues/102382 | **Resolved** | Authors notified https://github.com/dotnet/perf-autofiling-issues/issues/33133, https://github.com/dotnet/perf-autofiling-issues/issues/33094 | **Resolved** | Need intrinsifying BitCast API https://github.com/dotnet/runtime/issues/101495 https://github.com/dotnet/runtime/issues/104777 | Assigned | `System.Reflection.Invoke`, Authors notified https://github.com/dotnet/runtime/pull/100846 https://github.com/dotnet/runtime/issues/101124, https://github.com/dotnet/runtime/issues/101127 | Resolving | Authors notified https://github.com/dotnet/runtime/pull/99982 https://github.com/dotnet/perf-autofiling-issues/issues/32511 | **Resolved** ### Preview 3 Issue | Status | Comment -- | -- | -- https://github.com/dotnet/runtime/issues/106462 | **Closed** | `EventSourceLogger.NetScopes_TwoMessages` regressions, long range of commits, x64/arm64, caused by BDN change. https://github.com/dotnet/runtime/issues/106479 | Investigation | `Reflection.Invoke` regression, nothing suspicious in the range, x64/arm64 https://github.com/dotnet/runtime/issues/102865 | **Resolved** | `Perf_Single/Double.TryParse`, affecting CoreCLR as well ### Preview 2 Issue | Status -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/30456, https://github.com/dotnet/perf-autofiling-issues/issues/30503 | **Resolved** ### Preview 1 Issue | Status -- | -- https://github.com/dotnet/runtime/issues/102381 | Investigating https://github.com/dotnet/perf-autofiling-issues/issues/27018, https://github.com/dotnet/perf-autofiling-issues/issues/27047 | **Resolved** (no action) https://github.com/dotnet/perf-autofiling-issues/issues/25809 | **Archived** (https://github.com/dotnet/runtime/issues/97988) https://github.com/dotnet/perf-autofiling-issues/issues/24512 | **No action** https://github.com/dotnet/perf-autofiling-issues/issues/27020 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/27019 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/26504 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/26154 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/25812 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/25806 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/25089 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/25426 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/25421 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/24200 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/24203 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23314 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23579 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/24786 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/24198 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/22570 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/22569 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23306 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23307 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23309 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23311 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23580 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23582 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23870 | **Resolved**

Mono Interpreter

Summary of the most severe unresolved regressions from .NET 9:

Details ### RC1 Issue | Status -- | -- ### Preview 7 Issue | Status -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/38386 | Resolving https://github.com/dotnet/perf-autofiling-issues/issues/38426 | Resolving https://github.com/dotnet/perf-autofiling-issues/issues/38837 | Resolving https://github.com/dotnet/runtime/issues/106144 | Resolving https://github.com/dotnet/perf-autofiling-issues/issues/39545 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/38827 | **Resolved** ### Preview 6 Issue | Status -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/36491 | Resolving https://github.com/dotnet/perf-autofiling-issues/issues/37950 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/37898 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/37892 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/37889 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/36078, https://github.com/dotnet/perf-autofiling-issues/issues/36098 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/36973 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/36976, https://github.com/dotnet/perf-autofiling-issues/issues/36978, https://github.com/dotnet/perf-autofiling-issues/issues/37047 | **Resolved** ### Preview 5 Issue | Status -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/35112 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/35186 | **Resolved** ### Preview 4 Issue | Status -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/33948 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/32813 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/32770, https://github.com/dotnet/perf-autofiling-issues/issues/32770, https://github.com/dotnet/perf-autofiling-issues/issues/32772, https://github.com/dotnet/perf-autofiling-issues/issues/32812 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/34651 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/34665 | **Resolved** ### Preview 3 Issue | Status -- | -- ### Preview 2 Issue | Status -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/30463 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/29872 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/30464 | **Resolved** ### Preview 1 Issue | Status -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/29871 | Resolving https://github.com/dotnet/perf-autofiling-issues/issues/27704 | Resolving https://github.com/dotnet/perf-autofiling-issues/issues/29026 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23222, https://github.com/dotnet/perf-autofiling-issues/issues/23611, https://github.com/dotnet/perf-autofiling-issues/issues/23635 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/26185 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/29868 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/27348 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/27350, https://github.com/dotnet/perf-autofiling-issues/issues/27351, https://github.com/dotnet/perf-autofiling-issues/issues/27352 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/26192 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/22914, https://github.com/dotnet/perf-autofiling-issues/issues/22938 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/25465 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23877, https://github.com/dotnet/perf-autofiling-issues/issues/23880 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/22915 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/25084 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/24816 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/24812 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23615 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/23614 | **Resolved** https://github.com/dotnet/perf-autofiling-issues/issues/22539, https://github.com/dotnet/perf-autofiling-issues/issues/22533 | **Resolved**

Mono JIT

Summary of the most severe unresolved regressions from .NET 9

Mono Android

Issue Status Comment
https://github.com/dotnet/runtime/issues/106818 By design Android size regression on 07/08/24, expected by https://github.com/dotnet/runtime/pull/105714

Mono Size - iOS

Summary of the most severe unresolved regressions from .NET 9

Details ### Preview 7 Issue | Status | Comment -- | -- | -- https://github.com/dotnet/runtime/issues/104952 | **Tracked at #105701** | Probably caused by new APIs/generics ### Preview 6 Issue | Status | Comment -- | -- | -- https://github.com/dotnet/perf-autofiling-issues/issues/35656 | **Closed** | Tagged author, only 0.03MB https://github.com/dotnet/perf-autofiling-issues/issues/35768 | **Improvement** | ### Preview 5 Issue | Status | Comment --|--|-- https://github.com/dotnet/runtime/issues/104073 | **Tracked at #105701** | Tagged author https://github.com/dotnet/perf-autofiling-issues/issues/35226 | **MAUI improvement** | Long range, not clear cause ### Preview 4 (NAOT) Issue | Status | Comment --|--|-- https://github.com/dotnet/runtime/issues/104074 | **Resolved** | No action, triggered by our build setup (disabled full stripping of symbols) ### Preview 4 Date | Diff | Change | Size increase (bytes) | Size increase (%) | Tracking issue | Status --|--|--|--|--|--|-- 08 Apr 2024 | https://github.com/dotnet/runtime/compare/700d724f6abadb372cb38894bdd342b193afc518...8508806863c840203c5a652667b5f108c3a439f6 | https://github.com/dotnet/runtime/pull/99982 | 202000 | 3.63 | https://github.com/dotnet/runtime/issues/100975 | **Tracked at #105701** 19 Apr 2024 | https://github.com/dotnet/runtime/compare/a3bf7d4804c86f17087b216fec74d9b496ec0001...b067ce009ab5992168d1d635a9a9beb387008608 | https://github.com/dotnet/runtime/pull/101258 | 290000 | 1.4 | https://github.com/dotnet/runtime/issues/101488 | **Tracked at #105701** ### Preview 1 Date | Diff | Change | Size increase (bytes) | Size increase (%) | Tracking issue | Status --|--|--|--|--|--|-- 03 Oct 2023 | https://github.com/dotnet/runtime/compare/88f25474334bfb0da10e0b5acb0219a64a04828b...d2e24fa59d5259000b803cb4efc0dfd2c7ea0b37 | https://github.com/dotnet/runtime/pull/90764 | 066800 | 5.01 | https://github.com/dotnet/runtime/issues/93072 | **Tracked at #105701**

Mono Startup - iOS

Preview 1

Date Diff Change Slowdown (ms) Slowdown (%) Tracking issue Status
16 Dec 2023 https://github.com/dotnet/runtime/compare/22844957b51e7020215196d2896dbda6fefd449e...4dffd80c4d77c27e772a0be26e8036af77fbb26e https://github.com/dotnet/runtime/commit/4dffd80c4d77c27e772a0be26e8036af77fbb26e 8.3 18.8 https://github.com/dotnet/runtime/issues/96449 Resolved (no action)

MAUI

Issue Status Comment
https://github.com/dotnet/runtime/issues/104083 Investigating Incrementally slower startup times for MAUI-NAOT
vitek-karas commented 3 weeks ago

We may do some specific perf improvements still, but overall we're not going to address all of the changes in the release. Moving this to 10.