search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.27k stars 4.73k forks source link

GC.AllocateUninitializedArray has regressed on macOS #65198

Open adamsitnik opened 2 years ago

adamsitnik commented 2 years ago

GC.AllocateUninitializedArray benchmarks have regressed by 15% for "smaller size" (1000 elements) only on macOS (x64, I don't have arm64 data). Other Unixes are not affected and larger sizes (10000 elements) are not affected..

.NET 6: | Type | Method | length | pinned | Mean | Error | StdDev | Median | Min | Max | Gen 0 | Gen 1 | Gen 2 | Allocated | |-------------- |--------------------------- |------- |------- |------------:|-----------:|-----------:|------------:|------------:|-----------:|--------:|--------:|-------:|----------:| | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | False | 99.14 ns | 2.948 ns | 3.276 ns | 97.28 ns | 96.46 ns | 108.0 ns | 0.4891 | - | - | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | False | 192.16 ns | 0.779 ns | 0.729 ns | 191.89 ns | 191.14 ns | 193.4 ns | 0.9669 | - | - | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | True | 378.25 ns | 4.336 ns | 3.843 ns | 376.98 ns | 373.63 ns | 385.4 ns | 1.9396 | 1.9396 | 0.3246 | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | True | 527.51 ns | 35.474 ns | 34.840 ns | 537.93 ns | 440.44 ns | 558.6 ns | 3.8341 | 3.8341 | 0.6409 | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | False | 460.30 ns | 13.229 ns | 12.993 ns | 465.62 ns | 433.22 ns | 472.1 ns | 4.7617 | - | - | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | False | 863.26 ns | 28.192 ns | 31.336 ns | 868.93 ns | 799.42 ns | 905.1 ns | 9.5207 | - | - | 19.55 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | True | 2,079.43 ns | 193.755 ns | 215.358 ns | 2,184.01 ns | 1,616.18 ns | 2,218.1 ns | 18.7916 | 18.7916 | 3.1319 | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | True | 4,294.48 ns | 339.558 ns | 391.035 ns | 4,411.58 ns | 3,194.01 ns | 4,566.0 ns | 36.7647 | 36.7647 | 6.1275 | 19.56 KB | .NET 7 Preview 1: | Type | Method | length | pinned | Mean | Error | StdDev | Median | Min | Max | Gen 0 | Gen 1 | Gen 2 | Allocated | |-------------- |--------------------------- |------- |------- |-----------:|----------:|----------:|-----------:|-----------:|-----------:|--------:|--------:|-------:|----------:| | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | False | 115.0 ns | 3.62 ns | 4.02 ns | 115.7 ns | 107.7 ns | 120.7 ns | 0.4894 | - | - | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | False | 218.5 ns | 10.22 ns | 11.76 ns | 221.6 ns | 197.3 ns | 238.9 ns | 0.9663 | - | - | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | True | 408.4 ns | 44.27 ns | 49.21 ns | 403.0 ns | 314.0 ns | 498.6 ns | 1.9366 | 1.9366 | 0.3239 | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | True | 622.0 ns | 7.05 ns | 6.25 ns | 621.5 ns | 607.3 ns | 630.9 ns | 3.8293 | 3.8293 | 0.6402 | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | False | 502.1 ns | 36.32 ns | 40.37 ns | 500.5 ns | 441.9 ns | 573.4 ns | 4.7603 | - | - | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | False | 834.6 ns | 24.70 ns | 25.36 ns | 844.3 ns | 785.4 ns | 860.9 ns | 9.5210 | - | - | 19.55 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | True | 2,328.8 ns | 272.49 ns | 313.79 ns | 2,413.5 ns | 1,747.3 ns | 2,679.2 ns | 18.7466 | 18.7466 | 3.1301 | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | True | 4,128.7 ns | 395.66 ns | 439.78 ns | 4,314.0 ns | 3,130.7 ns | 4,385.4 ns | 36.7561 | 36.7561 | 6.1260 | 19.56 KB |

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter '*AllocateUninitializedArray' --bdn-arguments "--join true"
## System.Tests.Perf_GC.AllocateArray(length: 10000, pinned: False) | Result | Base | Diff | Ratio | Alloc Delta | Modality | Operating System | Bit | Processor Name | Base V | Diff V | | ------ | -------:| -------:| -----:| -----------:| -------- | -------------------- | ----- | ----------------------------------------------- | ------------- | ----------- | | Same | 248.92 | 243.83 | 1.02 | +0 | | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 179.83 | 184.15 | 0.98 | +0 | | Windows 11 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 459.82 | 461.10 | 1.00 | +0 | | Windows 10 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Faster | 914.81 | 709.96 | 1.29 | +0 | | Windows 11 | X64 | Intel Core i5-4300U CPU 1.90GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Same | 312.96 | 304.02 | 1.03 | +0 | bimodal | Windows 10 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 298.12 | 327.61 | 0.91 | +0 | | Windows 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 307.02 | 307.68 | 1.00 | +0 | several? | Windows 11 | X64 | Intel Core i9-9900T CPU 2.10GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 584.23 | 542.31 | 1.08 | +0 | several? | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 420.18 | 431.82 | 0.97 | +0 | | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 301.75 | 307.80 | 0.98 | +0 | | ubuntu 20.04 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 483.25 | 487.37 | 0.99 | +0 | | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 1248.19 | 1157.88 | 1.08 | +0 | | centos 7 | X64 | Intel Xeon CPU E5530 2.40GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 807.84 | 747.88 | 1.08 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge) | 6.0.121.56705 | 7.0.22.7608| | Same | 372.58 | 389.82 | 0.96 | +0 | | alpine 3.13 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 391.98 | 388.72 | 1.01 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 365.02 | 365.98 | 1.00 | +0 | | ubuntu 20.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 610.43 | 598.45 | 1.02 | +0 | | Windows 10 | Arm64 | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 269.72 | 275.66 | 0.98 | +0 | | Windows 11 | X86 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 489.28 | 492.36 | 0.99 | +0 | | Windows 10 | X86 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 827.06 | 821.72 | 1.01 | +0 | | Windows 10 | Arm | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Slower | 853.39 | 1028.78 | 0.83 | +0 | | macOS Big Sur 11.6.3 | X64 | Intel Core i5-4278U CPU 2.60GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Slower | 734.86 | 859.05 | 0.86 | +0 | | macOS Big Sur 11.4 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | 6.0.121.56705 | 7.0.22.7608| ## System.Tests.Perf_GC.AllocateUninitializedArray(length: 1000, pinned: False) | Result | Base | Diff | Ratio | Alloc Delta | Modality | Operating System | Bit | Processor Name | Base V | Diff V | | ------ | ------:| ------:| -----:| -----------:| ---------- | -------------------- | ----- | ----------------------------------------------- | ------------- | ----------- | | Same | 28.63 | 28.43 | 1.01 | +0 | | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 19.49 | 20.59 | 0.95 | +0 | | Windows 11 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 47.97 | 48.42 | 0.99 | +0 | | Windows 10 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Faster | 96.60 | 73.23 | 1.32 | +0 | | Windows 11 | X64 | Intel Core i5-4300U CPU 1.90GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Same | 32.77 | 32.21 | 1.02 | +0 | several? | Windows 10 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 30.69 | 32.90 | 0.93 | +0 | | Windows 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 31.55 | 32.59 | 0.97 | +0 | several? | Windows 11 | X64 | Intel Core i9-9900T CPU 2.10GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 59.60 | 62.74 | 0.95 | +0 | | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 42.11 | 46.20 | 0.91 | +0 | several? | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 37.46 | 38.63 | 0.97 | +0 | | ubuntu 20.04 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 53.49 | 54.05 | 0.99 | +0 | | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 148.24 | 138.95 | 1.07 | +0 | | centos 7 | X64 | Intel Xeon CPU E5530 2.40GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 87.90 | 85.15 | 1.03 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge) | 6.0.121.56705 | 7.0.22.7608| | Same | 43.12 | 43.76 | 0.99 | +0 | | alpine 3.13 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 50.21 | 48.88 | 1.03 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 46.33 | 46.50 | 1.00 | +0 | multimodal | ubuntu 20.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 64.90 | 65.37 | 0.99 | +0 | | Windows 10 | Arm64 | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 29.11 | 30.75 | 0.95 | +0 | | Windows 11 | X86 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 50.59 | 52.71 | 0.96 | +0 | | Windows 10 | X86 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 94.16 | 96.78 | 0.97 | +0 | | Windows 10 | Arm | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Slower | 92.65 | 109.98 | 0.84 | +0 | | macOS Big Sur 11.6.3 | X64 | Intel Core i5-4278U CPU 2.60GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Slower | 77.08 | 91.51 | 0.84 | +0 | | macOS Big Sur 11.4 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | 6.0.121.56705 | 7.0.22.7608|

cc @VSadov

ghost commented 2 years ago

Tagging subscribers to this area: @dotnet/area-system-runtime See info in area-owners.md if you want to be subscribed.

Issue Details
`GC.AllocateUninitializedArray` benchmarks have regressed by 15% for "smaller size" (1000 elements) only on macOS (x64, I don't have arm64 data). Other Unixes are not affected.
.NET 6: | Type | Method | length | pinned | Mean | Error | StdDev | Median | Min | Max | Gen 0 | Gen 1 | Gen 2 | Allocated | |-------------- |--------------------------- |------- |------- |------------:|-----------:|-----------:|------------:|------------:|-----------:|--------:|--------:|-------:|----------:| | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | False | 99.14 ns | 2.948 ns | 3.276 ns | 97.28 ns | 96.46 ns | 108.0 ns | 0.4891 | - | - | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | False | 192.16 ns | 0.779 ns | 0.729 ns | 191.89 ns | 191.14 ns | 193.4 ns | 0.9669 | - | - | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | True | 378.25 ns | 4.336 ns | 3.843 ns | 376.98 ns | 373.63 ns | 385.4 ns | 1.9396 | 1.9396 | 0.3246 | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | True | 527.51 ns | 35.474 ns | 34.840 ns | 537.93 ns | 440.44 ns | 558.6 ns | 3.8341 | 3.8341 | 0.6409 | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | False | 460.30 ns | 13.229 ns | 12.993 ns | 465.62 ns | 433.22 ns | 472.1 ns | 4.7617 | - | - | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | False | 863.26 ns | 28.192 ns | 31.336 ns | 868.93 ns | 799.42 ns | 905.1 ns | 9.5207 | - | - | 19.55 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | True | 2,079.43 ns | 193.755 ns | 215.358 ns | 2,184.01 ns | 1,616.18 ns | 2,218.1 ns | 18.7916 | 18.7916 | 3.1319 | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | True | 4,294.48 ns | 339.558 ns | 391.035 ns | 4,411.58 ns | 3,194.01 ns | 4,566.0 ns | 36.7647 | 36.7647 | 6.1275 | 19.56 KB | .NET 7 Preview 1: | Type | Method | length | pinned | Mean | Error | StdDev | Median | Min | Max | Gen 0 | Gen 1 | Gen 2 | Allocated | |-------------- |--------------------------- |------- |------- |-----------:|----------:|----------:|-----------:|-----------:|-----------:|--------:|--------:|-------:|----------:| | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | False | 115.0 ns | 3.62 ns | 4.02 ns | 115.7 ns | 107.7 ns | 120.7 ns | 0.4894 | - | - | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | False | 218.5 ns | 10.22 ns | 11.76 ns | 221.6 ns | 197.3 ns | 238.9 ns | 0.9663 | - | - | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | True | 408.4 ns | 44.27 ns | 49.21 ns | 403.0 ns | 314.0 ns | 498.6 ns | 1.9366 | 1.9366 | 0.3239 | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | True | 622.0 ns | 7.05 ns | 6.25 ns | 621.5 ns | 607.3 ns | 630.9 ns | 3.8293 | 3.8293 | 0.6402 | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | False | 502.1 ns | 36.32 ns | 40.37 ns | 500.5 ns | 441.9 ns | 573.4 ns | 4.7603 | - | - | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | False | 834.6 ns | 24.70 ns | 25.36 ns | 844.3 ns | 785.4 ns | 860.9 ns | 9.5210 | - | - | 19.55 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | True | 2,328.8 ns | 272.49 ns | 313.79 ns | 2,413.5 ns | 1,747.3 ns | 2,679.2 ns | 18.7466 | 18.7466 | 3.1301 | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | True | 4,128.7 ns | 395.66 ns | 439.78 ns | 4,314.0 ns | 3,130.7 ns | 4,385.4 ns | 36.7561 | 36.7561 | 6.1260 | 19.56 KB |
Repro: ```cmd git clone https://github.com/dotnet/performance.git python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter '*AllocateUninitializedArray' --bdn-arguments "--join true" ```
## System.Tests.Perf_GC.AllocateArray(length: 10000, pinned: False) | Result | Base | Diff | Ratio | Alloc Delta | Modality | Operating System | Bit | Processor Name | Base V | Diff V | | ------ | -------:| -------:| -----:| -----------:| -------- | -------------------- | ----- | ----------------------------------------------- | ------------- | ----------- | | Same | 248.92 | 243.83 | 1.02 | +0 | | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 179.83 | 184.15 | 0.98 | +0 | | Windows 11 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 459.82 | 461.10 | 1.00 | +0 | | Windows 10 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Faster | 914.81 | 709.96 | 1.29 | +0 | | Windows 11 | X64 | Intel Core i5-4300U CPU 1.90GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Same | 312.96 | 304.02 | 1.03 | +0 | bimodal | Windows 10 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 298.12 | 327.61 | 0.91 | +0 | | Windows 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 307.02 | 307.68 | 1.00 | +0 | several? | Windows 11 | X64 | Intel Core i9-9900T CPU 2.10GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 584.23 | 542.31 | 1.08 | +0 | several? | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 420.18 | 431.82 | 0.97 | +0 | | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 301.75 | 307.80 | 0.98 | +0 | | ubuntu 20.04 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 483.25 | 487.37 | 0.99 | +0 | | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 1248.19 | 1157.88 | 1.08 | +0 | | centos 7 | X64 | Intel Xeon CPU E5530 2.40GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 807.84 | 747.88 | 1.08 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge) | 6.0.121.56705 | 7.0.22.7608| | Same | 372.58 | 389.82 | 0.96 | +0 | | alpine 3.13 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 391.98 | 388.72 | 1.01 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 365.02 | 365.98 | 1.00 | +0 | | ubuntu 20.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 610.43 | 598.45 | 1.02 | +0 | | Windows 10 | Arm64 | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 269.72 | 275.66 | 0.98 | +0 | | Windows 11 | X86 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 489.28 | 492.36 | 0.99 | +0 | | Windows 10 | X86 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 827.06 | 821.72 | 1.01 | +0 | | Windows 10 | Arm | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Slower | 853.39 | 1028.78 | 0.83 | +0 | | macOS Big Sur 11.6.3 | X64 | Intel Core i5-4278U CPU 2.60GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Slower | 734.86 | 859.05 | 0.86 | +0 | | macOS Big Sur 11.4 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | 6.0.121.56705 | 7.0.22.7608| ## System.Tests.Perf_GC.AllocateUninitializedArray(length: 1000, pinned: False) | Result | Base | Diff | Ratio | Alloc Delta | Modality | Operating System | Bit | Processor Name | Base V | Diff V | | ------ | ------:| ------:| -----:| -----------:| ---------- | -------------------- | ----- | ----------------------------------------------- | ------------- | ----------- | | Same | 28.63 | 28.43 | 1.01 | +0 | | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 19.49 | 20.59 | 0.95 | +0 | | Windows 11 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 47.97 | 48.42 | 0.99 | +0 | | Windows 10 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Faster | 96.60 | 73.23 | 1.32 | +0 | | Windows 11 | X64 | Intel Core i5-4300U CPU 1.90GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Same | 32.77 | 32.21 | 1.02 | +0 | several? | Windows 10 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 30.69 | 32.90 | 0.93 | +0 | | Windows 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 31.55 | 32.59 | 0.97 | +0 | several? | Windows 11 | X64 | Intel Core i9-9900T CPU 2.10GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 59.60 | 62.74 | 0.95 | +0 | | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 42.11 | 46.20 | 0.91 | +0 | several? | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 37.46 | 38.63 | 0.97 | +0 | | ubuntu 20.04 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 53.49 | 54.05 | 0.99 | +0 | | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 148.24 | 138.95 | 1.07 | +0 | | centos 7 | X64 | Intel Xeon CPU E5530 2.40GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 87.90 | 85.15 | 1.03 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge) | 6.0.121.56705 | 7.0.22.7608| | Same | 43.12 | 43.76 | 0.99 | +0 | | alpine 3.13 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 50.21 | 48.88 | 1.03 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 46.33 | 46.50 | 1.00 | +0 | multimodal | ubuntu 20.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 64.90 | 65.37 | 0.99 | +0 | | Windows 10 | Arm64 | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 29.11 | 30.75 | 0.95 | +0 | | Windows 11 | X86 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 50.59 | 52.71 | 0.96 | +0 | | Windows 10 | X86 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 94.16 | 96.78 | 0.97 | +0 | | Windows 10 | Arm | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Slower | 92.65 | 109.98 | 0.84 | +0 | | macOS Big Sur 11.6.3 | X64 | Intel Core i5-4278U CPU 2.60GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Slower | 77.08 | 91.51 | 0.84 | +0 | | macOS Big Sur 11.4 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | 6.0.121.56705 | 7.0.22.7608|
cc @VSadov
Author: adamsitnik
Assignees: -
Labels: `area-System.Runtime`, `os-mac-os-x`, `tenet-performance`
Milestone: -
adamsitnik commented 2 years ago

System.Tests.Perf_GC<Byte>.NewOperator_Array(length: 1000) and few similar benchmarks has regressed as well.

ghost commented 2 years ago

Tagging subscribers to this area: @dotnet/gc See info in area-owners.md if you want to be subscribed.

Issue Details
`GC.AllocateUninitializedArray` benchmarks have regressed by 15% for "smaller size" (1000 elements) only on macOS (x64, I don't have arm64 data). Other Unixes are not affected and larger sizes (10000 elements) are not affected..
.NET 6: | Type | Method | length | pinned | Mean | Error | StdDev | Median | Min | Max | Gen 0 | Gen 1 | Gen 2 | Allocated | |-------------- |--------------------------- |------- |------- |------------:|-----------:|-----------:|------------:|------------:|-----------:|--------:|--------:|-------:|----------:| | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | False | 99.14 ns | 2.948 ns | 3.276 ns | 97.28 ns | 96.46 ns | 108.0 ns | 0.4891 | - | - | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | False | 192.16 ns | 0.779 ns | 0.729 ns | 191.89 ns | 191.14 ns | 193.4 ns | 0.9669 | - | - | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | True | 378.25 ns | 4.336 ns | 3.843 ns | 376.98 ns | 373.63 ns | 385.4 ns | 1.9396 | 1.9396 | 0.3246 | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | True | 527.51 ns | 35.474 ns | 34.840 ns | 537.93 ns | 440.44 ns | 558.6 ns | 3.8341 | 3.8341 | 0.6409 | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | False | 460.30 ns | 13.229 ns | 12.993 ns | 465.62 ns | 433.22 ns | 472.1 ns | 4.7617 | - | - | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | False | 863.26 ns | 28.192 ns | 31.336 ns | 868.93 ns | 799.42 ns | 905.1 ns | 9.5207 | - | - | 19.55 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | True | 2,079.43 ns | 193.755 ns | 215.358 ns | 2,184.01 ns | 1,616.18 ns | 2,218.1 ns | 18.7916 | 18.7916 | 3.1319 | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | True | 4,294.48 ns | 339.558 ns | 391.035 ns | 4,411.58 ns | 3,194.01 ns | 4,566.0 ns | 36.7647 | 36.7647 | 6.1275 | 19.56 KB | .NET 7 Preview 1: | Type | Method | length | pinned | Mean | Error | StdDev | Median | Min | Max | Gen 0 | Gen 1 | Gen 2 | Allocated | |-------------- |--------------------------- |------- |------- |-----------:|----------:|----------:|-----------:|-----------:|-----------:|--------:|--------:|-------:|----------:| | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | False | 115.0 ns | 3.62 ns | 4.02 ns | 115.7 ns | 107.7 ns | 120.7 ns | 0.4894 | - | - | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | False | 218.5 ns | 10.22 ns | 11.76 ns | 221.6 ns | 197.3 ns | 238.9 ns | 0.9663 | - | - | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 1000 | True | 408.4 ns | 44.27 ns | 49.21 ns | 403.0 ns | 314.0 ns | 498.6 ns | 1.9366 | 1.9366 | 0.3239 | 1 KB | | Perf_GC<Char> | AllocateUninitializedArray | 1000 | True | 622.0 ns | 7.05 ns | 6.25 ns | 621.5 ns | 607.3 ns | 630.9 ns | 3.8293 | 3.8293 | 0.6402 | 1.98 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | False | 502.1 ns | 36.32 ns | 40.37 ns | 500.5 ns | 441.9 ns | 573.4 ns | 4.7603 | - | - | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | False | 834.6 ns | 24.70 ns | 25.36 ns | 844.3 ns | 785.4 ns | 860.9 ns | 9.5210 | - | - | 19.55 KB | | Perf_GC<Byte> | AllocateUninitializedArray | 10000 | True | 2,328.8 ns | 272.49 ns | 313.79 ns | 2,413.5 ns | 1,747.3 ns | 2,679.2 ns | 18.7466 | 18.7466 | 3.1301 | 9.79 KB | | Perf_GC<Char> | AllocateUninitializedArray | 10000 | True | 4,128.7 ns | 395.66 ns | 439.78 ns | 4,314.0 ns | 3,130.7 ns | 4,385.4 ns | 36.7561 | 36.7561 | 6.1260 | 19.56 KB |
Repro: ```cmd git clone https://github.com/dotnet/performance.git python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter '*AllocateUninitializedArray' --bdn-arguments "--join true" ```
## System.Tests.Perf_GC.AllocateArray(length: 10000, pinned: False) | Result | Base | Diff | Ratio | Alloc Delta | Modality | Operating System | Bit | Processor Name | Base V | Diff V | | ------ | -------:| -------:| -----:| -----------:| -------- | -------------------- | ----- | ----------------------------------------------- | ------------- | ----------- | | Same | 248.92 | 243.83 | 1.02 | +0 | | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 179.83 | 184.15 | 0.98 | +0 | | Windows 11 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 459.82 | 461.10 | 1.00 | +0 | | Windows 10 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Faster | 914.81 | 709.96 | 1.29 | +0 | | Windows 11 | X64 | Intel Core i5-4300U CPU 1.90GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Same | 312.96 | 304.02 | 1.03 | +0 | bimodal | Windows 10 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 298.12 | 327.61 | 0.91 | +0 | | Windows 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 307.02 | 307.68 | 1.00 | +0 | several? | Windows 11 | X64 | Intel Core i9-9900T CPU 2.10GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 584.23 | 542.31 | 1.08 | +0 | several? | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 420.18 | 431.82 | 0.97 | +0 | | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 301.75 | 307.80 | 0.98 | +0 | | ubuntu 20.04 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 483.25 | 487.37 | 0.99 | +0 | | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 1248.19 | 1157.88 | 1.08 | +0 | | centos 7 | X64 | Intel Xeon CPU E5530 2.40GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 807.84 | 747.88 | 1.08 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge) | 6.0.121.56705 | 7.0.22.7608| | Same | 372.58 | 389.82 | 0.96 | +0 | | alpine 3.13 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 391.98 | 388.72 | 1.01 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 365.02 | 365.98 | 1.00 | +0 | | ubuntu 20.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 610.43 | 598.45 | 1.02 | +0 | | Windows 10 | Arm64 | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 269.72 | 275.66 | 0.98 | +0 | | Windows 11 | X86 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 489.28 | 492.36 | 0.99 | +0 | | Windows 10 | X86 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 827.06 | 821.72 | 1.01 | +0 | | Windows 10 | Arm | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Slower | 853.39 | 1028.78 | 0.83 | +0 | | macOS Big Sur 11.6.3 | X64 | Intel Core i5-4278U CPU 2.60GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Slower | 734.86 | 859.05 | 0.86 | +0 | | macOS Big Sur 11.4 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | 6.0.121.56705 | 7.0.22.7608| ## System.Tests.Perf_GC.AllocateUninitializedArray(length: 1000, pinned: False) | Result | Base | Diff | Ratio | Alloc Delta | Modality | Operating System | Bit | Processor Name | Base V | Diff V | | ------ | ------:| ------:| -----:| -----------:| ---------- | -------------------- | ----- | ----------------------------------------------- | ------------- | ----------- | | Same | 28.63 | 28.43 | 1.01 | +0 | | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 19.49 | 20.59 | 0.95 | +0 | | Windows 11 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 47.97 | 48.42 | 0.99 | +0 | | Windows 10 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Faster | 96.60 | 73.23 | 1.32 | +0 | | Windows 11 | X64 | Intel Core i5-4300U CPU 1.90GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Same | 32.77 | 32.21 | 1.02 | +0 | several? | Windows 10 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 30.69 | 32.90 | 0.93 | +0 | | Windows 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 31.55 | 32.59 | 0.97 | +0 | several? | Windows 11 | X64 | Intel Core i9-9900T CPU 2.10GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 59.60 | 62.74 | 0.95 | +0 | | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 42.11 | 46.20 | 0.91 | +0 | several? | Windows 11 | X64 | Unknown processor | 6.0.121.56705 | 7.0.22.7608| | Same | 37.46 | 38.63 | 0.97 | +0 | | ubuntu 20.04 | X64 | AMD Ryzen 9 5900X | 6.0.121.56705 | 7.0.22.7608| | Same | 53.49 | 54.05 | 0.99 | +0 | | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 148.24 | 138.95 | 1.07 | +0 | | centos 7 | X64 | Intel Xeon CPU E5530 2.40GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 87.90 | 85.15 | 1.03 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge) | 6.0.121.56705 | 7.0.22.7608| | Same | 43.12 | 43.76 | 0.99 | +0 | | alpine 3.13 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 50.21 | 48.88 | 1.03 | +0 | | ubuntu 18.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 46.33 | 46.50 | 1.00 | +0 | multimodal | ubuntu 20.04 | X64 | Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) | 6.0.121.56705 | 7.0.22.7608| | Same | 64.90 | 65.37 | 0.99 | +0 | | Windows 10 | Arm64 | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 29.11 | 30.75 | 0.95 | +0 | | Windows 11 | X86 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | 6.0.121.56705 | 7.0.22.7608| | Same | 50.59 | 52.71 | 0.96 | +0 | | Windows 10 | X86 | Intel Xeon CPU E5-1650 v4 3.60GHz | 6.0.121.56705 | 7.0.22.7608| | Same | 94.16 | 96.78 | 0.97 | +0 | | Windows 10 | Arm | Microsoft SQ1 3.0 GHz | 6.0.121.56705 | 7.0.22.7608| | Slower | 92.65 | 109.98 | 0.84 | +0 | | macOS Big Sur 11.6.3 | X64 | Intel Core i5-4278U CPU 2.60GHz (Haswell) | 6.0.121.56705 | 7.0.22.7608| | Slower | 77.08 | 91.51 | 0.84 | +0 | | macOS Big Sur 11.4 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | 6.0.121.56705 | 7.0.22.7608|
cc @VSadov
Author: adamsitnik
Assignees: -
Labels: `os-mac-os-x`, `tenet-performance`, `area-GC-coreclr`, `untriaged`
Milestone: -
adamsitnik commented 2 years ago

The issue still persists with preview2, some benchmarks are even up to 5 times slower. It's specific to macOS x64.

Type Method Job Runtime Toolchain Size Mean Error StdDev Median Min Max Ratio RatioSD Gen 0 Allocated Alloc Ratio
CtorGivenSize<Int32> Array Job-YIPOBQ .NET 6.0 net6.0 512 196.4 ns 4.55 ns 5.24 ns 193.7 ns 190.9 ns 209.7 ns 1.00 0.00 0.9899 2.02 KB 1.00
CtorGivenSize<Int32> Array Job-FCGZAV .NET 7.0 net7.0 512 1,062.0 ns 71.56 ns 82.40 ns 1,005.3 ns 992.6 ns 1,224.5 ns 5.40 0.34 12.6558 2.02 KB 1.00
CtorGivenSize<String> Array Job-YIPOBQ .NET 6.0 net6.0 512 414.2 ns 8.11 ns 8.32 ns 415.9 ns 382.5 ns 419.2 ns 1.00 0.00 1.9677 4.02 KB 1.00
CtorGivenSize<String> Array Job-FCGZAV .NET 7.0 net7.0 512 2,074.3 ns 7.78 ns 6.07 ns 2,075.9 ns 2,066.8 ns 2,086.6 ns 5.02 0.14 25.0000 4.02 KB 1.00
Maoni0 commented 2 years ago

thanks for reporting! @mokosan will be doing investigation on this.

mrsharm commented 2 years ago

Was able to repro this on my Mac after upgrading to Monterey. The pattern I am observing is that we have regressed for cases where we are not pinning and have improved for cases where we are pinning. Will be investigating this further. As a note, this issue regressed before we enabled Regions and is seemingly an OS specific regression.

.NET 6

BenchmarkDotNet=v0.13.1.1823-nightly, OS=macOS Monterey 12.3 (21E230) [Darwin 21.4.0] Intel Core i5-8210Y CPU 1.60GHz (Amber Lake Y), 1 CPU, 4 logical and 2 physical cores .NET SDK=7.0.100-rc.1.22409.23 [Host] : .NET 6.0.8 (6.0.822.36306), X64 RyuJIT Job-WCHAYM : .NET 6.0.8 (6.0.822.36306), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250.0000 ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1

Type Method length pinned Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
Perf_GC AllocateUninitializedArray 1000 False 116.9 ns 16.03 ns 17.15 ns 109.6 ns 98.29 ns 148.8 ns 0.4892 - - 1 KB
Perf_GC AllocateUninitializedArray 1000 False 191.5 ns 5.27 ns 5.64 ns 191.1 ns 181.34 ns 204.6 ns 0.9671 - - 1.98 KB
Perf_GC AllocateUninitializedArray 1000 True 524.4 ns 68.27 ns 75.88 ns 519.6 ns 401.84 ns 672.0 ns 1.9287 1.9287 0.3236 1 KB
Perf_GC AllocateUninitializedArray 1000 True 724.6 ns 31.27 ns 36.01 ns 736.8 ns 604.18 ns 762.8 ns 3.8248 3.8248 0.6398 1.98 KB
Perf_GC AllocateUninitializedArray 10000 False 595.0 ns 23.05 ns 25.62 ns 591.3 ns 541.17 ns 638.5 ns 4.7601 - - 9.79 KB
Perf_GC AllocateUninitializedArray 10000 False 1,048.9 ns 37.94 ns 43.69 ns 1,047.5 ns 926.95 ns 1,106.3 ns 9.5235 - - 19.55 KB
Perf_GC AllocateUninitializedArray 10000 True 2,625.9 ns 267.35 ns 286.07 ns 2,653.5 ns 1,862.02 ns 2,958.7 ns 18.7467 18.7467 3.1244 9.79 KB
Perf_GC AllocateUninitializedArray 10000 True 5,790.2 ns 185.26 ns 205.92 ns 5,788.5 ns 5,389.56 ns 6,071.2 ns 36.6209 36.6209 6.1206 19.56 KB

.NET 7

BenchmarkDotNet=v0.13.1.1823-nightly, OS=macOS Monterey 12.3 (21E230) [Darwin 21.4.0] Intel Core i5-8210Y CPU 1.60GHz (Amber Lake Y), 1 CPU, 4 logical and 2 physical cores .NET SDK=7.0.100-rc.1.22409.23 [Host] : .NET 7.0.0 (7.0.22.40308), X64 RyuJIT Job-LJYITB : .NET 7.0.0 (7.0.22.40308), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250.0000 ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1

Type Method length pinned Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
Perf_GC AllocateUninitializedArray 1000 False 583.9 ns 21.98 ns 23.52 ns 578.2 ns 560.3 ns 641.4 ns 6.2490 - - 1 KB
Perf_GC AllocateUninitializedArray 1000 False 1,189.7 ns 23.07 ns 21.58 ns 1,194.1 ns 1,153.3 ns 1,238.0 ns 12.3433 - - 1.98 KB
Perf_GC AllocateUninitializedArray 1000 True 356.9 ns 4.14 ns 3.87 ns 356.9 ns 349.5 ns 362.9 ns 1.9396 1.9396 0.3244 1 KB
Perf_GC AllocateUninitializedArray 1000 True 535.5 ns 17.27 ns 19.89 ns 533.8 ns 505.7 ns 575.0 ns 3.8257 3.8257 0.6394 1.98 KB
Perf_GC AllocateUninitializedArray 10000 False 5,174.0 ns 100.51 ns 98.72 ns 5,149.5 ns 5,060.3 ns 5,379.8 ns 57.1254 2.0353 - 9.79 KB
Perf_GC AllocateUninitializedArray 10000 False 5,411.1 ns 135.97 ns 151.14 ns 5,351.5 ns 5,211.1 ns 5,692.5 ns 57.2404 3.8174 - 19.55 KB
Perf_GC AllocateUninitializedArray 10000 True 2,052.9 ns 45.34 ns 52.22 ns 2,058.8 ns 1,974.3 ns 2,177.5 ns 18.7315 18.7315 3.1286 9.79 KB
Perf_GC AllocateUninitializedArray 10000 True 3,935.5 ns 63.14 ns 59.06 ns 3,921.6 ns 3,871.0 ns 4,079.2 ns 36.6549 36.6549 6.1220 19.56 KB
jozkee commented 2 years ago

Here are the results from the 6.0 vs 7.0-rc2 report, we will tag this as "Under investigation" in the report.

## System.Tests.Perf_GC.AllocateUninitializedArray(length: 10000, pinned: True) | Result | Ratio | Alloc Delta | Operating System | Bit | Processor Name | Modality| | ------ | -----:| -----------:| ------------------- | ----- | --------------------------------------------- | -------- | | Same | 0.97 | +0 | ubuntu 18.04 | Arm64 | Unknown processor | | | Same | 0.97 | +0 | Windows 11 | Arm64 | Unknown processor | | | Slower | 0.85 | +0 | Windows 11 | Arm64 | Microsoft SQ1 3.0 GHz | | | Slower | 0.81 | +0 | Windows 11 | Arm64 | Microsoft SQ1 3.0 GHz | | | Same | 0.98 | +0 | macOS Monterey 12.6 | Arm64 | Apple M1 | | | Slower | 0.85 | +0 | macOS Monterey 12.6 | Arm64 | Apple M1 Max | | | Slower | 0.71 | +0 | Windows 10 | X64 | Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) | | | Same | 0.99 | +0 | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | | | Slower | 0.90 | +0 | Windows 11 | X64 | AMD Ryzen 9 5900X | | | Same | 0.92 | +0 | Windows 11 | X64 | AMD Ryzen 9 7950X | | | Slower | 0.81 | -1 | Windows 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | | | Same | 0.96 | +0 | debian 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | | | Same | 0.98 | +0 | ubuntu 18.04 | X64 | AMD Ryzen 9 5900X | | | Same | 0.96 | +0 | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | | | Same | 0.95 | +0 | ubuntu 20.04 | X64 | AMD Ryzen 9 5900X | | | Slower | 0.67 | -1 | ubuntu 20.04 | X64 | Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) | | | Same | 0.94 | +0 | ubuntu 20.04 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | several?| | Slower | 0.27 | -1 | macOS Big Sur 11.7 | X64 | Intel Core i5-4278U CPU 2.60GHz (Haswell) | | | Same | 0.95 | +0 | macOS Monterey 12.6 | X64 | Intel Core i7-4870HQ CPU 2.50GHz (Haswell) | | ## System.Tests.Perf_GC.AllocateUninitializedArray(length: 10000, pinned: True) | Result | Ratio | Alloc Delta | Operating System | Bit | Processor Name | Modality| | ------ | -----:| -----------:| ------------------- | ----- | --------------------------------------------- | --------:| | Same | 0.93 | +0 | ubuntu 18.04 | Arm64 | Unknown processor | | | Same | 0.99 | +0 | Windows 11 | Arm64 | Unknown processor | | | Slower | 0.82 | +0 | Windows 11 | Arm64 | Microsoft SQ1 3.0 GHz | | | Slower | 0.81 | +0 | Windows 11 | Arm64 | Microsoft SQ1 3.0 GHz | | | Same | 1.03 | +0 | macOS Monterey 12.6 | Arm64 | Apple M1 | | | Same | 0.89 | +0 | macOS Monterey 12.6 | Arm64 | Apple M1 Max | | | Slower | 0.72 | +0 | Windows 10 | X64 | Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) | | | Same | 0.97 | +0 | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | | | Same | 0.92 | +0 | Windows 11 | X64 | AMD Ryzen 9 5900X | | | Same | 0.93 | +0 | Windows 11 | X64 | AMD Ryzen 9 7950X | | | Slower | 0.82 | +0 | Windows 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | | | Same | 0.92 | +0 | debian 11 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | | | Same | 0.99 | +0 | ubuntu 18.04 | X64 | AMD Ryzen 9 5900X | | | Same | 0.93 | +0 | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | | | Same | 0.90 | +0 | ubuntu 20.04 | X64 | AMD Ryzen 9 5900X | | | Slower | 0.47 | +1 | ubuntu 20.04 | X64 | Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) | | | Same | 0.90 | +0 | ubuntu 20.04 | X64 | Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) | | | Slower | 0.32 | +0 | macOS Big Sur 11.7 | X64 | Intel Core i5-4278U CPU 2.60GHz (Haswell) | | | Same | 0.92 | +0 | macOS Monterey 12.6 | X64 | Intel Core i7-4870HQ CPU 2.50GHz (Haswell) | |
dakersnar commented 1 year ago

Here are the results from the 7.0 vs 6.0 report confirming the regression.

## System.Tests.Perf_GC.AllocateUninitializedArray(length: 10000, pinned: True) | Result | Ratio | Alloc Delta | Operating System | Bit | Processor Name | Modality| | ------ | -----:| -----------:| ------------------- | ----- | --------------------------------------------- | --------:| | Slower | 0.51 | +0 | Windows 11 | Arm64 | Microsoft SQ1 3.0 GHz | | | Same | 1.04 | +1 | macOS 13.0 | Arm64 | Apple M1 | | | Slower | 0.53 | +0 | Windows 10 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | | | Slower | 0.85 | +0 | Windows 10 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | | | Slower | 0.78 | +0 | Windows 10 | X64 | Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) | | | Same | 1.00 | +0 | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | | | Same | 0.92 | +0 | Windows 11 | X64 | 11th Gen Intel Core i9-11900 2.50GHz | | | Same | 0.92 | +0 | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | | | Same | 1.04 | +0 | raspbian 11 | Arm | ARMv7 Processor rev 3 (v7l) | | | Slower | 0.90 | +0 | macOS Monterey 12.6 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | | ## System.Tests.Perf_GC.AllocateUninitializedArray(length: 10000, pinned: True) | Result | Ratio | Alloc Delta | Operating System | Bit | Processor Name | Modality| | ------ | -----:| -----------:| ------------------- | ----- | --------------------------------------------- | --------:| | Slower | 0.49 | +0 | Windows 11 | Arm64 | Microsoft SQ1 3.0 GHz | | | Same | 0.98 | +0 | macOS 13.0 | Arm64 | Apple M1 | | | Slower | 0.49 | +0 | Windows 10 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | | | Slower | 0.86 | +0 | Windows 10 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | | | Slower | 0.73 | -2 | Windows 10 | X64 | Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) | | | Same | 1.01 | +0 | Windows 11 | X64 | AMD Ryzen Threadripper PRO 3945WX 12-Cores | | | Same | 0.94 | +0 | Windows 11 | X64 | 11th Gen Intel Core i9-11900 2.50GHz | | | Same | 0.91 | +0 | ubuntu 18.04 | X64 | Intel Xeon CPU E5-1650 v4 3.60GHz | | | Same | 1.03 | +0 | raspbian 11 | Arm | ARMv7 Processor rev 3 (v7l) | | | Same | 0.97 | +0 | macOS Monterey 12.6 | X64 | Intel Core i7-5557U CPU 3.10GHz (Broadwell) | |
mrsharm commented 1 year ago

Closing in lieu of https://github.com/dotnet/runtime/issues/73592

mrsharm commented 1 year ago

Reopening to test this thoroughly on MacOS.