search

EgorBot / runtime-utils

MIT License
0 stars 1 forks source link

CET tests #157

Closed EgorBo closed 1 week ago

EgorBo commented 1 week ago

@EgorBot -pr 103654 -windows_aws_genoa -windows_aws_sapphirelake

using BenchmarkDotNet.Attributes;

public class Perf_Int16
{
    public static IEnumerable<object> StringValues => Values.Select(value => value.ToString()).ToArray();

    public static IEnumerable<object> Values => new object[]
    {
        short.MinValue,
        0,
        short.MaxValue
    };

    [Benchmark]
    [ArgumentsSource(nameof(StringValues))]
    [MemoryRandomization]
    public short Parse(string value) => short.Parse(value);

    [Benchmark]
    [ArgumentsSource(nameof(StringValues))]
    [MemoryRandomization]
    public bool TryParse(string value) => short.TryParse(value, out _);

    [Benchmark]
    [ArgumentsSource(nameof(Values))]
    [MemoryRandomization]
    public string ToString(short value) => value.ToString();
}
EgorBot commented 1 week ago

Benchmark results on windows_genoa

BenchmarkDotNet v0.14.0, Windows 10 (10.0.20348.2762)
AMD EPYC 9R14, 1 CPU, 16 logical and 16 physical cores
  Job-GRXFHJ : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  Job-ROCTGW : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
OutlierMode=DontRemove  MemoryRandomization=True
Method Job Toolchain value Mean Error Ratio
Parse Job-GRXFHJ \core_root_base\corerun.exe -32768 10.177 ns 0.0038 ns 1.00
Parse Job-ROCTGW \core_root_diff\corerun.exe -32768 10.240 ns 0.0210 ns 1.01
TryParse Job-GRXFHJ \core_root_base\corerun.exe -32768 9.997 ns 0.0431 ns 1.00
TryParse Job-ROCTGW \core_root_diff\corerun.exe -32768 10.105 ns 0.2204 ns 1.01
ToString Job-GRXFHJ \core_root_base\corerun.exe -32768 8.387 ns 0.2147 ns 1.00
ToString Job-ROCTGW \core_root_diff\corerun.exe -32768 8.990 ns 0.2310 ns 1.07
Parse Job-GRXFHJ \core_root_base\corerun.exe 0 6.139 ns 0.0212 ns 1.00
Parse Job-ROCTGW \core_root_diff\corerun.exe 0 6.134 ns 0.0250 ns 1.00
TryParse Job-GRXFHJ \core_root_base\corerun.exe 0 5.919 ns 0.0198 ns 1.00
TryParse Job-ROCTGW \core_root_diff\corerun.exe 0 5.891 ns 0.0022 ns 1.00
ToString Job-GRXFHJ \core_root_base\corerun.exe 0 1.877 ns 0.0742 ns 1.00
ToString Job-ROCTGW \core_root_diff\corerun.exe 0 1.599 ns 0.0941 ns 0.85
Parse Job-GRXFHJ \core_root_base\corerun.exe 32767 10.031 ns 0.0497 ns 1.00
Parse Job-ROCTGW \core_root_diff\corerun.exe 32767 10.039 ns 0.2156 ns 1.00
TryParse Job-GRXFHJ \core_root_base\corerun.exe 32767 10.088 ns 0.2298 ns 1.00
TryParse Job-ROCTGW \core_root_diff\corerun.exe 32767 9.766 ns 0.0296 ns 0.97
ToString Job-GRXFHJ \core_root_base\corerun.exe 32767 7.254 ns 0.2024 ns 1.00
ToString Job-ROCTGW \core_root_diff\corerun.exe 32767 6.670 ns 0.1949 ns 0.92

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt)

EgorBot commented 1 week ago

Benchmark results on windows_sapphirelake

BenchmarkDotNet v0.14.0, Windows 10 (10.0.20348.2762)
Intel Xeon Platinum 8488C, 1 CPU, 16 logical and 8 physical cores
  Job-BWKUJM : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  Job-IYIIYI : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
OutlierMode=DontRemove  MemoryRandomization=True
Method Job Toolchain value Mean Error Ratio
Parse Job-BWKUJM \core_root_base\corerun.exe -32768 7.815 ns 0.1208 ns 1.00
Parse Job-IYIIYI \core_root_diff\corerun.exe -32768 7.830 ns 0.1187 ns 1.00
TryParse Job-BWKUJM \core_root_base\corerun.exe -32768 7.180 ns 0.0483 ns 1.00
TryParse Job-IYIIYI \core_root_diff\corerun.exe -32768 7.216 ns 0.1312 ns 1.01
ToString Job-BWKUJM \core_root_base\corerun.exe -32768 11.605 ns 0.1548 ns 1.00
ToString Job-IYIIYI \core_root_diff\corerun.exe -32768 11.832 ns 0.3139 ns 1.02
Parse Job-BWKUJM \core_root_base\corerun.exe 0 4.151 ns 0.1229 ns 1.00
Parse Job-IYIIYI \core_root_diff\corerun.exe 0 4.141 ns 0.1233 ns 1.00
TryParse Job-BWKUJM \core_root_base\corerun.exe 0 4.562 ns 0.1317 ns 1.00
TryParse Job-IYIIYI \core_root_diff\corerun.exe 0 4.075 ns 0.0581 ns 0.90
ToString Job-BWKUJM \core_root_base\corerun.exe 0 2.021 ns 0.0251 ns 1.00
ToString Job-IYIIYI \core_root_diff\corerun.exe 0 2.035 ns 0.0341 ns 1.01
Parse Job-BWKUJM \core_root_base\corerun.exe 32767 7.543 ns 0.0955 ns 1.00
Parse Job-IYIIYI \core_root_diff\corerun.exe 32767 7.236 ns 0.1805 ns 0.96
TryParse Job-BWKUJM \core_root_base\corerun.exe 32767 7.870 ns 0.0887 ns 1.00
TryParse Job-IYIIYI \core_root_diff\corerun.exe 32767 7.761 ns 0.1661 ns 0.99
ToString Job-BWKUJM \core_root_base\corerun.exe 32767 10.009 ns 0.1273 ns 1.00
ToString Job-IYIIYI \core_root_diff\corerun.exe 32767 9.926 ns 0.1785 ns 0.99

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt)

EgorBo commented 1 week ago

@EgorBot -pr 103654 -windows_aws_milano -windows_aws_icelake

using BenchmarkDotNet.Attributes;

public class Perf_Int16
{
    static string _hex = Convert.ToHexString("This is a test of Convert."u8.ToArray());

    [Benchmark]
    public byte[] FromHexString() => Convert.FromHexString(_hex);
}
EgorBot commented 1 week ago

Benchmark results on windows_icelake

BenchmarkDotNet v0.14.0, Windows 10 (10.0.20348.2762)
Intel Xeon Platinum 8375C CPU 2.90GHz, 1 CPU, 16 logical and 8 physical cores
  Job-KWOVIE : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  Job-NCVQGH : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Method Job Toolchain Mean Error Ratio
FromHexString Job-KWOVIE \core_root_base\corerun.exe 19.48 ns 0.208 ns 1.00
FromHexString Job-NCVQGH \core_root_diff\corerun.exe 17.35 ns 0.197 ns 0.89

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt)

EgorBot commented 1 week ago

Benchmark results on windows_milano

BenchmarkDotNet v0.14.0, Windows 10 (10.0.20348.2762)
AMD EPYC 7R13 Processor, 1 CPU, 16 logical and 8 physical cores
  Job-BQHVZU : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX2
  Job-OYOSNM : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX2
Method Job Toolchain Mean Error Ratio
FromHexString Job-BQHVZU \core_root_base\corerun.exe 25.95 ns 0.167 ns 1.00
FromHexString Job-OYOSNM \core_root_diff\corerun.exe 25.87 ns 0.108 ns 1.00

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt)

EgorBo commented 1 week ago

@EgorBot -pr 103654 -azure_cobalt100

using BenchmarkDotNet.Attributes;

public class Perf_Int16
{
    static string _hex = Convert.ToHexString("This is a test of Convert."u8.ToArray());

    [Benchmark]
    public byte[] FromHexString() => Convert.FromHexString(_hex);
}
EgorBot commented 1 week ago

Benchmark results on linux_cobalt100

BenchmarkDotNet v0.14.0, Ubuntu 22.04.5 LTS (Jammy Jellyfish)
cobalt100
  Job-GBPDTF : .NET 10.0.0 (42.42.42.42424), Arm64 RyuJIT AdvSIMD
  Job-XSPGMC : .NET 10.0.0 (42.42.42.42424), Arm64 RyuJIT AdvSIMD
Method Toolchain Mean Error Ratio
FromHexString Main 50.52 ns 0.568 ns 1.00
FromHexString PR 48.22 ns 0.590 ns 0.95

BDN_Artifacts.zip

cc @EgorBo (agent_logs.txt)

EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBot commented 1 week ago

❌ Failed on AwsGenoa: Force delete (batch)

cc @EgorBo ([logs]())

EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBot commented 1 week ago

Memory safety check for 899c6e20:

Before:

]9;4;3;\]9;4;0;\Total methods: 120885, among them:
- P/Invokes: 2596
- With 'unsafe' context: 5540
- With unsafe APIs: 1625
- Total methods with non-safe code: 9761 (8.07%)

After:

]9;4;3;\]9;4;0;\Total methods: 120887, among them:
- P/Invokes: 2596
- With 'unsafe' context: 5540
- With unsafe APIs: 1625
- Total methods with non-safe code: 9761 (8.07%)
EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBot commented 1 week ago

Memory safety report

Before:

Assembly Total methods Are P/Invokes Contain 'unsafe' context Contain Unsafe API calls
All 120885 2596 5540 1625

After:

Assembly Total methods Are P/Invokes Contain 'unsafe' context Contain Unsafe API calls
All 120887 2596 5540 1625
EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBot commented 1 week ago

Code safety report for @EgorBo

Click to expand #### Before: | Assembly | Total
methods | P/Invokes | Contain 'unsafe'
context | Contain Unsafe
API calls | | ---| ---| ---| ---| ---| | System.Private.CoreLib | 21724 | 323 | 2473 | 900 | | Common | 5989 | 2143 | 479 | 110 | | System.Net.HttpListener | 786 | 0 | 320 | 0 | | System.Security.Cryptography | 4823 | 0 | 238 | 51 | | System.Numerics.Tensors | 1598 | 0 | 141 | 126 | | System.Reflection.Metadata | 2612 | 0 | 196 | 3 | | System.Net.Sockets | 1032 | 0 | 184 | 8 | | System.Net.Http | 2261 | 90 | 86 | 16 | | System.Net.Quic | 369 | 2 | 134 | 36 | | System.Runtime.InteropServices.JavaScript | 382 | 0 | 123 | 0 | | *Other* | 79309 | 38 | 1166 | 375 | | **Total** | **120885** | **2596** | **5540** | **1625** | #### After: | Assembly | Total
methods | P/Invokes | Contain 'unsafe'
context | Contain Unsafe
API calls | | ---| ---| ---| ---| ---| | System.Private.CoreLib | 21724 | 323 | 2473 | 900 | | Common | 5989 | 2143 | 479 | 110 | | System.Net.HttpListener | 786 | 0 | 320 | 0 | | System.Security.Cryptography | 4823 | 0 | 238 | 51 | | System.Numerics.Tensors | 1598 | 0 | 141 | 126 | | System.Reflection.Metadata | 2612 | 0 | 196 | 3 | | System.Net.Sockets | 1032 | 0 | 184 | 8 | | System.Net.Http | 2261 | 90 | 86 | 16 | | System.Net.Quic | 369 | 2 | 134 | 36 | | System.Runtime.InteropServices.JavaScript | 382 | 0 | 123 | 0 | | *Other* | 79311 | 38 | 1166 | 375 | | **Total** | **120887** | **2596** | **5540** | **1625** | [Analyzer](https://github.com/EgorBo/UnsafeCodeAnalyzer)

cc @EgorBo

EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBot commented 1 week ago

Memory safety report

Before:

Assembly Total
methods		 P/Invokes Contain 'unsafe'
context		 Contain Unsafe
API calls
System.Private.CoreLib 21724 323 2473 900
Common 5989 2143 479 110
System.Net.HttpListener 786 0 320 0
System.Security.Cryptography 4823 0 238 51
System.Numerics.Tensors 1598 0 141 126
System.Reflection.Metadata 2612 0 196 3
System.Net.Sockets 1032 0 184 8
System.Net.Http 2261 90 86 16
System.Net.Quic 369 2 134 36
System.Runtime.InteropServices.JavaScript 382 0 123 0
Other 79309 38 1166 375
Total 120885 2596 5540 1625

After:

Assembly Total
methods		 P/Invokes Contain 'unsafe'
context		 Contain Unsafe
API calls
System.Private.CoreLib 21724 323 2473 900
Common 5989 2143 479 110
System.Net.HttpListener 786 0 320 0
System.Security.Cryptography 4823 0 238 51
System.Numerics.Tensors 1598 0 141 126
System.Reflection.Metadata 2612 0 196 3
System.Net.Sockets 1032 0 184 8
System.Net.Http 2261 90 86 16
System.Net.Quic 369 2 134 36
System.Runtime.InteropServices.JavaScript 382 0 123 0
Other 79311 38 1166 375
Total 120887 2596 5540 1625
EgorBot commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBo commented 1 week ago

@EgorBot -codesafety -pr 103654

EgorBot commented 1 week ago

Code safety report

Click to expand #### Main: | Assembly | Total
methods | P/Invokes | Contain 'unsafe'
context | Contain Unsafe
API calls | | ---| ---| ---| ---| ---| | System.Private.CoreLib | 21724 | 323 | 2473 | 900 | | Common | 5989 | 2143 | 479 | 110 | | System.Net.HttpListener | 786 | 0 | 320 | 0 | | System.Security.Cryptography | 4823 | 0 | 238 | 51 | | System.Numerics.Tensors | 1598 | 0 | 141 | 126 | | System.Reflection.Metadata | 2612 | 0 | 196 | 3 | | System.Net.Sockets | 1032 | 0 | 184 | 8 | | System.Net.Http | 2261 | 90 | 86 | 16 | | System.Net.Quic | 369 | 2 | 134 | 36 | | System.Runtime.InteropServices.JavaScript | 382 | 0 | 123 | 0 | | *Other* | 79309 | 38 | 1166 | 375 | | **Total** | **120885** | **2596** | **5540** | **1625** | #### PR: | Assembly | Total
methods | P/Invokes | Contain 'unsafe'
context | Contain Unsafe
API calls | | ---| ---| ---| ---| ---| | System.Private.CoreLib | 21724 | 323 | 2473 | 900 | | Common | 5989 | 2143 | 479 | 110 | | System.Net.HttpListener | 786 | 0 | 320 | 0 | | System.Security.Cryptography | 4823 | 0 | 238 | 51 | | System.Numerics.Tensors | 1598 | 0 | 141 | 126 | | System.Reflection.Metadata | 2612 | 0 | 196 | 3 | | System.Net.Sockets | 1032 | 0 | 184 | 8 | | System.Net.Http | 2261 | 90 | 86 | 16 | | System.Net.Quic | 369 | 2 | 134 | 36 | | System.Runtime.InteropServices.JavaScript | 382 | 0 | 123 | 0 | | *Other* | 79311 | 38 | 1166 | 375 | | **Total** | **120887** | **2596** | **5540** | **1625** | [UnsafeCodeAnalyzer](https://github.com/EgorBo/UnsafeCodeAnalyzer)

cc @EgorBo