dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15k stars 4.67k forks source link

[perf] System.Tests.Perf.** and System.Perf.** performance regressions in Mono AOT x64 microbenchmarks from Preview 5 to Preview 7 #89243

Closed kotlarmilos closed 1 month ago

kotlarmilos commented 1 year ago

Description

The following performance regressions occurred between the Preview 4 snapshot (May 2nd, 2023) and Preview 7 snapshot (July 19th, 2023).

The configuration used for the benchmarks is Mono AOT with LLVM on x64 ubuntu 18.04.

Name Baseline Value Compare Value % Difference
System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: https://contoso.com/path/with?key=value#fragment) 888.36 992.32 11.7
System.Tests.Perf_Uri.Ctor(input: https://contoso.com) 286.02 347.22 21.4
System.Tests.Perf_UInt32.ParseSpan(value: 12345) 40.52 42.6 5.13
System.Tests.Perf_UInt16.TryParse(value: 0) 29.02 32.44 11.78
System.Tests.Perf_String.Substring_Int(s: dzsdzsDDZSDZSDZSddsz, i: 0) 2.22 2.39 7.66
System.Tests.Perf_String.Split(s: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z, arr: [' '], options: None) 771.45 1034.74 34.13
System.Tests.Perf_String.Replace_String(text: This is a very nice sentence. This is another very nice sentence., oldValue: a, newValue: ) 208.84 315.4 51.02
System.Tests.Perf_String.Replace_Char(text: This is a very nice sentence, oldChar: 'z', newChar: 'y') 28.82 47.39 64.43
System.Tests.Perf_String.IndexOfAny 115.42 136.59 18.34
System.Tests.Perf_String.CtorCharCount(size: 10) 25.82 28.31 9.64
System.Tests.Perf_String.CtorCharCount(size: 1) 21.53 23.2 7.76
System.Tests.Perf_Single.Parse(value: 12345) 112.2 175.82 56.7
System.Tests.Perf_SByte.Parse(value: -128) 41.35 45.24 9.41
System.Tests.Perf_Random.NextSingle 13.88 14.61 5.26
System.Tests.Perf_Int64.TryParseSpan(value: 12345) 41.94 46.03 9.75
System.Tests.Perf_Int32.TryParse(value: 2147483647) 55.94 65.38 16.88
System.Tests.Perf_Int32.TryParse(value: 12345) 41.25 47.36 14.81
System.Tests.Perf_Int32.Parse(value: 2147483647) 58.57 67.48 15.21
System.Tests.Perf_Int16.TryParse(value: 32767) 46.11 55.15 19.61
System.Tests.Perf_Environment.SetEnvironmentVariable 149.69 204.17 36.4
System.Tests.Perf_Enum.HasFlag 0.08 0.28 250
System.Tests.Perf_Enum.Compare 2.97 3.47 16.84
System.Tests.Perf_Double.Parse(value: 1.7976931348623157e+308) 221.62 274.76 23.98
System.Tests.Perf_Double.Parse(value: -1.7976931348623157e+308) 224.55 283.09 26.07
System.Tests.Perf_Char.Char_ToLower(c: '你', cultureName: zh-Hans) 85.97 91.74 6.71
System.Tests.Perf_Boolean.ToString(value: True) 0.21 0.35 66.67
System.Perf_Convert.ToBase64CharArray(binaryDataSize: 1024, formattingOptions: None) 1836.06 7554.54 311.45
kotlarmilos commented 1 year ago

@vargaz @stephentoub Do you have any changes in mind between May 2nd, 2023 and July 19th, 2023 that may have caused the regressions?

stephentoub commented 1 year ago

Purely given all the calls involving "Parse", I'd speculate #86875

kotlarmilos commented 1 year ago

Thanks! Let's check it out :)