search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.91k stars 4.63k forks source link

[apple] Mobile failures in `System.Diagnostics.DiagnosticSource` runtime metrics #105202

Closed matouskozak closed 1 month ago

matouskozak commented 1 month ago

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=747439 Build error leg or test failing: System.Diagnostics.DiagnosticSource.Tests Affected CI: [ios][tvos]simulator_x64_release_allsubsets_mono, maccatalyst_[x64/arm64]_release_allsubsests_mono (runtime-extra-platforms) Range of commits: https://github.com/dotnet/runtime/compare/0806470e...7cdd2e90

Stack trace example from MacCatalyst x64:

2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10] 1)  [FAIL] System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateMeasurements   Test name: System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateMeasurements<Int64>(metricName: "dotnet.jit.compiled_il.size", valueAssertion: Func`2 { Method = System.ValueTuple`2[System.Boolean,System.String] <.cctor>b__21_1(Int64), Target = <>c { } }, beforeRecord: null)   Test case: System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateMeasurements
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10]    Assembly:  [System.Diagnostics.DiagnosticSource.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10]    Exception messages: Expected value to be greater than zero. Actual value was: 0.   Exception stack traces:    at System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateSingleMeasurement[Int64](String metricName, Func`2 valueAssertion, Func`1 beforeRecord)
   at System.Object.InvokeStub_RuntimeMetricsTests.ValidateMeasurements(Object , Span`1 )
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10]    at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   Execution time: 0.1050582
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10] 2)  [FAIL] System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateMeasurements   Test name: System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateMeasurements<Int64>(metricName: "dotnet.jit.compiled_methods", valueAssertion: Func`2 { Method = System.ValueTuple`2[System.Boolean,System.String] <.cctor>b__21_1(Int64), Target = <>c { } }, beforeRecord: null)   Test case: System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateMeasurements
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10]    Assembly:  [System.Diagnostics.DiagnosticSource.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10]    Exception messages: Expected value to be greater than zero. Actual value was: 0.   Exception stack traces:    at System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateSingleMeasurement[Int64](String metricName, Func`2 valueAssertion, Func`1 beforeRecord)
   at System.Object.InvokeStub_RuntimeMetricsTests.ValidateMeasurements(Object , Span`1 )
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10]    at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   Execution time: 0.1024496
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10] 3)  [FAIL] System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateMeasurements   Test name: System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateMeasurements<Double>(metricName: "dotnet.jit.compilation.time", valueAssertion: Func`2 { Method = System.ValueTuple`2[System.Boolean,System.String] <.cctor>b__21_3(Double), Target = <>c { } }, beforeRecord: null)   Test case: System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateMeasurements
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10]    Assembly:  [System.Diagnostics.DiagnosticSource.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
2024-07-19 06:05:58.025 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10]    Exception messages: Expected value to be greater than zero. Actual value was: 0.   Exception stack traces:    at System.Diagnostics.Metrics.Tests.RuntimeMetricsTests.ValidateSingleMeasurement[Double](String metricName, Func`2 valueAssertion, Func`1 beforeRecord)
   at System.Object.InvokeStub_RuntimeMetricsTests.ValidateMeasurements(Object , Span`1 )
2024-07-19 06:05:58.026 Df System.Diagnostics.DiagnosticSource.Tests[5417:db10]    at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   Execution time: 0.1059831

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "Expected value to be greater than zero. Actual value was: 0.",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=747439 Error message validated: [Expected value to be greater than zero. Actual value was: 0.] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 7/21/2024 4:35:30 PM UTC

Report

Build Definition Test Pull Request
758343 dotnet/runtime System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution
755747 dotnet/runtime System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution
751577 dotnet/runtime System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution
750924 dotnet/runtime System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution
748618 dotnet/runtime System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution
747439 dotnet/runtime System.Diagnostics.DiagnosticSource.Tests.WorkItemExecution

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 2 6
dotnet-policy-service[bot] commented 1 month ago

Tagging subscribers to 'os-ios': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger See info in area-owners.md if you want to be subscribed.

dotnet-policy-service[bot] commented 1 month ago

Tagging subscribers to 'os-tvos': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger See info in area-owners.md if you want to be subscribed.

dotnet-policy-service[bot] commented 1 month ago

Tagging subscribers to 'os-maccatalyst': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger See info in area-owners.md if you want to be subscribed.

matouskozak commented 1 month ago

Possibly related to the new tests introduced in https://github.com/dotnet/runtime/pull/104680. Do you know if these tests should be disabled or same changes are needed for them to work on Apple mobile targets? @stevejgordon @tarekgh ?

stevejgordon commented 1 month ago

@matouskozak I think we can safely skip this platform in the runtime metrics tests.

matouskozak commented 1 month ago

@matouskozak I think we can safely skip this platform in the runtime metrics tests.

Thank you, would you like to open a PR to disable them or should I take a look?

Do you know if this is an issue that should be fixed on the Mono runtime side or is it by design of the test? It looks like only a few tests are failing so I suppose the RuntimeMetricsTests suit isn't CoreCLR specific.

stevejgordon commented 1 month ago

@matouskozak I've opened a PR and took a different approach after reviewing the failures more closely.