search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.88k stars 4.63k forks source link

[wasm] 2 runtime tests failing with emscripten 3.1.34 #84786

Open radekdoulik opened 1 year ago

radekdoulik commented 1 year ago

These failures might be related to change in handling of ExitStatus exception in emscripten.

Failing tests

=== TEST EXECUTION SUMMARY ===
   GC.API.XUnitWrapper                                 Total: 17, Errors: 0, Failed: 1, Skipped: 0, Time:  49.980s
   tracing.eventcounter.XUnitWrapper                   Total:  2, Errors: 0, Failed: 1, Skipped: 0, Time: 600.040s
                                                              --          -          -           -        --------
                                                 GRAND TOTAL: 97          0          2           0        798.676s (799.073s)

Logs

  Starting:    GC.API.XUnitWrapper (parallel test collections = on, max threads = 2)
    GC/API/GCHandleCollector/Usage/Usage.sh [FAIL]
      {"name":"ExitStatus","message":"Program terminated with exit(100)","status":100}

      Unhandled Exception:
      System.InvalidOperationException: Handle collector count overflows or underflows.
         at System.Runtime.InteropServices.HandleCollector.Remove()
         at HandleCollectorTest.Finalize()

      Unhandled Exception:
      System.InvalidOperationException: Handle collector count overflows or underflows.
         at System.Runtime.InteropServices.HandleCollector.Remove()
         at HandleCollectorTest.Finalize()
      [ERROR] FATAL UNHANDLED EXCEPTION: System.InvalidOperationException: Handle collector count overflows or underflows.
         at System.Runtime.InteropServices.HandleCollector.Remove()
         at HandleCollectorTest.Finalize()

      Return code:      1
      Raw output file:      /root/helix/work/workitem/uploads/Reports/GC.API/GCHandleCollector/Usage/Usage.output.txt
      Raw output:
      BEGIN EXECUTION
      MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET
        AppDir: /root/helix/work/workitem/e/GC/API/GCHandleCollector/Usage/WasmApp/
        TestBinDir: /root/helix/work/workitem/e/GC/API/GCHandleCollector/Usage
        ArtifactsBinDir: 
        Generated app bundle at /root/helix/work/workitem/e/GC/API/GCHandleCollector/Usage/WasmApp/
      Incoming arguments: --run Usage.dll
      Application arguments: --run Usage.dll
      console.info: Initializing dotnet version 8.0.0-ci commit hash 09041216c7a8afd4c5fc26abad1d77bf6c35ea22
      Case 1 Passed!
      100, 100
      Case 2 Passed!
      Case 3 Passed!

      Test Passed!
      WASM EXIT 100
   tracing.eventcounter.XUnitWrapper: [Long Running Test] 'tracing/eventcounter/regression-25709/regression-25709.sh', Elapsed: 00:08:12
    tracing/eventcounter/regression-25709/regression-25709.sh [FAIL]
      {"name":"ExitStatus","message":"Program terminated with exit(100)","status":100}

      cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM)

      Return code:      -100
      Raw output file:      /root/helix/work/workitem/uploads/Reports/tracing.eventcounter/regression-25709/regression-25709.output.txt
      Raw output:
      BEGIN EXECUTION
      MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET
        AppDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/
        TestBinDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709
        ArtifactsBinDir: 
        Generated app bundle at /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/
      Incoming arguments: --run regression-25709.dll
      Application arguments: --run regression-25709.dll
      console.info: Initializing dotnet version 8.0.0-ci commit hash 09041216c7a8afd4c5fc26abad1d77bf6c35ea22
      Test passed
      WASM EXIT 100

      cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM)
      Test Harness Exitcode is : -100
      To run the test:
      > set CORE_ROOT=/root/helix/work/correlation
      > /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh
      Expected: True
      Actual:   False
      Stack Trace:
           at tracing_eventcounter._regression_25709_regression_25709_._regression_25709_regression_25709_sh()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
      Output:
        {"name":"ExitStatus","message":"Program terminated with exit(100)","status":100}

        cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM)

        Return code:      -100
        Raw output file:      /root/helix/work/workitem/uploads/Reports/tracing.eventcounter/regression-25709/regression-25709.output.txt
        Raw output:
        BEGIN EXECUTION
        MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET
          AppDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/
          TestBinDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709
          ArtifactsBinDir: 
          Generated app bundle at /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/
        Incoming arguments: --run regression-25709.dll
        Application arguments: --run regression-25709.dll
        console.info: Initializing dotnet version 8.0.0-ci commit hash 09041216c7a8afd4c5fc26abad1d77bf6c35ea22
        Test passed
        WASM EXIT 100

        cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM)
        Test Harness Exitcode is : -100
        To run the test:
        > set CORE_ROOT=/root/helix/work/correlation
        > /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh
ghost commented 1 year ago

Tagging subscribers to this area: @directhex See info in area-owners.md if you want to be subscribed.

Issue Details
These failures might be related to change in handling of `ExitStatus` exception in emscripten. Failing tests ``` === TEST EXECUTION SUMMARY === GC.API.XUnitWrapper Total: 17, Errors: 0, Failed: 1, Skipped: 0, Time: 49.980s tracing.eventcounter.XUnitWrapper Total: 2, Errors: 0, Failed: 1, Skipped: 0, Time: 600.040s -- - - - -------- GRAND TOTAL: 97 0 2 0 798.676s (799.073s) ``` Logs ``` Starting: GC.API.XUnitWrapper (parallel test collections = on, max threads = 2) GC/API/GCHandleCollector/Usage/Usage.sh [FAIL] {"name":"ExitStatus","message":"Program terminated with exit(100)","status":100} Unhandled Exception: System.InvalidOperationException: Handle collector count overflows or underflows. at System.Runtime.InteropServices.HandleCollector.Remove() at HandleCollectorTest.Finalize() Unhandled Exception: System.InvalidOperationException: Handle collector count overflows or underflows. at System.Runtime.InteropServices.HandleCollector.Remove() at HandleCollectorTest.Finalize() [ERROR] FATAL UNHANDLED EXCEPTION: System.InvalidOperationException: Handle collector count overflows or underflows. at System.Runtime.InteropServices.HandleCollector.Remove() at HandleCollectorTest.Finalize() Return code: 1 Raw output file: /root/helix/work/workitem/uploads/Reports/GC.API/GCHandleCollector/Usage/Usage.output.txt Raw output: BEGIN EXECUTION MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET AppDir: /root/helix/work/workitem/e/GC/API/GCHandleCollector/Usage/WasmApp/ TestBinDir: /root/helix/work/workitem/e/GC/API/GCHandleCollector/Usage ArtifactsBinDir: Generated app bundle at /root/helix/work/workitem/e/GC/API/GCHandleCollector/Usage/WasmApp/ Incoming arguments: --run Usage.dll Application arguments: --run Usage.dll console.info: Initializing dotnet version 8.0.0-ci commit hash 09041216c7a8afd4c5fc26abad1d77bf6c35ea22 Case 1 Passed! 100, 100 Case 2 Passed! Case 3 Passed! Test Passed! WASM EXIT 100 ``` ``` tracing.eventcounter.XUnitWrapper: [Long Running Test] 'tracing/eventcounter/regression-25709/regression-25709.sh', Elapsed: 00:08:12 tracing/eventcounter/regression-25709/regression-25709.sh [FAIL] {"name":"ExitStatus","message":"Program terminated with exit(100)","status":100} cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM) Return code: -100 Raw output file: /root/helix/work/workitem/uploads/Reports/tracing.eventcounter/regression-25709/regression-25709.output.txt Raw output: BEGIN EXECUTION MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET AppDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/ TestBinDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709 ArtifactsBinDir: Generated app bundle at /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/ Incoming arguments: --run regression-25709.dll Application arguments: --run regression-25709.dll console.info: Initializing dotnet version 8.0.0-ci commit hash 09041216c7a8afd4c5fc26abad1d77bf6c35ea22 Test passed WASM EXIT 100 cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM) Test Harness Exitcode is : -100 To run the test: > set CORE_ROOT=/root/helix/work/correlation > /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Expected: True Actual: False Stack Trace: at tracing_eventcounter._regression_25709_regression_25709_._regression_25709_regression_25709_sh() at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) Output: {"name":"ExitStatus","message":"Program terminated with exit(100)","status":100} cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM) Return code: -100 Raw output file: /root/helix/work/workitem/uploads/Reports/tracing.eventcounter/regression-25709/regression-25709.output.txt Raw output: BEGIN EXECUTION MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET AppDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/ TestBinDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709 ArtifactsBinDir: Generated app bundle at /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/ Incoming arguments: --run regression-25709.dll Application arguments: --run regression-25709.dll console.info: Initializing dotnet version 8.0.0-ci commit hash 09041216c7a8afd4c5fc26abad1d77bf6c35ea22 Test passed WASM EXIT 100 cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM) Test Harness Exitcode is : -100 To run the test: > set CORE_ROOT=/root/helix/work/correlation > /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh ```
Author: radekdoulik
Assignees: -
Labels: `untriaged`, `area-Infrastructure-mono`, `runtime-mono`, `needs-area-label`
Milestone: 8.0.0
carlossanlop commented 1 year ago

@radekdoulik the first error message is already being tracked with https://github.com/dotnet/runtime/issues/81383 : Handle collector count overflows or underflows.. We are already tracking the hits of that one with Known Build Error.

For the second error message, can you please make sure this issue is linked with Known Build Error? The instructions can be found in our template: https://github.com/dotnet/runtime/issues/new?assignees=&labels=blocking-clean-ci%2CKnown+Build+Error&template=04_ci_known_issue.yml

radekdoulik commented 1 year ago

@radekdoulik the first error message is already being tracked with #81383 : Handle collector count overflows or underflows.. We are already tracking the hits of that one with Known Build Error.

For the second error message, can you please make sure this issue is linked with Known Build Error? The instructions can be found in our template: https://github.com/dotnet/runtime/issues/new?assignees=&labels=blocking-clean-ci%2CKnown+Build+Error&template=04_ci_known_issue.yml

These are now disabled on wasm, so they shouldn't fail on CI.

Looking at the logs, it also seems different than #81383, because in our case the tests passes, but xharness evaluates it as failed.

ghost commented 1 year ago

Tagging subscribers to 'arch-wasm': @lewing See info in area-owners.md if you want to be subscribed.

Issue Details
These failures might be related to change in handling of `ExitStatus` exception in emscripten. Failing tests ``` === TEST EXECUTION SUMMARY === GC.API.XUnitWrapper Total: 17, Errors: 0, Failed: 1, Skipped: 0, Time: 49.980s tracing.eventcounter.XUnitWrapper Total: 2, Errors: 0, Failed: 1, Skipped: 0, Time: 600.040s -- - - - -------- GRAND TOTAL: 97 0 2 0 798.676s (799.073s) ``` Logs ``` Starting: GC.API.XUnitWrapper (parallel test collections = on, max threads = 2) GC/API/GCHandleCollector/Usage/Usage.sh [FAIL] {"name":"ExitStatus","message":"Program terminated with exit(100)","status":100} Unhandled Exception: System.InvalidOperationException: Handle collector count overflows or underflows. at System.Runtime.InteropServices.HandleCollector.Remove() at HandleCollectorTest.Finalize() Unhandled Exception: System.InvalidOperationException: Handle collector count overflows or underflows. at System.Runtime.InteropServices.HandleCollector.Remove() at HandleCollectorTest.Finalize() [ERROR] FATAL UNHANDLED EXCEPTION: System.InvalidOperationException: Handle collector count overflows or underflows. at System.Runtime.InteropServices.HandleCollector.Remove() at HandleCollectorTest.Finalize() Return code: 1 Raw output file: /root/helix/work/workitem/uploads/Reports/GC.API/GCHandleCollector/Usage/Usage.output.txt Raw output: BEGIN EXECUTION MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET AppDir: /root/helix/work/workitem/e/GC/API/GCHandleCollector/Usage/WasmApp/ TestBinDir: /root/helix/work/workitem/e/GC/API/GCHandleCollector/Usage ArtifactsBinDir: Generated app bundle at /root/helix/work/workitem/e/GC/API/GCHandleCollector/Usage/WasmApp/ Incoming arguments: --run Usage.dll Application arguments: --run Usage.dll console.info: Initializing dotnet version 8.0.0-ci commit hash 09041216c7a8afd4c5fc26abad1d77bf6c35ea22 Case 1 Passed! 100, 100 Case 2 Passed! Case 3 Passed! Test Passed! WASM EXIT 100 ``` ``` tracing.eventcounter.XUnitWrapper: [Long Running Test] 'tracing/eventcounter/regression-25709/regression-25709.sh', Elapsed: 00:08:12 tracing/eventcounter/regression-25709/regression-25709.sh [FAIL] {"name":"ExitStatus","message":"Program terminated with exit(100)","status":100} cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM) Return code: -100 Raw output file: /root/helix/work/workitem/uploads/Reports/tracing.eventcounter/regression-25709/regression-25709.output.txt Raw output: BEGIN EXECUTION MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET AppDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/ TestBinDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709 ArtifactsBinDir: Generated app bundle at /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/ Incoming arguments: --run regression-25709.dll Application arguments: --run regression-25709.dll console.info: Initializing dotnet version 8.0.0-ci commit hash 09041216c7a8afd4c5fc26abad1d77bf6c35ea22 Test passed WASM EXIT 100 cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM) Test Harness Exitcode is : -100 To run the test: > set CORE_ROOT=/root/helix/work/correlation > /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Expected: True Actual: False Stack Trace: at tracing_eventcounter._regression_25709_regression_25709_._regression_25709_regression_25709_sh() at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) Output: {"name":"ExitStatus","message":"Program terminated with exit(100)","status":100} cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM) Return code: -100 Raw output file: /root/helix/work/workitem/uploads/Reports/tracing.eventcounter/regression-25709/regression-25709.output.txt Raw output: BEGIN EXECUTION MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET AppDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/ TestBinDir: /root/helix/work/workitem/e/tracing/eventcounter/regression-25709 ArtifactsBinDir: Generated app bundle at /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/WasmApp/ Incoming arguments: --run regression-25709.dll Application arguments: --run regression-25709.dll console.info: Initializing dotnet version 8.0.0-ci commit hash 09041216c7a8afd4c5fc26abad1d77bf6c35ea22 Test passed WASM EXIT 100 cmdLine:/root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 4/13/2023 10:40:59 AM, end: 4/13/2023 10:50:59 AM) Test Harness Exitcode is : -100 To run the test: > set CORE_ROOT=/root/helix/work/correlation > /root/helix/work/workitem/e/tracing/eventcounter/regression-25709/regression-25709.sh ```
Author: radekdoulik
Assignees: -
Labels: `arch-wasm`, `area-Infrastructure-mono`, `runtime-mono`
Milestone: 8.0.0
lewing commented 1 year ago

@radekdoulik please update the status of this issue

ilonatommy commented 1 month ago

@radekdoulik, should https://github.com/dotnet/runtime/pull/100334 try to enable the tests? Or a separate PR when the bump is merged.