search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.98k stars 4.66k forks source link

Test Failed: Assert failure: Consistency check failed: FAILED: !implSlot.IsNull() && \"Valid method implementation was not found.\" #13188

Closed VincentBu closed 3 years ago

VincentBu commented 5 years ago

Job: https://mc.dot.net/#/user/coreclr-jitstress/ci~2Fdotnet~2Fcoreclr~2Frefs~2Fheads~2Fmaster/test~2Ffunctional~2Fcli~2F/20190730.1/workItem/Interop/analysis/xunit/Interop_ICustomMarshaler._Primitives_ICustomMarshaler_ICustomMarshaler_~2F_Primitives_ICustomMarshaler_ICustomMarshaler_cmd

Failed tests: Interop_ICustomMarshaler._Primitives_ICustomMarshalerICustomMarshaler._Primitives_ICustomMarshaler_ICustomMarshaler_cmd

Log:

Assert failure(PID 6012 [0x0000177c], Thread: 4212 [0x1074]): Consistency check failed: FAILED: !implSlot.IsNull() && \"Valid method implementation was not found.\"

CORECLR! coreclr_shutdown_2 + 0xEB34 (0x00007ffb`486ddae4)
CORECLR! GetCLRRuntimeHost + 0x2E5E45 (0x00007ffb`48a25fd5)
CORECLR! GetCLRRuntimeHost + 0x2E3D79 (0x00007ffb`48a23f09)
CORECLR! GetCLRRuntimeHost + 0x2E8483 (0x00007ffb`48a28613)
CORECLR! GetCLRRuntimeHost + 0x68CA0D (0x00007ffb`48dccb9d)
<no module>! <no symbol> + 0x0 (0x00007ffa`e9bed883)
<no module>! <no symbol> + 0x0 (0x000001ab`5be36ac8)
<no module>! <no symbol> + 0x0 (0x000001ab`5bf81398)
<no module>! <no symbol> + 0x0 (0x000001ab`5be752c8)
CORECLR! GetCLRRuntimeHost + 0xAE7298 (0x00007ffb`49227428)
    File: f:\\workspace.2\\_work\\1\\s\\src\\vm\\virtualcallstub.cpp Line: 2382
    Image: C:\\dotnetbuild\\work\\5b525dc6-89e4-42e9-b409-ec6d0f1a5412\\Payload\\CoreRun.exe

Return code:      1
Raw output file:      C:\\dotnetbuild\\work\\5b525dc6-89e4-42e9-b409-ec6d0f1a5412\\Work\\d7258f20-7849-4c2f-97fc-cd98b82e0046\\Exec\\Interop\\ICustomMarshaler\\Reports\\Interop.ICustomMarshaler\\Primitives\\ICustomMarshaler\\ICustomMarshaler.output.txt
Raw output:
BEGIN EXECUTION
 \"C:\\dotnetbuild\\work\\5b525dc6-89e4-42e9-b409-ec6d0f1a5412\\Payload\\corerun.exe\" ICustomMarshaler.exe 
Discovering...
Running 25 of 25 tests...
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=C:\\dotnetbuild\\work\\5b525dc6-89e4-42e9-b409-ec6d0f1a5412\\Payload
> C:\\dotnetbuild\\work\\5b525dc6-89e4-42e9-b409-ec6d0f1a5412\\Work\\d7258f20-7849-4c2f-97fc-cd98b82e0046\\Exec\\Interop\\ICustomMarshaler\\Primitives\\ICustomMarshaler\\ICustomMarshaler.cmd
Expected: True
Actual:   False

category:testing theme:tail-call skill-level:intermediate cost:medium

jeffschwMSFT commented 5 years ago

cc @jkoritzinsky

jeffschwMSFT commented 5 years ago

Running with TailCallStress=1, TieredCompliation=0 and I am able to easily see this failure.

Handing this one over to the JIT team as the frame in question that is failing in this test is within xunit.

I am not familiar with vsd but it looks like there is type confusion with the frame information.

In the failing case, the frame is associated with this type:

2:025> !dumpmt 0x7fff7fc3b700 EEClass: 00007fff7fc4dd90 Module: 00007fff7fc07068 Name: Xunit.Sdk.TestFrameworkProxy+MessageSinkWrapper mdToken: 00000000020000d5 File: ...\Windows_NT.x64.Debug\Tests\Core_Root\xunit.execution.dotnet.dll BaseSize: 0x18 ComponentSize: 0x0 Slots in VTable: 6 Number of IFaces in IFaceMap: 1

But in a working case, the frame is actually associated with this type: 2:026> !dumpmt 0x00007fff`7fe6ef08 EEClass: 00007fff7fe78ee8 Module: 00007fff7fc06df8 Name: Xunit.Sdk.SynchronousMessageBus mdToken: 000000000200004f File: ...\Windows_NT.x64.Debug\Tests\Core_Root\xunit.execution.dotnet.dll BaseSize: 0x18 ComponentSize: 0x0 Slots in VTable: 7 Number of IFaces in IFaceMap: 2

The failing case does not successfully find the dispatchslot for the token at https://github.com/dotnet/coreclr/blob/master/src/vm/virtualcallstub.cpp#L2204

VincentBu commented 5 years ago

Job: coreclr-jitstress

Detail: last failure: https://dev.azure.com/dnceng/public/_build/results?buildId=334520 console log: https://helix.dot.net/api/2019-06-17/jobs/89588bc3-189d-4fff-9809-999c0f650d46/workitems/Interop/console

OS & Arch Windows x64

Mode: set COMPlus_TieredCompilation=0 set COMPlus_TailcallStress=1

Note: This failure repro on our windows 10 x64 machine again

Log:

Interop\ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.cmd [FAIL]

      Assert failure(PID 6804 [0x00001a94], Thread: 8188 [0x1ffc]): Consistency check failed: FAILED: !implSlot.IsNull() && "Valid method implementation was not found."

      CORECLR! coreclr_shutdown_2 + 0xEB34 (0x00007ffe`0635dae4)
      CORECLR! GetCLRRuntimeHost + 0x2E5ED5 (0x00007ffe`066a5da5)
      CORECLR! GetCLRRuntimeHost + 0x2E3E09 (0x00007ffe`066a3cd9)
      CORECLR! GetCLRRuntimeHost + 0x2E8513 (0x00007ffe`066a83e3)
      CORECLR! GetCLRRuntimeHost + 0x68F34D (0x00007ffe`06a4f21d)
      <no module>! <no symbol> + 0x0 (0x00007ffd`a787d463)
      <no module>! <no symbol> + 0x0 (0x0000013a`15e56ac8)
      <no module>! <no symbol> + 0x0 (0x0000013a`15fa17f8)
      <no module>! <no symbol> + 0x0 (0x0000013a`15e95290)
      CORECLR! GetCLRRuntimeHost + 0xAE5348 (0x00007ffe`06ea5218)
          File: f:\workspace\_work\1\s\src\vm\virtualcallstub.cpp Line: 2382
          Image: C:\dotnetbuild\work\89588bc3-189d-4fff-9809-999c0f650d46\Payload\CoreRun.exe

      Return code:      1
      Raw output file:      C:\dotnetbuild\work\89588bc3-189d-4fff-9809-999c0f650d46\Work\58257ecc-185c-4682-8c17-be76cac0954e\Exec\Interop\ICustomMarshaler\Reports\Interop.ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.output.txt
      Raw output:
      BEGIN EXECUTION
       "C:\dotnetbuild\work\89588bc3-189d-4fff-9809-999c0f650d46\Payload\corerun.exe" ICustomMarshaler.dll 
      Discovering...
      Running 25 of 25 tests...
      Expected: 100
      Actual: -1073740286
      END EXECUTION - FAILED
      FAILED
      Test Harness Exitcode is : 1
      To run the test:
      > set CORE_ROOT=C:\dotnetbuild\work\89588bc3-189d-4fff-9809-999c0f650d46\Payload
      > C:\dotnetbuild\work\89588bc3-189d-4fff-9809-999c0f650d46\Work\58257ecc-185c-4682-8c17-be76cac0954e\Exec\Interop\ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.cmd
      Expected: True
      Actual:   False
      Stack Trace:
        F:\workspace\_work\1\s\bin\tests\Windows_NT.x64.Checked\TestWrappers\Interop.ICustomMarshaler\Interop.ICustomMarshaler.XUnitWrapper.cs(334,0): at Interop_ICustomMarshaler._Primitives_ICustomMarshaler_ICustomMarshaler_._Primitives_ICustomMarshaler_ICustomMarshaler_cmd()
      Output:

        Assert failure(PID 6804 [0x00001a94], Thread: 8188 [0x1ffc]): Consistency check failed: FAILED: !implSlot.IsNull() && "Valid method implementation was not found."

        CORECLR! coreclr_shutdown_2 + 0xEB34 (0x00007ffe`0635dae4)
        CORECLR! GetCLRRuntimeHost + 0x2E5ED5 (0x00007ffe`066a5da5)
        CORECLR! GetCLRRuntimeHost + 0x2E3E09 (0x00007ffe`066a3cd9)
        CORECLR! GetCLRRuntimeHost + 0x2E8513 (0x00007ffe`066a83e3)
        CORECLR! GetCLRRuntimeHost + 0x68F34D (0x00007ffe`06a4f21d)
        <no module>! <no symbol> + 0x0 (0x00007ffd`a787d463)
        <no module>! <no symbol> + 0x0 (0x0000013a`15e56ac8)
        <no module>! <no symbol> + 0x0 (0x0000013a`15fa17f8)
        <no module>! <no symbol> + 0x0 (0x0000013a`15e95290)
        CORECLR! GetCLRRuntimeHost + 0xAE5348 (0x00007ffe`06ea5218)
            File: f:\workspace\_work\1\s\src\vm\virtualcallstub.cpp Line: 2382
            Image: C:\dotnetbuild\work\89588bc3-189d-4fff-9809-999c0f650d46\Payload\CoreRun.exe

        Return code:      1
        Raw output file:      C:\dotnetbuild\work\89588bc3-189d-4fff-9809-999c0f650d46\Work\58257ecc-185c-4682-8c17-be76cac0954e\Exec\Interop\ICustomMarshaler\Reports\Interop.ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.output.txt
        Raw output:
        BEGIN EXECUTION
         "C:\dotnetbuild\work\89588bc3-189d-4fff-9809-999c0f650d46\Payload\corerun.exe" ICustomMarshaler.dll 
        Discovering...
        Running 25 of 25 tests...
        Expected: 100
        Actual: -1073740286
        END EXECUTION - FAILED
        FAILED
        Test Harness Exitcode is : 1
BruceForstall commented 5 years ago

@jashook @jakobbotsch Another TailcallStress issue. Should we just close it as a dup of https://github.com/dotnet/coreclr/issues/25961?

VincentBu commented 5 years ago

Job: coreclr-jitstress #20190921.1 Run Test Pri1 Windows_NT x64 checked

Detail: https://helix.dot.net/api/2019-06-17/jobs/a9128218-717e-4097-a87a-f3ad9e76a167/workitems/Interop/console

OS & Arch: windows x64

Mode: COMPlus_TieredCompilation=0 COMPlus_TailcallStress=1

Note: This failure repros on our windows x64 machine

Log:

Interop\ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.cmd [FAIL]

      Assert failure(PID 7104 [0x00001bc0], Thread: 7016 [0x1b68]): Consistency check failed: FAILED: !implSlot.IsNull() && "Valid method implementation was not found."

      CORECLR! CHECK::Trigger + 0x2C4 (0x00007ffc`f69adae4)
      CORECLR! VirtualCallStubManager::Resolver + 0x14BD (0x00007ffc`f6cf5995)
      CORECLR! VirtualCallStubManager::ResolveWorker + 0xB85 (0x00007ffc`f6cf38c9)
      CORECLR! VSD_ResolveWorker + 0xA83 (0x00007ffc`f6cf7fd3)
      CORECLR! ResolveWorkerAsmStub + 0x6D (0x00007ffc`f709ed1d)
      <no module>! <no symbol> + 0x0 (0x00007ffc`97f2cb43)
      <no module>! <no symbol> + 0x0 (0x00000205`04bb6b58)
      <no module>! <no symbol> + 0x0 (0x00000205`04cfd300)
      <no module>! <no symbol> + 0x0 (0x00000205`04bf1440)
      CORECLR! `string' + 0x0 (0x00007ffc`f74f52e8)
          File: f:\workspace.3\_work\1\s\src\vm\virtualcallstub.cpp Line: 2382
          Image: C:\dotnetbuild\work\a9128218-717e-4097-a87a-f3ad9e76a167\Payload\CoreRun.exe

      Return code:      1
      Raw output file:      C:\dotnetbuild\work\a9128218-717e-4097-a87a-f3ad9e76a167\Work\ecf6fd36-b2e7-4601-8f50-d5804072d301\Exec\Interop\ICustomMarshaler\Reports\Interop.ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.output.txt
      Raw output:
      BEGIN EXECUTION
       "C:\dotnetbuild\work\a9128218-717e-4097-a87a-f3ad9e76a167\Payload\corerun.exe" ICustomMarshaler.dll 
      Discovering...
      Running 25 of 25 tests...
      Expected: 100
      Actual: -1073740286
      END EXECUTION - FAILED
      FAILED
      Test Harness Exitcode is : 1
jakobbotsch commented 5 years ago

@BruceForstall the stack trace from the newest occurrence makes me believe this is a duplicate of dotnet/coreclr#26311. More details over in dotnet/coreclr#26818.

jeffschwMSFT commented 5 years ago

@jkoritzinsky can you help find the dump associated with this assert failure? I would expect there to be one. Thanks

jkoritzinsky commented 5 years ago

The dump is at https://helix.dot.net/api/2019-06-17/jobs/a9128218-717e-4097-a87a-f3ad9e76a167/workitems/Interop/files/CoreRun.exe.7104.dmp

You can find the instructions for how to find the dumps in dotnet/arcade at https://github.com/dotnet/arcade/blob/master/Documentation/Dumps/Dumps.md

VincentBu commented 4 years ago

failed again Job: coreclr-jitstress #20191006.1 Run Test Pri1 Windows_NT x64 checked

Detail: https://helix.dot.net/api/2019-06-17/jobs/50010b2a-6157-48bb-9f13-7ec3103a4d51/workitems/Interop/console

OS & Arch: windows x64

Mode: COMPlus_TieredCompilation=0 COMPlus_TailcallStress=1

Note: This failure repros on our windows x64 machine

Log:

Interop\ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.cmd [FAIL]

      Assert failure(PID 3116 [0x00000c2c], Thread: 8936 [0x22e8]): Consistency check failed: FAILED: !implSlot.IsNull() && "Valid method implementation was not found."

      CORECLR! CHECK::Trigger + 0x2C4 (0x00007ff8`ce87dca4)
      CORECLR! VirtualCallStubManager::Resolver + 0x14B4 (0x00007ff8`cebc5bec)
      CORECLR! VirtualCallStubManager::ResolveWorker + 0xB85 (0x00007ff8`cebc3b29)
      CORECLR! VSD_ResolveWorker + 0xA86 (0x00007ff8`cebc8236)
      CORECLR! ResolveWorkerAsmStub + 0x6D (0x00007ff8`cef6c3ad)
      <no module>! <no symbol> + 0x0 (0x00007ff8`6fdddf73)
      <no module>! <no symbol> + 0x0 (0x00000188`8002aeb0)
      <no module>! <no symbol> + 0x0 (0x00000188`801712c0)
      <no module>! <no symbol> + 0x0 (0x00000188`80067c60)
      CORECLR! `string' + 0x0 (0x00007ff8`cf3c52a8)
          File: f:\workspace.13\_work\1\s\src\vm\virtualcallstub.cpp Line: 2382
          Image: C:\dotnetbuild\work\50010b2a-6157-48bb-9f13-7ec3103a4d51\Payload\CoreRun.exe

      Return code:      1
      Raw output file:      C:\dotnetbuild\work\50010b2a-6157-48bb-9f13-7ec3103a4d51\Work\e83e949f-8af7-4d11-a653-5d6ea283ab7f\Exec\Interop\ICustomMarshaler\Reports\Interop.ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.output.txt
      Raw output:
      BEGIN EXECUTION
       "C:\dotnetbuild\work\50010b2a-6157-48bb-9f13-7ec3103a4d51\Payload\corerun.exe" ICustomMarshaler.dll 
      Discovering...
      Running 25 of 25 tests...
      Expected: 100
      Actual: -1073740286
      END EXECUTION - FAILED
      FAILED
      Test Harness Exitcode is : 1
VincentBu commented 4 years ago

Job: coreclr-jitstress #20191014.2 Run Test Pri1 Windows_NT x64 checked

Detail: https://helix.dot.net/api/2019-06-17/jobs/7c1786e6-2823-4b87-b0ba-11326d41c708/workitems/Interop/console

OS & Arch: windows x64

Mode: COMPlus_TieredCompilation=0 COMPlus_TailcallStress=1

Note: This failure repros on our windows x64 machine

Log:

Interop\ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.cmd [FAIL]

      Assert failure(PID 6308 [0x000018a4], Thread: 7656 [0x1de8]): Consistency check failed: FAILED: !implSlot.IsNull() && "Valid method implementation was not found."

      CORECLR! CHECK::Trigger + 0x2C4 (0x00007fff`049bdca4)
      CORECLR! VirtualCallStubManager::Resolver + 0x14BC (0x00007fff`04d059f4)
      CORECLR! VirtualCallStubManager::ResolveWorker + 0xB85 (0x00007fff`04d03929)
      CORECLR! VSD_ResolveWorker + 0xA8E (0x00007fff`04d0803e)
      CORECLR! ResolveWorkerAsmStub + 0x6D (0x00007fff`050ac70d)
      <no module>! <no symbol> + 0x0 (0x00007ffe`a5efdf33)
      <no module>! <no symbol> + 0x0 (0x00000196`80028db0)
      <no module>! <no symbol> + 0x0 (0x00000196`801712c0)
      <no module>! <no symbol> + 0x0 (0x00000196`80067c60)
      CORECLR! `string' + 0x0 (0x00007fff`05504198)
          File: f:\workspace\_work\1\s\src\vm\virtualcallstub.cpp Line: 2382
          Image: C:\dotnetbuild\work\7c1786e6-2823-4b87-b0ba-11326d41c708\Payload\CoreRun.exe

      Return code:      1
      Raw output file:      C:\dotnetbuild\work\7c1786e6-2823-4b87-b0ba-11326d41c708\Work\158756df-09c2-4928-8d40-2039cf026ed9\Exec\Interop\ICustomMarshaler\Reports\Interop.ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.output.txt
      Raw output:
      BEGIN EXECUTION
       "C:\dotnetbuild\work\7c1786e6-2823-4b87-b0ba-11326d41c708\Payload\corerun.exe" ICustomMarshaler.dll 
      Discovering...
      Running 25 of 25 tests...
      Expected: 100
      Actual: -1073740286
      END EXECUTION - FAILED
      FAILED
      Test Harness Exitcode is : 1
VincentBu commented 4 years ago

Job: coreclr-jitstress #20191022.2 Run Test Pri1 Windows_NT x64 checked

Detail: https://helix.dot.net/api/2019-06-17/jobs/4444b70b-f010-4b5f-8256-af7ac93af02e/workitems/Interop/console

OS & Arch: windows x64

Mode: COMPlus_TieredCompilation=0 COMPlus_TailcallStress=1

Note: This failure repros on our windows x64 machine.

Log:

Interop\ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.cmd [FAIL]

      Assert failure(PID 2248 [0x000008c8], Thread: 7952 [0x1f10]): Consistency check failed: FAILED: !implSlot.IsNull() && "Valid method implementation was not found."

      CORECLR! CHECK::Trigger + 0x2C4 (0x00007ffb`a808dca4)
      CORECLR! VirtualCallStubManager::Resolver + 0x14BC (0x00007ffb`a83d6d74)
      CORECLR! VirtualCallStubManager::ResolveWorker + 0xB85 (0x00007ffb`a83d4ca9)
      CORECLR! VSD_ResolveWorker + 0xA8E (0x00007ffb`a83d93be)
      CORECLR! ResolveWorkerAsmStub + 0x6D (0x00007ffb`a877cf4d)
      <no module>! <no symbol> + 0x0 (0x00007ffb`495fe2d3)
      <no module>! <no symbol> + 0x0 (0x000001bf`0ef4aeb0)
      <no module>! <no symbol> + 0x0 (0x000001bf`0f0912c0)
      <no module>! <no symbol> + 0x0 (0x000001bf`0ef87c60)
      CORECLR! `string' + 0x0 (0x00007ffb`a8bda178)
          File: f:\workspace\_work\1\s\src\vm\virtualcallstub.cpp Line: 2382
          Image: C:\dotnetbuild\work\4444b70b-f010-4b5f-8256-af7ac93af02e\Payload\CoreRun.exe

      Return code:      1
      Raw output file:      C:\dotnetbuild\work\4444b70b-f010-4b5f-8256-af7ac93af02e\Work\be814de6-69db-44ad-b214-ef45e0652d9b\Exec\Interop\ICustomMarshaler\Reports\Interop.ICustomMarshaler\Primitives\ICustomMarshaler\ICustomMarshaler.output.txt
      Raw output:
      BEGIN EXECUTION
       "C:\dotnetbuild\work\4444b70b-f010-4b5f-8256-af7ac93af02e\Payload\corerun.exe" ICustomMarshaler.dll 
      Discovering...
      Running 25 of 25 tests...
      Expected: 100
      Actual: -1073740286
      END EXECUTION - FAILED
      FAILED
      Test Harness Exitcode is : 1
BruceForstall commented 3 years ago

This no longer repros