CoreCLR tests on Mono runtime : Failures running Reflection/* tests

SamMonoRT commented 4 years ago

Several failures seen when running the reflection/* tests. These are CoreCLR tests run on the mono runtime from the dotnet\runtime repo. It is quite possible, these may need to split in separate issues based on individual issues/missing implementations etc.

Repro : Command to run individual tests :

cd src/mono/netcore
make run-tests-coreclr CoreClrTest="bash ../../artifacts/tests/coreclr/OSX.x64.Debug/reflection/DefaultInterfaceMethods/InvokeConsumer/InvokeConsumer.sh"

reflection/DefaultInterfaceMethods/InvokeConsumer/InvokeConsumer.sh
reflection/DefaultInterfaceMethods/Emit/Emit.sh
reflection/Modifiers/modifiers/modifiers.sh
reflection/SetValue/TrySetReadonlyStaticField/TrySetReadonlyStaticField.sh

Individual failure stacks below :

***** reflection/DefaultInterfaceMethods/InvokeConsumer/InvokeConsumer.sh [FAIL]

      Return code:      1
      Raw output file:      /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/reflection/DefaultInterfaceMethods/Reports/reflection.DefaultInterfaceMethods/InvokeConsumer/InvokeConsumer.output.txt
      Raw output:
      BEGIN EXECUTION
      /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root/corerun InvokeConsumer.dll ''
      Expected: 100
      Actual: 11
      END EXECUTION - FAILED
      Test Harness Exitcode is : 1
      To run the test:
      > set CORE_ROOT=/Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root
      > /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/reflection/DefaultInterfaceMethods/InvokeConsumer/InvokeConsumer.sh
      Expected: True
      Actual:   False
      Stack Trace:
        /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/TestWrappers/reflection.DefaultInterfaceMethods/reflection.DefaultInterfaceMethods.XUnitWrapper.cs(334,0): at reflection_DefaultInterfaceMethods._InvokeConsumer_InvokeConsumer_._InvokeConsumer_InvokeConsumer_sh()
        /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.cs(339,0): at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

***** reflection/DefaultInterfaceMethods/Emit/Emit.sh [FAIL]

      Unhandled Exception:
      System.ArgumentException: Interface method must be abstract and virtual.
         at System.Reflection.Emit.TypeBuilder.DefineMethod(String name, MethodAttributes attributes, CallingConventions callingConvention, Type returnType, Type[] returnTypeRequiredCustomModifiers, Type[] returnTypeOptionalCustomModifiers, Type[] parameterTypes, Type[][] parameterTypeRequiredCustomModifiers, Type[][] parameterTypeOptionalCustomModifiers) in /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/Emit/TypeBuilder.Mono.cs:line 531
         at System.Reflection.Emit.TypeBuilder.DefineMethod(String name, MethodAttributes attributes, CallingConventions callingConvention, Type returnType, Type[] parameterTypes) in /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/Emit/TypeBuilder.Mono.cs:line 508
         at System.Reflection.Emit.TypeBuilder.DefineMethod(String name, MethodAttributes attributes, Type returnType, Type[] parameterTypes) in /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/Emit/TypeBuilder.Mono.cs:line 502
         at Program.Main() in /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/coreclr/tests/src/reflection/DefaultInterfaceMethods/Emit.cs:line 25
      [ERROR] FATAL UNHANDLED EXCEPTION: System.ArgumentException: Interface method must be abstract and virtual.
         at System.Reflection.Emit.TypeBuilder.DefineMethod(String name, MethodAttributes attributes, CallingConventions callingConvention, Type returnType, Type[] returnTypeRequiredCustomModifiers, Type[] returnTypeOptionalCustomModifiers, Type[] parameterTypes, Type[][] parameterTypeRequiredCustomModifiers, Type[][] parameterTypeOptionalCustomModifiers) in /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/Emit/TypeBuilder.Mono.cs:line 531
         at System.Reflection.Emit.TypeBuilder.DefineMethod(String name, MethodAttributes attributes, CallingConventions callingConvention, Type returnType, Type[] parameterTypes) in /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/Emit/TypeBuilder.Mono.cs:line 508
         at System.Reflection.Emit.TypeBuilder.DefineMethod(String name, MethodAttributes attributes, Type returnType, Type[] parameterTypes) in /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/Emit/TypeBuilder.Mono.cs:line 502
         at Program.Main() in /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/coreclr/tests/src/reflection/DefaultInterfaceMethods/Emit.cs:line 25

      Return code:      1
      Raw output file:      /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/reflection/DefaultInterfaceMethods/Reports/reflection.DefaultInterfaceMethods/Emit/Emit.output.txt
      Raw output:
      BEGIN EXECUTION
      /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root/corerun Emit.dll ''
      Expected: 100
      Actual: 1
      END EXECUTION - FAILED
      Test Harness Exitcode is : 1
      To run the test:
      > set CORE_ROOT=/Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root
      > /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/reflection/DefaultInterfaceMethods/Emit/Emit.sh
      Expected: True
      Actual:   False
      Stack Trace:
        /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/TestWrappers/reflection.DefaultInterfaceMethods/reflection.DefaultInterfaceMethods.XUnitWrapper.cs(140,0): at reflection_DefaultInterfaceMethods._Emit_Emit_._Emit_Emit_sh()
        /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.cs(339,0): at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

***** reflection/Modifiers/modifiers/modifiers.sh [FAIL]

      Return code:      1
      Raw output file:      /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/reflection/Modifiers/Reports/reflection.Modifiers/modifiers/modifiers.output.txt
      Raw output:
      BEGIN EXECUTION
      /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root/corerun modifiers.dll ''
      In int32 BaseClass::Override(int32)
      In int32 DerivedClass::Override(int32)
      In int32 SuperDerivedClass::Override(int32)
      In int32 BaseClass::Override(int32 modopt (FooModifier))
      In int32 BaseClass::Override(int32 modopt (BarModifier))
      In int32 BaseClass::Override(int32 modreq (FooModifier))
      In int32 BaseClass::Override(int32)
      In int32 modopt (FooModifier) BaseClass::Override(int32)
      In int32 DerivedClass::Override(int32 modopt (FooModifier))
      In int32 DerivedClass::Override(int32 modopt (BarModifier))
      In int32 DerivedClass::Override(int32 modreq (FooModifier))
      In int32 DerivedClass::Override(int32)
      In int32 modopt (FooModifier) DerivedClass::Override(int32)
      In int32 BaseClass::Override(int32 modopt (FooModifier)[])
      In int32 DerivedClass::Override(int32 modopt (FooModifier)[])
      In int32 BaseClass::Override(int32 modopt (BarModifier)[])
      In int32 BaseClass::Override(int32 modopt (BarModifier)[])
      Expected: 100
      Actual: 46
      END EXECUTION - FAILED
      Test Harness Exitcode is : 1
      To run the test:
      > set CORE_ROOT=/Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root
      > /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/reflection/Modifiers/modifiers/modifiers.sh
      Expected: True
      Actual:   False
      Stack Trace:
        /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/TestWrappers/reflection.Modifiers/reflection.Modifiers.XUnitWrapper.cs(140,0): at reflection_Modifiers._modifiers_modifiers_._modifiers_modifiers_sh()
        /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.cs(339,0): at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

**** reflection/SetValue/TrySetReadonlyStaticField/TrySetReadonlyStaticField.sh [FAIL]

      Return code:      1
      Raw output file:      /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/reflection/SetValue/Reports/reflection.SetValue/TrySetReadonlyStaticField/TrySetReadonlyStaticField.output.txt
      Raw output:
      BEGIN EXECUTION
      /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root/corerun TrySetReadonlyStaticField.dll ''
      Begin initializing class X
      Attempting to update S via SetValue, current value is '0', desired new value is '1'
      Updated S to '1'
      Attempting to update S via SetValueDirect, current value is '1', desired new value is '2'
      Updated S to '2'
      Done initializing class X
      Attempting to update S via SetValue, current value is '2', desired new value is '3'
      Updated S to '3'
      Attempting to update S via SetValueDirect, current value is '3', desired new value is '4'
      Updated S to '4'
      SetValue after class init didn't throw as expected
      SetValueDirect after class init didn't throw as expected
      S is '4' , should be '2'
      FAIL
      Expected: 100
      Actual: 255
      END EXECUTION - FAILED
      Test Harness Exitcode is : 1
      To run the test:
      > set CORE_ROOT=/Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/Tests/Core_Root
      > /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/reflection/SetValue/TrySetReadonlyStaticField/TrySetReadonlyStaticField.sh
      Expected: True
      Actual:   False
      Stack Trace:
        /Users/samarthpatel/Documents/DotNetRunTime/runtime/artifacts/tests/coreclr/OSX.x64.Debug/TestWrappers/reflection.SetValue/reflection.SetValue.XUnitWrapper.cs(140,0): at reflection_SetValue._TrySetReadonlyStaticField_TrySetReadonlyStaticField_._TrySetReadonlyStaticField_TrySetReadonlyStaticField_sh()
        /Users/samarthpatel/Documents/DotNetRunTime/runtime/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.cs(339,0): at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

fanyang-mono commented 2 years ago

Low priority - Moving to Milestone 8.0.0

lambdageek commented 2 years ago

A little bit of triage after I tried enabling them in a PR:

reflection/DefaultInterfaceMethods/Emit fails everywhere. This looks like potentially we just need to remove a check somewhere in SRE. Might be easy.

reflection/DefaultInterfaceMethods/Emit/Emit.sh [FAIL]
      Error: Interface method must be abstract and virtual.
          at $o (/datadisks/disk1/work/AB64091E/w/BE3F0A44/e/reflection/DefaultInterfaceMethods/Emit/WasmApp/dotnet.js:5:37754)
          at Ws (/datadisks/disk1/work/AB64091E/w/BE3F0A44/e/reflection/DefaultInterfaceMethods/Emit/WasmApp/dotnet.js:5:55546)
          at Object.b.javaScriptExports.call_entry_point (/datadisks/disk1/work/AB64091E/w/BE3F0A44/e/reflection/DefaultInterfaceMethods/Emit/WasmApp/dotnet.js:5:58159)
          at Object.Te [as runMain] (/datadisks/disk1/work/AB64091E/w/BE3F0A44/e/reflection/DefaultInterfaceMethods/Emit/WasmApp/dotnet.js:5:2713)
          at run (test-main.js:330:50)
      Error: Interface method must be abstract and virtual.
          at $o (/datadisks/disk1/work/AB64091E/w/BE3F0A44/e/reflection/DefaultInterfaceMethods/Emit/WasmApp/dotnet.js:5:37754)
          at Ws (/datadisks/disk1/work/AB64091E/w/BE3F0A44/e/reflection/DefaultInterfaceMethods/Emit/WasmApp/dotnet.js:5:55546)
          at Object.b.javaScriptExports.call_entry_point (/datadisks/disk1/work/AB64091E/w/BE3F0A44/e/reflection/DefaultInterfaceMethods/Emit/WasmApp/dotnet.js:5:58159)
          at Object.Te [as runMain] (/datadisks/disk1/work/AB64091E/w/BE3F0A44/e/reflection/DefaultInterfaceMethods/Emit/WasmApp/dotnet.js:5:2713)
          at run (test-main.js:330:50)
      exiting due to exception: ManagedError(gc_handle: 54)

On the other hand reflection/DefaultInterfaceMethods/InvokeConsumer is pretty mysterious. Needs more investigation

BEGIN EXECUTION
      /tmp/helix/working/AAF20982/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false InvokeConsumer.dll ''
      Expected: 100
      Actual: 11
      END EXECUTION - FAILED
      Test Harness Exitcode is : 1
      To run the test:
      > set CORE_ROOT=/tmp/helix/working/AAF20982/p
      > /private/tmp/helix/working/AAF20982/w/B6AA09FE/e/reflection/DefaultInterfaceMethods/InvokeConsumer/InvokeConsumer.sh
      Expected: True
      Actual:   False
      Stack Trace:
           at reflection_DefaultInterfaceMethods._InvokeConsumer_InvokeConsumer_._InvokeConsumer_InvokeConsumer_sh()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

fanyang-mono commented 1 year ago

Moving to 9.0.0

fanyang-mono commented 1 year ago

cc: @ivanpovazan

lewing commented 8 months ago

@artl93 this looks like libraries ownership?

cc @steveisok @jeffschwMSFT

fanyang-mono commented 2 months ago

@buyaa-n Could you verify if these tests are still failing, since you've worked on unifying Reflection related infrastructure?

buyaa-n commented 2 months ago

@buyaa-n Could you verify if these tests are still failing, since you've worked on unifying Reflection related infrastructure?

I don't even know how these tests in reflection/DefaultInterfaceMethods works, I'll leave it to you

dotnet / runtime

CoreCLR tests on Mono runtime : Failures running Reflection/* tests #36113