dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.27k stars 4.73k forks source link

Update BinderTracingTest.AssemblyLoadContextResolving_ReturnNull to not assume reflection semantics #68521

Open steveharter opened 2 years ago

steveharter commented 2 years ago

Description

The PR https://github.com/dotnet/runtime/pull/67917 caused the following tests to the tests to fail due to differences between the current Native-based Invoke and the new Emit-based Invoke:

AssemblyLoadFromResolveHandler_MissingDependency
FindInLoadContext_CustomALC_IncompatibleVersion
ApplicationAssemblies_MismatchedAssemblyName
AssemblyLoadContextResolvingEvent_CustomALC
AssemblyLoadContextResolvingEvent_CustomALC_Exception
AppDomainAssemblyResolveEvent_CustomALC
AppDomainAssemblyResolveEvent_Exception

It appears extra load event(s) are occurring in some cases such as for a referenced assembly ("AssemblyToLoad_Subdirectory") when using the Emit-based Invoke. These are all believed to be natural managed behavior that differs from the native behavior that doesn't require assembly dependencies to be loaded early.

Reproduction Steps

Force IL.Emit to be used:

Useful shortcut example for debugging purposes:

set CORE_ROOT=C:\git\REPO\artifacts\tests\coreclr\windows.x64.Checked\Tests\Core_Root
C:\REPO\artifacts\tests\coreclr\windows.x64.Checked\Loader\binding\tracing\BinderTracingTest.ResolutionFlow\BinderTracingTest.ResolutionFlow.dll

The CI output from the PR above:

[12:39:01 AM] Running AssemblyLoadFromResolveHandler_MissingDependency...
Test AssemblyLoadFromResolveHandler_MissingDependency failed: Xunit.Sdk.TrueException: Bind event count for AssemblyToLoadDependency, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null - expected: 1, actual: 2
Expected: True
Actual:   False
at Xunit.Assert.True(Nullable`1 condition, String userMessage) in //src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
at Xunit.Assert.True(Boolean condition, String userMessage) in //src/xunit.assert/Asserts/BooleanAsserts.cs:line 116
at BinderTracingTests.BinderTracingTest.ValidateSingleBind(BinderEventListener listener, AssemblyName assemblyName, BindOperation expected)
at BinderTracingTests.BinderTracingTest.RunSingleTest(MethodInfo method)

Expected: 100
Actual: 101
END EXECUTION - FAILED
FAILED
Test Harness Exitcode is : 1
To run the test:

set CORE_ROOT=C:\h\w\AE49091F\p
C:\h\w\AE49091F\w\ABA1097B\e\Loader\binding\tracing\BinderTracingTest.ResolutionFlow\BinderTracingTest.ResolutionFlow.cmd
Expected: True
Actual:   False

Expected behavior

The test updated to support both Emit and Native modes.

It is also possible we add support to force the invoke call to either go to Emit or Native deterministically and the test split into two to handle each case, such as by using BindingFlags options.

Actual behavior

See the Repro steps; the test fails when Invoke uses the IL Emit path.

Regression?

Yes; as mentioned this occurs in the PR above. Note if the test was written to not use reflection, it would have worked as-is. The test appears to be influenced by the managed\Emit path.

Known Workarounds

No response

Configuration

No response

Other information

No response

ghost commented 2 years ago

Tagging subscribers to this area: @vitek-karas, @agocke, @vsadov See info in area-owners.md if you want to be subscribed.

Issue Details
### Description This is a placeholder until https://github.com/dotnet/runtime/pull/67917 is in While preparing the PR https://github.com/dotnet/runtime/pull/67917 the test `BinderTracingTest.AssemblyLoadContextResolving_ReturnNull` (in `\src\tests\Loader\binding\tracing\BinderTracingTest.EventHandlers.cs`) fails due to differences between the current Native-based Invoke and the new Emit-based Invoke where a managed call is made to the target method. An extra load event for a referenced assembly ("AssemblyToLoad_Subdirectory") is performed when using the Emit-based Invoke. ### Reproduction Steps Force IL.Emit to be used. _Todo: update this once the PR is in._ To run the test on Windows: ``` src\tests\build.cmd x64 Release src\tests\run.cmd x64 Release ``` Useful shortcut for debugging purposes: ``` set CORE_ROOT=C:\REPO\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Ro%CORE_ROOT%\corerun.exe C:\REPO\artifacts\tests\coreclr\windows.x64.Release\Loader\binding\tracing\BinderTracingTest.ResolutionFlow\BinderTracingTest.ResolutionFlow.dll ``` The CI output from the PR above: ``` [12:39:01 AM] Running AssemblyLoadFromResolveHandler_MissingDependency... Test AssemblyLoadFromResolveHandler_MissingDependency failed: Xunit.Sdk.TrueException: Bind event count for AssemblyToLoadDependency, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null - expected: 1, actual: 2 Expected: True Actual: False at Xunit.Assert.True(Nullable`1 condition, String userMessage) in //src/xunit.assert/Asserts/BooleanAsserts.cs:line 132 at Xunit.Assert.True(Boolean condition, String userMessage) in //src/xunit.assert/Asserts/BooleanAsserts.cs:line 116 at BinderTracingTests.BinderTracingTest.ValidateSingleBind(BinderEventListener listener, AssemblyName assemblyName, BindOperation expected) at BinderTracingTests.BinderTracingTest.RunSingleTest(MethodInfo method) Expected: 100 Actual: 101 END EXECUTION - FAILED FAILED Test Harness Exitcode is : 1 To run the test: set CORE_ROOT=C:\h\w\AE49091F\p C:\h\w\AE49091F\w\ABA1097B\e\Loader\binding\tracing\BinderTracingTest.ResolutionFlow\BinderTracingTest.ResolutionFlow.cmd Expected: True Actual: False ``` ### Expected behavior The test updated to support both Emit and Native modes. It is also possible we add support to force the invoke call to either go to Emit or Native deterministically and the test split into two to handle each case. ### Actual behavior See the Repro steps; the test fails when Invoke uses the IL Emit path. ### Regression? Yes; as mentioned this occurs in the PR above. Note if the test was written to not use reflection, it would have worked as-is. The test appears to be influenced by the managed\Emit path. ### Known Workarounds _No response_ ### Configuration _No response_ ### Other information _No response_
Author: steveharter
Assignees: -
Labels: `disabled-test`, `area-AssemblyLoader-coreclr`
Milestone: -