Shadow copying broken - creates very subtle bugs in tests!

[drauch]

Shadow copying for the mspec R# plugin (tested with R# 9.x) is broken.

The test assembly is loaded twice (once from the shadow copy location and once from the bin\Debug folder). This does not only break the ability to re-compile while executing tests, it also introduces very subtle bugs when using static variables in assembly contexts: the assembly context class is loaded twice into the same app domain and OnAssemblyStart() is executed on the bin\Debug-type while all tests reference the shadow copied-type - i.e. all static variables (which we set in OnAssemblyStart) are null when executing the tests! As you can imagine, it took us quite some time to find the source of the problem...

You can find a repro solution at https://www.dropbox.com/s/0d1wwnfk4mbyon4/MyTest.zip?dl=0. The MyTest.when_executing_test.succeeds_in_same_project works only with disabled shadow copying. Note that MyTest.when_executing_test.succeeds_in_other_project always succeeds, as MyTestBase.dll is only loaded once.

We're not 100% sure, however, it looks like the problem is the Assembly.LoadFrom() call in /Source/Machine.Specifications/Runner/Impl/Listener/AssemblyContextRunListener.cs

[drauch]

Proof screenshot with ProcessExplorer that the assembly is loaded twice:

(edit: unfortunately, VS2013's "Modules" debug window shows only one of the two loaded assemblies...you really have to use ProcessExplorer to see the problem).

[NameOfTheDragon]

This looks like it may be related: http://stackoverflow.com/questions/30643046/mspec-json-net-deserialization-test-fails-in-resharper-but-passes-in-ncrunch/30656635#30656635

When he turned off shadow copying, the problem went away.

I found this in the fusion log:

*** Assembly Binder Log Entry  (05/06/2015 @ 02:01:38) ***
The operation was successful.
Bind result: hr = 0x0. The operation completed successfully.
Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Running under executable  C:\Users\Tim\AppData\Local\JetBrains\Installations\ReSharperPlatformVs12_001\JetBrains.ReSharper.TaskRunner.CLR45.x64.exe
--- A detailed error log follows. 
WRN: The same assembly was loaded into multiple contexts of an application domain:
WRN: Context: Default | Domain ID: 2 | Assembly Name: StackOverflow.30643046, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
WRN: Context: Neither | Domain ID: 2 | Assembly Name: StackOverflow.30643046, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
WRN: This might lead to runtime failures.
WRN: It is recommended to inspect your application on whether this is intentional or not.
WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for more information and common solutions to this issue.
*** Assembly Binder Log Entry  (05/06/2015 @ 02:04:41) ***
The operation was successful.
Bind result: hr = 0x0. The operation completed successfully.
Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Running under executable  C:\Users\Tim\AppData\Local\JetBrains\Installations\ReSharperPlatformVs12_001\JetBrains.ReSharper.TaskRunner.CLR45.x64.exe
--- A detailed error log follows. 
WRN: The same assembly was loaded into multiple contexts of an application domain:
WRN: Context: Default | Domain ID: 2 | Assembly Name: StackOverflow.30643046, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
WRN: Context: Neither | Domain ID: 2 | Assembly Name: StackOverflow.30643046, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
WRN: This might lead to runtime failures.
WRN: It is recommended to inspect your application on whether this is intentional or not.
WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for more information and common solutions to this issue.
*** Assembly Binder Log Entry  (05/06/2015 @ 02:04:42) ***
The operation was successful.
Bind result: hr = 0x0. The operation completed successfully.
Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Running under executable  C:\Users\Tim\AppData\Local\JetBrains\Installations\ReSharperPlatformVs12_001\JetBrains.ReSharper.TaskRunner.CLR45.x64.exe
--- A detailed error log follows. 
WRN: The same assembly was loaded into multiple contexts of an application domain:
WRN: Context: Default | Domain ID: 2 | Assembly Name: StackOverflow.30643046, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
WRN: Context: Neither | Domain ID: 2 | Assembly Name: StackOverflow.30643046, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
WRN: This might lead to runtime failures.
WRN: It is recommended to inspect your application on whether this is intentional or not.
WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for more information and common solutions to this issue.

[ulrichb]

@NameOfTheDragon Good catch! ... I can confirm this, see http://stackoverflow.com/a/30769666/50890.