Gallio does not support Visual Studio 2012 Fakes

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Create a Visual Studio 2012 test project that uses Fakes (available in all 
VS versions with Update 2).
2. Right-click a reference ('System' for example) and click "Add Fakes 
Assembly".
3. Use a fakes shim in one of your unit tests. Eg:
[TestMethod]
public void FakeShouldPass()
{
  using (ShimsContext.Create())
  {
    System.Fakes.ShimDateTime.NowGet = () => new DateTime(2000, 1, 1);
    Assert.IsTrue(DateTime.Now.Year == 2000);
  }
}

What is the expected output? What do you see instead?
The test should pass normally and does so when run in Visual Studio. When run 
using Gallio you get the following exception:

Test method UnitTestProject1.UnitTest1.FakeShouldPass threw exception: 
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationExceptio
n: UnitTestIsolation instrumentation failed to initialize. Please restart 
Visual Studio and rerun this test
    at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
   at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
   at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
   at UnitTestProject1.UnitTest1.FakeShouldPass() in s:\Dev\Gallio\UnitTestProject1\UnitTest1.cs:line 13

What version of the product are you using? On what operating system?
Gallio 3.4
Windows 8 x64.
Visual Studio 2012 Ultimate with Update 3

Please provide any additional information below.
Attached the test project example.

Looking at the code I believe this is probably happening because v10.0 of the 
quality tools and MSTest are being loaded in the MSTestAdapter. The fakes 
assemblies will only load when using v11.0 of MSTest (and v12.0 with Visual 
Studio 2013) are being run. The test structure is still the same so you 
probably only need to create a new test runner class for this just to load up 
the newer version of MSTest.exe and corresponding command-line tools.

Original issue reported on code.google.com by werne...@gmail.com on 7 Jul 2013 at 4:34

Attachments:

• UnitTestProject1.zip

[GoogleCodeExporter]

After debugging the Gallio code I figured out it's not as simple as running the 
latest MSTest.exe. The UnitTestFramework assembly is on version 10.0 for VS 
2010, 2012 and 2013 so you need to use a bit of extra magic to determine the 
actual file version not the assembly version to load up the correct test runner.

The second show stopper is that MSTest.exe is not capable of running tests that 
make use of Fakes: 
http://msdn.microsoft.com/en-us/library/vstudio/ms253138.aspx#Runner_VS

You need to use the VS test runner console 
(http://blogs.msdn.com/b/bhuvaneshwari/archive/2012/06/16/vstest-console-exe-com
mandline-test-runner.aspx) to execute these tests in isolation. The commands 
are very similar to MSTest and you can also use /logger:trx to get the same 
consumable results files that Gallio understands.

After some horrible hacking and hard-coding I got things to run in Icarus. When 
I get some time to cleanup I'll submit a patch and hopefully it can make it 
into an official build one day..

Original comment by werne...@gmail.com on 7 Jul 2013 at 6:47

[GoogleCodeExporter]

Any news on a fix for this issue?  were you able to create a patch?

Original comment by zacharia...@gmail.com on 31 Dec 2013 at 3:27

[GoogleCodeExporter]

I have abandoned a fix for this :(

A proper solution is to create a new test extension in Gallio for VSTest. The 
code for the MSTest extension is pretty complex and I found it difficult to 
test writing a new extension.

In the end we just refactored things to use an adapter pattern for statics and 
create a test adapter that replaces the real static calls for unit tests. This 
was easier to maintain and since moving away from Fakes (shims specifically) we 
can still use MSTest (and in turn Gallio) to run the tests successfully.

Only static shims don't work in MSTest, you can use other Fakes such as stubs. 
For anything else you need VSTest.

Perhaps one day I will pick this up again, but there doesn't seem to be a great 
demand for this feature. The driving force for us was using Sonar which uses 
Gallio to run you unit tests, refactoring the code to use adapters was easier 
and IMHO the cleaner approach.

Original comment by werne...@gmail.com on 10 Jan 2014 at 6:43