microsoft / testfx

MSTest framework and adapter
MIT License
780 stars 259 forks source link

Broken backwards compatibility with TestContext? #594

Closed veljkoz closed 5 years ago

veljkoz commented 5 years ago

Description

When using 1.4.0 (also doesn't look to be working with 1.3.2), it seems we can no longer use the SqlDatabaseTestClass (from Microsoft.Data.Tools.Schema.Sql.UnitTesting). The error thrown during test run:

Unable to set TestContext property for the class MyNamespace.MyClass. Error: System.ArgumentException: Object of type 
'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.TestContextImplementation' cannot be converted to type 'Microsoft.VisualStudio.TestTools.UnitTesting.TestContext'..

How do I approach this issue? Any hints on how to resolve it? I thought about subclassing the SqlDatabaseTestClass and hiding the TestContext property with "new", but then I'm still left with an issue of adding a reference to the UnitTesting dll.

Steps to reproduce

  1. Create new Test project for desktop

  2. Add the following nugets:

    <package id="Microsoft.Data.Tools.UnitTest" version="10.0.60611.0" targetFramework="net472" />
    <package id="MSTest.TestAdapter" version="1.4.0" targetFramework="net472" />
    <package id="MSTest.TestFramework" version="1.4.0" targetFramework="net472" />
  3. Change the default UnitTest1 to inherit from "SqlDatabaseTestClass"

  4. Run the test

Expected behavior

Test completes successfully.

Actual behavior

Test fails with error:

Test Name: TestMethod1 Test FullName: UnitTestProject2.UnitTest1.TestMethod1 Test Source: C:\git\source\repos\UnitTestProject1\UnitTestProject2\UnitTest1.cs : line 11 Test Outcome: Failed Test Duration: 0:00:00.1263181

Result StackTrace:
at System.RuntimeType.TryChangeType(Object value, Binder binder, CultureInfo culture, Boolean needsSpecialCast) at System.RuntimeType.CheckValue(Object value, Binder binder, CultureInfo culture, BindingFlags invokeAttr) at System.Reflection.MethodBase.CheckArguments(Object[] parameters, Binder binder, BindingFlags invokeAttr, CultureInfo culture, Signature sig) at System.Reflection.RuntimeMethodInfo.InvokeArgumentsCheck(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Reflection.RuntimePropertyInfo.SetValue(Object obj, Object value, BindingFlags invokeAttr, Binder binder, Object[] index, CultureInfo culture) at System.Reflection.RuntimePropertyInfo.SetValue(Object obj, Object value, Object[] index) Result Message: Unable to set TestContext property for the class UnitTestProject2.UnitTest1. Error: System.ArgumentException: Object of type 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.TestContextImplementation' cannot be converted to type 'Microsoft.VisualStudio.TestTools.UnitTesting.TestContext'..

Environment

VS2019, VS2017, and also from respective vstest.console versions.

veljkoz commented 5 years ago

At this moment I'm not sure if this was ever working (I'm new to the project that had this).

In anycase, I solved it via adapters, or more specifically:

  1. Adding a reference assembly alias in csproj:
    <Reference Include="Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
    <Aliases>LegacyUnitTestFramework</Aliases>
    </Reference>
  2. Wrapping the instance around TestContext providing adapter to the behavior:

extern alias LegacyUnitTestFramework;

using Microsoft.Data.Tools.Schema.Sql.UnitTesting; using Microsoft.VisualStudio.TestTools.UnitTesting; using System.Collections;

namespace MyUnitTestProject { public class CustomSqlDbClass { private class SubClassSqlDatabaseTestClass : SqlDatabaseTestClass { public void CleanupTestPublic() { CleanupTest(); }

        public void InitializeTestPublic()
        {
            InitializeTest();
        }

        public ConnectionContext ExecutionContextPublic => ExecutionContext;
        public ConnectionContext PrivilegedContextPublic => PrivilegedContext;
    }

    public class CustomTestContext : LegacyUnitTestFramework::Microsoft.VisualStudio.TestTools.UnitTesting.TestContext
    {
        public TestContext OriginalTestContext { get; }

        public CustomTestContext(TestContext context)
        {
            OriginalTestContext = context;
        }

        public override IDictionary Properties => OriginalTestContext.Properties;

        public override System.Data.DataRow DataRow => OriginalTestContext.DataRow;

        public override System.Data.Common.DbConnection DataConnection => OriginalTestContext.DataConnection;

        public override void AddResultFile(string fileName)
        {
            OriginalTestContext.AddResultFile(fileName);
        }

        public override void BeginTimer(string timerName)
        {
            OriginalTestContext.BeginTimer(timerName);
        }

        public override void EndTimer(string timerName)
        {
            OriginalTestContext.EndTimer(timerName);
        }

        public override void WriteLine(string format, params object[] args)
        {
            OriginalTestContext.WriteLine(format, args);
        }
    }

    private readonly SubClassSqlDatabaseTestClass adapterSqlDatabaseTestClass;

    public CustomSqlDbClass()
    {
        adapterSqlDatabaseTestClass = new SubClassSqlDatabaseTestClass();
    }

    public static SqlDatabaseTestService TestService => SqlDatabaseTestClass.TestService;

    public TestContext TestContext
    {
        get
        {
            return ((CustomTestContext)adapterSqlDatabaseTestClass.TestContext).OriginalTestContext;
        }
        set
        {
            adapterSqlDatabaseTestClass.TestContext = new CustomTestContext(value);
        }
    }

    public SqlDatabaseTestAction TestInitializeAction => adapterSqlDatabaseTestClass.TestInitializeAction;
    public SqlDatabaseTestAction TestCleanupAction => adapterSqlDatabaseTestClass.TestCleanupAction;
    protected ConnectionContext ExecutionContext => adapterSqlDatabaseTestClass.ExecutionContextPublic;
    protected ConnectionContext PrivilegedContext => adapterSqlDatabaseTestClass.PrivilegedContextPublic;

    protected void CleanupTest()
    {
        adapterSqlDatabaseTestClass.CleanupTestPublic();
    }

    protected void InitializeTest()
    {
        adapterSqlDatabaseTestClass.InitializeTestPublic();
    }
}

}


I haven't tested this thoroughly, because for now we're pursuing a different strategy, but in any case it would be a start. With this above, a simple test works, i.e. the issue is fixed that was without it throwing the exception in the original post:

[TestClass] public class UnitTest1 : CustomSqlDbClass { [TestMethod] public void TestMethod1() { } }