Support DataSourceAttribute in .Net Core.

[magol]

Description

We have following code

[TestMethod]
[DeploymentItem("MyTests.Configuration.xml")]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "|DataDirectory|\\MyTests.Configuration.xml", "MyTestCase", DataAccessMethod.Sequential)]
public void GetFiles_OnRootDirectory_CallsApiCorrectly()
{
    var foo = TestContext.DataRow["MyFooData"]);
}

with following xml file

<?xml version="1.0" encoding="utf-8" ?>
<Rows>
  <MyTestCase>
    <MyFooData>true</MyFooData>
  </MyTestCase>
  <MyTestCase>
    <MyFooData>false</MyFooData>
  </MyTestCase>

  <OtherTestCase>
    <MyFooData>false</MyFooData>
  </OtherTestCase>
</Rows>

Is TestContext.DataRow expected to come to MsTest v2, or is it any other way to do the same thing?

Steps to reproduce

1. Install .NET Core 2.0.0-preview2-006497
2. Target the test project to ,NET Core 2.0
3. Use MSTest.TestAdapter version 1.2.0-beta
4. Try to build

Expected behavior

The compile past

Actual behavior

I get following build error

error CS1061: 'TestContext' does not contain a definition for 'DataRow' and no extension method 'DataRow' accepting a first argument of type 'TestContext' could be found (are you missing a using directive or an assembly reference?)

Environment

.NET Core 2.0.0-preview2-006497 Visual Studio 15.3.0 preview 7.0 Windows 10

[AbhitejJohn]

@magol: Apologies for the delay. The .Net Core version of the adapter does not support DataSource yet. I'll update the title and mark this as a request for adding in that support. As a workaround, you could update to the latest beta and use DynamicData instead.

[magol]

@AbhitejJohn Thanks for the answer. What is you time framme to add this feature? When can I expert to be able to use this?

[AbhitejJohn]

@magol: Don't have one yet. Put this temporarily in S123 where it would most likely be addressed.

[nvianhd]

I hope MS still support DataRow on Dot Net Core

[AbhitejJohn]

@nvianhd: It still does and much more!

[magol]

@AbhitejJohn What is the status on this? The S123 milestone have past due by about 1 month.

[AbhitejJohn]

@magol : Sorry, we had a few other things that came up and this took the back seat. Tagging @sudiptadmsft and @pvlakshm to see when we can bring this in.

[omkarmore83]

I am waiting on this too.. @AbhitejJohn any ETA?

[rhyous]

Trying to move my Unit Tests to .NET Core 2.0. Of course, I have many row tests. I'll have a look at Dynamic data. But the problem is all the existing row tests use DataSourceAttribute.

[alex182]

this would be great to have working

[magol]

Any progress on this?

[AbhitejJohn]

Sorry, I haven't been working on the framework for a while now. Tagging @cltshivash @pvlakshm .

[alenjalex-zz]

I am also waiting on this.

[sirkris]

More than a year since the initial post and still waiting. This is unacceptable, as it makes it impossible to make use of data-driven unit tests in .NET Core. Please fix this! I'm gonna have to hack in a really sloppy workaround for my test project to work in the meantime....

[cilerler]

As @AbhitejJohn already mentioned above DynamicData is way to go at this point. Here is an example for CSV reading

private static string[] SplitCsv(string input)
{
    var csvSplit = new Regex("(?:^|,)(\"(?:[^\"]+|\"\")*\"|[^,]*)", RegexOptions.Compiled);
    var list = new List<string>();
    foreach (Match match in csvSplit.Matches(input))
    {
        string value = match.Value;
        if (value.Length == 0)
        {
            list.Add(string.Empty);
        }

        list.Add(value.TrimStart(','));
    }
    return list.ToArray();
}

private static IEnumerable<string[]> GetData()
{
    IEnumerable<string> rows = System.IO.File.ReadAllLines(@"Resources\NameAddressCityStateZip.csv").Skip(1);
    foreach (string row in rows)
    {
        yield return SplitCsv(row);
    }
}

[TestMethod]
[DynamicData(nameof(GetData), DynamicDataSourceType.Method)]
//x [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", @"Resources\NameAddressCityStateZip.csv", "NameAddressCityStateZip#csv", DataAccessMethod.Sequential)]
public void TestMethod1(string input, string expected)
{
    // Arrange
    //x string input = _testContext.Properties["Data"].ToString(); //x _testContext.DataRow["Data"].ToString();
    //x string expected = _testContext.Properties["Expected"].ToString(); //x _testContext.DataRow["Expected"].ToString();
    var parser = _serviceProvider.GetService<Parser>();

    // Act
    string actual = parser.MultiParser(input, ModeType.NameAddressCityStateZipCountry).ToString();

    // Assert
    Assert.AreEqual(expected, actual);
}

[rhyous]

@cilerler It is NOT that easy.

Is the DynamicDataAttribute available in other .NET platforms yet?

We have a bunch of code that compiles to multiple .Net versions. The code and unit tests are shared between platforms. So if I change the unit tests to use DynamicData, I break them for .Net Framework. So basically, I cannot share unit test code.

At best I could add a compiler directive to all my unit tests so in .NET Framework it uses the old code and in .NetCore it uses the new code. Actually, I already did this in my open source projects, only the unit tests are simply not run in .NetCore as it doesn't support the DataSourceAttribute.

Not to mention that the GetData() method you have above 1) doesn't take into account Xml row tests or Excel row tests, and 2) is far too naive to be used by product code. We need to get this working for Csv, Xml, and Excel. I have a CsvParser, but I have to figure out what Xml parser to use or write my own, figure out what Excel parser to use or write my own, etc. Now I have to bring in either new code, which requires approval and sprint time, etc, or new open source projects, which means a security team involved to vet the open source project and paperwork and process.

So again, if it were that easy, we would have done it already.

[sirkris]

@rhyous is spot-on. Relying on DynamicData instead of just implementing this (is there some pressing reason not to?) is not a viable long-term solution to this problem, in my opinion.

[cilerler]

@rhyous

1. The very basic example I provided above is for demonstration purpose only to let others see how easy it is to implement it. I literally didn't expect no one to use it for production.
2. As you already mentioned above you have compiler directives which means it should not break your code base. (you can't expect everyone having a need to support multiple frameworks, many developers are developing fresh new projects and the method I presented may help them to continue with the latest stuff)
3. The situation is pretty simple writing your own parser VS waiting for years to get it from Microsoft. (Well, I usually prefer to write my own at this point in order to be able to use the latest stuff)

Therefore please take it into the consideration that it could be very easy workaround where things are not complicated as it in your projects.

[rhyous]

Wow. Let's ignore that unhelpful response and talk about implementation of a single solution in the right place, which is in this project, instead having customers implement many one-off solutions.

It would be better to implement this once than to have many users implement their own hacks. It looks like this feature is tagged with up-for-grabs so anyone can implement it. That is greateness of open source.

So if DynamicDataAttribute is the future, why not implement DataSourceAttribute in this project as a child of DynamicDataAttribute.

We could probably write an attribute that inherits from DynamicDataAttribute (except the class is sealed, but that is easy to change, it is an open source project after all) and simply have child class that acts exactly like the DynamicDataAttribute but has the signature of the DataSourceAttribute and we enhance the DataSourceAttribute to populate the GetData needs.

Assuming the posters here decide to fork testfx, implement this ourselves, and do a pull request. We could use the source for DataSourceAttribute and associated classes. Can someone at Microsoft comment on if that code can be made available?

[sebainones]

I'm facing the same situation. I was trying to move my Tests to .Net Core but for me is strictly necessary to use DataRow as my information comes only from the DB. So, without this, I wouldn't migrate to .Net Core. @AbhitejJohn It's been more than a year since the post. Is there any ETA or at least do you have on your product backlog. I want to know if it makes sense to wait or if I should be considering other options. Thanks

[Justin-Lloyd]

+1 on the above. Unit testing does not look particularly well supported by Microsoft.

[hansmbakker]

DataSourceAttribute / DataRow would be required for https://github.com/Microsoft/Recognizers-Text to move to .Net Core for tests.

[AbhitejJohn]

@AbhitejJohn It's been more than a year since the post. Is there any ETA or at least do you have on your product backlog.

@sebainones : I haven't been getting much time since I've moved off of working on this area. @cltshivash should be able to help answer if this is on the immediate backlog. What @cilerler has added seems to be good start to unblock folks. It does create an API diff between .NetCore and .NetFramework but the DynamicData schematics was what we were hoping we could converge on.

[mpettus]

Any update on this? I could really use DataRow in my project. Thanks.

[Sl1ckR1ck]

Waiting for this too before continuing porting to .NET Core. No point to start writing reading CSV, DB, etc code if this is coming soon?

I see that @nohwnd assigned this work, so i guess its being implemented?

[nohwnd]

@Sl1ckR1ck it is on my work list, but I did not even get a chance to read through this thread properly. Swamped with other issues. Sorry! 😔

[sebainones]

@nohwnd and @AbhitejJohn I would like to help (contribute) with this. Could you please tell me how could I help to move forward with this? Let me know if I can review some code or even do some coding. (I'm pretty new about contributing but I would love to)

[nohwnd]

@sebainones Thank you for showing interest in this, I think it would be nice to sum up few examples of what is the desired functionality is, and what is missing from core so we have a clear way of going forward. I will in the meantime try to figure out if we can take the existing Framework code and port it to core. I don't know what the procedure here is, and I did not see the code to be already publicly available in this repo.

[sebainones]

Desired Functionality

1. The example provided by @magol when he initially opened the issue. That's an XML as DataSource

2 This is my scenario using a Table (or View) as DataSource

    [TestClass]
    public class MyTestClass
    {
        static TestContext testContext;

        [ClassInitialize]
        public static void ClassInitialize(TestContext context)
        {        
            testContext = context;
        }

        [TestMethod]
        [DeploymentItem(@".\bin\EntityFramework.SqlServer.dll")]
        [DataTestMethod]
        [DataSource("System.Data.SqlClient", "ConnectionString", "tableName", DataAccessMethod.Sequential)]
        public void RunMyDataDrivenTests()
        {          
             var someFiled= testContext.DataRow["SomeColumnFromMyTable"];
        }

What is missing from .Net Core?

As the title originally said : "TestContext' does not contain a definition for 'DataRow' " So, if your create a MSTest Test Project with .NET Core when you try to access the DataRow property you will get an error because TestContext' does not contain a definition for DataRow

If you go to the .Net Framework TestContext Class you will get this:

namespace Microsoft.VisualStudio.TestTools.UnitTesting
{
    //
    // Summary:
    //     Used to store information that is provided to unit tests.
    public abstract class TestContext
    {
        protected TestContext();

         //
        // Summary:
        //     Gets the current data row when test is used for data driven testing.
        public abstract DataRow DataRow { get; }
    }

By contrast, if you go to the .NET Core TestContext Class you will get this:

namespace Microsoft.VisualStudio.TestTools.UnitTesting
{
    //
    // Summary:
    //     TestContext class. This class should be fully abstract and not contain any members.
    //     The adapter will implement the members. Users in the framework should only access
    //     this via a well-defined interface.
    public abstract class TestContext
    {
        protected TestContext();

     //There is NO DataRow property!
    }

Note: I have just copied the relevant parts of the code.

@nohwnd: Please let me know if you need that I put those examples in a github repo or something. In addition, I would like to help in taking the existing Framework code and port it to core (If that's possible).

[nohwnd]

@sebainones great. 👍

In addition, I would like to help in taking the existing Framework code and port it to core (If that's possible).

I am currently investigating where to get that code and how to make it open-source if it was not before.

[duaneking]

Any updates?

[AbhitejJohn]

@nohwnd : That code currently is used in the .NET Fx version of the adapter hereand here. I wonder if some of these types would just build against a later netstandard version. The reason these weren't initially available was because the standard did not support them. Looks like DataRow that's exposed in the MSTest Desktop Frameworks TestContext is available in netstandard2.0 and upwards.

[Purush-India]

Hello Everyone,

Do we have any update on DataRow being made available in .Net Core yet ? Is their any alternative that we can use to access Excel using data source in ,.net core

[kwacks]

Any update on this? This is a pivotal feature missing for data driven tests .net core.

[nohwnd]

Looked at this recently and the system.data is available since netstandard3.0, and mstest is bsaed on Protable259 which is equivalent to netstandard1.0. So we would have to update, or possibly publish a separate package.

[hansmbakker]

@kwacks and others: you can create your own attribute based on ITestDataSource that loads the data you want from files in your own format.

See https://github.com/microsoft/Recognizers-Text/pull/1050#issuecomment-447609178 and https://github.com/microsoft/Recognizers-Text/pull/1050/files#diff-31e5f7adcb60c743fdeb83b6c3ca08f7 for more details.

[sebainones]

@kwacks and others: you can create your own attribute based on ITestDataSource that loads the data you want from files in your own format.

See microsoft/Recognizers-Text#1050 (comment) and https://github.com/microsoft/Recognizers-Text/pull/1050/files#diff-31e5f7adcb60c743fdeb83b6c3ca08f7 for more details.

Thanks for the piece of advice. I will take a look at that (even I have solved on my own in a different way) Anyways, it would be great if we can have this already as part of .NET Core. And again, if I could help to code or reviewing, please let me know.

[shilpamalhotra-git]

Is there any update on this? How are we usually doing Data Driven testing in .net core with MsTest. I have now been struggling a lot on this. @sebainones Can you please help by sharing your approach.

[DustinKingen]

I encountered this issue porting .NET Framework style csproj to .NET Core style csproj (.NET FX 4.8). I will update my comment if I find a workaround.

[fhucko]

I have opened links presented here in comments, and they are useless or such voodoo that I do not understand it. Is there a clean replace solution? I guess no. I will just write my own solution, like rest of the world.

[gburghardt]

I found a solution I am happy with using the CsvHelper NuGet package and the [DynamicData] attribute.

My answer on StackOverflow to What is the replacement for TestContext.DataRow[“MyColumnName”] shows all the details. A brief summary is here.

The CSV File (Example.csv)

A,B,IsLessThanZero
1,2,FALSE
3,-5,TRUE

Important: Make sure this CSV file is included in your test project and "Copy To Output Directory" is set to "Always" in the properties for the CSV file in Solution Explorer.

Data Transfer Object Used By CsvHelper

public class AdditionData
{
    public int A { get; set; }
    public int B { get; set; }
    public bool IsLessThanZero { get; set; }
}

The Test Class

[TestClass]
public class ExampleTests
{
    // HINT: Look in {Your Test Project Folder}\bin\{Configuration}\netcore3.1\FolderYourCsvFileIsIn for the CSV file.
    //       Change this path to work with your test project folder structure.
    private static readonly string DataFilePath = Path.GetDirectoryName(typeof(ExampleTests).Assembly.Location) + @"\FolderYourCsvFileIsIn\Example.csv";

    [TestMethod]
    [DynamicData(nameof(GetData), DynamicDataSourceType.Method)]
    public void AddingTwoNumbers(AdditionData data)
    {
        bool isLessThanZero = data.A + data.B < 0;

        Assert.AreEqual(data.IsLessThanZero, isLessThanZero);
    }

    private static IEnumerable<object[]> GetData()
    {
        using var stream = new StreamReader(DataFilePath);
        using var reader = new CsvReader(stream, new CsvConfiguration(CultureInfo.CurrentCulture));
        var rows = reader.GetRecords<AdditionData>();

        foreach (var row in rows)
        {
            yield return new object[] { row };
        }
    }
}

After building your solution, you will see a single test named "AddingTwoNumbers" in Test Explorer. Running this single test runs all variants defined in your CSV file.

[fforjan]

If anyone is here, if you are using XML, this is what we have attempted on our side :

[AttributeUsage(AttributeTargets.Method)]
    public class XMLDataSourceAttribute : Attribute, ITestDataSource
    {
        private readonly string path;

        public XMLDataSourceAttribute (string path)
        {
            this.path = path;
        }

        public IEnumerable<object[]> GetData(MethodInfo methodInfo)
        {       
             var parameters = methodInfo.GetParameters(); 

            var type = parameters[0].ParameterType;  // we assume there is only one parameter
            var constructor = type.GetConstructors().First(); // we assume there is only constructor with the right signature

            var content = XElement.Load(path);
            foreach(var line in content.Elements())
            {
                yield return new[] { constructor.Invoke(new[] { line }) };
            }
        }

        public string GetDisplayName(MethodInfo methodInfo, object[] data)
        {
            return data[0].ToString();
        }            
    }

    public class Data
    {
        public Data(XElement element)
        {
            this.From = element.Element("From").Value;
            this.To = element.Element("To").Value;
            this.Tooltip = element.Element("Tooltip").Value;
            this.BecauseMessage = element.Element("BecauseMessage").Value;
            this.HasError = Convert.ToBoolean(element.Element("HasError").Value);
        }

        public string From { get; private set; }
        public string To { get; private set; }
        public bool HasError { get; private set; }
        public string Tooltip { get; private set; }
        public string BecauseMessage { get; private set; }

        public override string ToString()
        {
            return string.Format(CultureInfo.InvariantCulture, "From '{0}' To '{1}'", this.From, this.To);
        }
    }

and we switched the attribute from

 [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", @"|DataDirectory|\TestData.xml", "TestData",
            DataAccessMethod.Sequential)]
public void TestMethod() 

to

[XMLDataSource(@"TestData.xml")]
public void TestMethod(Data data)  

[sirkris]

Did the attempt succeed?

On Mon, Apr 18, 2022 at 6:37 AM Frederic Forjan @.***> wrote:

If anyone is here, if you are using XML, this is what we have attempted on our side :

[AttributeUsage(AttributeTargets.Method)] public class CurrentDataRowAttribute : Attribute, ITestDataSource { private readonly string path;

    public CurrentDataRowAttribute(string path)
    {
        this.path = path;
    }

    public IEnumerable<object[]> GetData(MethodInfo methodInfo)
    {
        var content = XElement.Load(path);
        foreach(var line in content.Elements())
        {
            yield return new[] { new Data(line) };
        }
    }

    public string GetDisplayName(MethodInfo methodInfo, object[] data)
    {
        return ((Data)data[0]).ToString();
    }
}

public class Data
{
    public Data(XElement element)
    {
        this.From = element.Element("From").Value;
        this.To = element.Element("To").Value;
        this.Tooltip = element.Element("Tooltip").Value;
        this.BecauseMessage = element.Element("BecauseMessage").Value;
        this.HasError = Convert.ToBoolean(element.Element("HasError").Value);
    }

    public string From { get; private set; }
    public string To { get; private set; }
    public bool HasError { get; private set; }
    public string Tooltip { get; private set; }
    public string BecauseMessage { get; private set; }

    public override string ToString()
    {
        return string.Format(CultureInfo.InvariantCulture, "From '{0}' To '{1}'", this.From, this.To);
    }
}

and we switched the attribute from

[DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", @"|DataDirectory|\TestData.xml", "TestData", DataAccessMethod.Sequential)] public void TestMethod()

to

[CurrentDataRow(@"TestData.xml")] public void TestMethod(Data data)

— Reply to this email directly, view it on GitHub https://github.com/microsoft/testfx/issues/233#issuecomment-1101415399, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFTJZJE2TWZPJW6P24TDEDVFVQR5ANCNFSM4DVWQD5A . You are receiving this because you commented.Message ID: @.***>

[rhyous]

@fforjan That is cool. I haven't fixed all my Xml ones. I have a project, github.com/rhyous/unittesting (also there is a NuGet package) that does Json, I haven't implement XML yet.

https://github.com/rhyous/UnitTesting

I kept meaning to get Xml done, but I ended up converting my Xml to json instead.

[fforjan]

@rhyous just curiositry, we have lot of comment into our XML. How did you solve the comment in JSON ?

[fforjan]

@sirkris we have done couple (3 test so far...) of migration and are going to discuss in our team. As @rhyous raised, the other option would be to move to JSON but we are not keen on that due to XML Comments

[rhyous]

Unlike Xml, Json doesn't support comments. They would be considered a syntax error. You can put them in the json as a property or you can't have them.

{
  "Prop1": "value1",
  "Prop1Comment":"Some comment",
}

However, you don't have to deserialize the comment. The object you serialize with can exclude them and they simply won't be serialized.

public class MyObject
{
    public string Prop1 { get; set; }
}

However, I don't usually do that. My only comment is usually a TestName and Message property the is deserialized. The TestName should show up as the test name in VS. The message I pass into the Assert.

{
  "TestName": "Some Test 1",
  ...
  "Message":"The expectation was . . .",
}