TestCodeFix Threw An Error

[broderickt2]

I am in the process of creating a unit test using TestCodeFix and this is my first time using it. I could be doing something wrong but want to confirm if there's something else wrong. Here is my unit test code below that is failing:

private string RevCustomAttributeNamespaceClass = @"
using System;
using System.ComponentModel.DataAnnotations;

namespace Rev.App.Validators.DataAnnotations
{
    [AttributeUsage(AttributeTargets.Property | AttributeTargets.Field | AttributeTargets.Parameter, AllowMultiple = false)]
    public class ValidEnumValueAttribute : ValidationAttribute
    {
    }
}";

[TestMethod]
public void AttributeAddedFixWorks()
{
    string code = $@"
using System;
using System.ComponentModel.DataAnnotations;

namespace App
{{
    public class Model
    {{
        [|public Enum_Sample MyProperty {{ get; set; }}|]
    }}

    public enum Enum_Sample
    {{
        Low,
        Medium,
        High
    }}
}}

/*EOD*/

{RevCustomAttributeNamespaceClass}";

    string code2 = $@"
using System;
using System.ComponentModel.DataAnnotations;
using Rev.App.Validators.DataAnnotations;

namespace App
{{
    public class Model
    {{
        [ValidEnumValue]
        public Enum_Sample MyProperty {{ get; set; }}
    }}

    public enum Enum_Sample
    {{
        Low,
        Medium,
        High
    }}
}}

/*EOD*/

{RevCustomAttributeNamespaceClass}";

    var fixture = RoslynFixtureFactory.Create<SyncValidEnumValueAttributeCodeFixProvider>(new CodeFixTestFixtureConfig
    {
        References = new[]
        {
            ReferenceSource.FromType<DataTypeAttribute>(),
            MetadataReference.CreateFromFile(Assembly.Load(new AssemblyName("netstandard")).Location)
        }
    });
    fixture.TestCodeFix(code, code2, SyncValidEnumValueAttributeAnalyzer.DiagnosticId);
}

I know a diagnostic is being thrown from the first code string because my other unit test is passing. Here is that test:

[TestMethod]
public void AttributeNeededDiagnosticThrown()
{
    string code = @"
using System;
using System.ComponentModel.DataAnnotations;

namespace App
{
    public class Model
    {
        [|public Enum_Sample MyProperty { get; set; }|]
    }

    public enum Enum_Sample
    {
        Low,
        Medium,
        High
    }
}";

    var fixture = RoslynFixtureFactory.Create<SyncValidEnumValueAttributeAnalyzer>(new AnalyzerTestFixtureConfig
    {
        References = new[]
        {
            ReferenceSource.FromType<DataTypeAttribute>(),
            MetadataReference.CreateFromFile(Assembly.Load(new AssemblyName("netstandard")).Location)
        }
    });
    fixture.HasDiagnostic(code, SyncValidEnumValueAttributeAnalyzer.DiagnosticId);
}

Full error message and stack trace

Message:  Test method Analyzers.Test.SyncValidEnumValueAttributeUnitTests.AttributeAddedFixWorks threw exception: System.ArgumentException: Argument cannot be empty. (Parameter 'analyzers')

Stack Trace:  CompilationWithAnalyzers.VerifyAnalyzersArgumentForStaticApis(ImmutableArray1 analyzers, Boolean allowDefaultOrEmpty) CompilationWithAnalyzers.VerifyArguments(Compilation compilation, ImmutableArray1 analyzers, CompilationWithAnalyzersOptions analysisOptions) CompilationWithAnalyzers.ctor(Compilation compilation, ImmutableArray1 analyzers, CompilationWithAnalyzersOptions analysisOptions, CancellationToken cancellationToken) CompilationWithAnalyzers.ctor(Compilation compilation, ImmutableArray1 analyzers, AnalyzerOptions options, CancellationToken cancellationToken) DiagnosticAnalyzerExtensions.WithAnalyzers(Compilation compilation, ImmutableArray1 analyzers, AnalyzerOptions options, CancellationToken cancellationToken) CodeFixTestFixture.GetAllReportedDiagnostics(Document document) CodeFixTestFixture.GetReportedDiagnostics(Document document, IDiagnosticLocator locator)+MoveNext() LargeArrayBuilder1.AddRange(IEnumerable1 items) EnumerableHelpers.ToArray[T](IEnumerable1 source) Enumerable.ToArray[TSource](IEnumerable`1 source) CodeFixTestFixture.GetDiagnostic(Document document, String diagnosticId, IDiagnosticLocator locator) CodeFixTestFixture.TestCodeFix(Document document, String expected, String diagnosticId, IDiagnosticLocator locator, ICodeActionSelector codeActionSelector) CodeFixTestFixture.TestCodeFix(String markupCode, String expected, String diagnosticId, ICodeActionSelector actionSelector) CodeFixTestFixture.TestCodeFix(String markupCode, String expected, String diagnosticId, Int32 codeFixIndex)

Here is the code I think the stack trace led to:

private IEnumerable<Diagnostic> GetAllReportedDiagnostics(Document document)
{
    var additionalAnalyzers = CreateAdditionalAnalyzers();
    if (additionalAnalyzers != null)
    {
        var documentTree = document.GetSyntaxTreeAsync().GetAwaiter().GetResult();

        var compilation = document.Project.GetCompilationAsync().GetAwaiter().GetResult();
        return compilation
            .WithAnalyzers(additionalAnalyzers.ToImmutableArray(), new AnalyzerOptions(this.AdditionalFiles?.ToImmutableArray() ?? ImmutableArray<AdditionalText>.Empty))
            .GetAnalyzerDiagnosticsAsync().GetAwaiter().GetResult()
            .Where(x=>x.Location.SourceTree == documentTree);
    }

    return document.GetSemanticModelAsync().GetAwaiter().GetResult().GetDiagnostics();
}

It seems like additionalAnalyzers might be empty, and that's causing the issue when WithAnalyzers is called, just a guess though. Let me know if you have any questions. Thanks!

[broderickt2]

Ok I think I figured it out. I just set the AdditionalAnalyzer when creating my fixture, I think that's what was missing. Assuming this is the right change, might be a good idea to throw a more descriptive exception in GetAllReportedDiagnostics explaining this all.

var fixture = RoslynFixtureFactory.Create<SyncValidEnumValueAttributeCodeFixProvider>(new CodeFixTestFixtureConfig
{
    References = new[]
    {
        ReferenceSource.FromType<DataTypeAttribute>(),
        MetadataReference.CreateFromFile(Assembly.Load(new AssemblyName("netstandard")).Location)
    },
    AdditionalAnalyzers = new[] { new SyncValidEnumValueAttributeAnalyzer() }
});

[broderickt2]

Now I am running into the TransformedCodeDifferentThanExpectedException. Here is what the diff looks like:

I think the EOD file separator isn't being respected because the right side doesn't have the Rev.App.Validators.DataAnnotations namespace I setup

[cezarypiatek]

Exactly, when you test a CodeFix that responds to your custom analyzer diagnostics, then you need to provide an instance of that analyzer.

The current version of RoslynTestKit is after significant API rework - I moved from the test class inheritance approach to the explicit creation of test fixture. Some things still might needs some adjustments. Thanks for the feedback.

UPDATE: The expected pattern should be exactly as the transformed document (single file with the marker)

[broderickt2]

Ok I got it to pass now. I basically removed the EOD part and the namespace class. Here is what it looks like now in case you were curious:

[TestMethod]
public void AttributeAddedFixWorks()
{
    string code = $@"
using System;
using System.ComponentModel.DataAnnotations;

namespace App
{{
    public class Model
    {{
        [|public Enum_Sample MyProperty {{ get; set; }}|]
    }}

    public enum Enum_Sample
    {{
        Low,
        Medium,
        High
    }}
}}";

string code2 = $@"
using System;
using System.ComponentModel.DataAnnotations;
using Rev.App.Validators.DataAnnotations;

namespace App
{{
    public class Model
    {{
        [ValidEnumValue]
        public Enum_Sample MyProperty {{ get; set; }}
    }}

    public enum Enum_Sample
    {{
        Low,
        Medium,
        High
    }}
}}";

    var fixture = RoslynFixtureFactory.Create<SyncValidEnumValueAttributeCodeFixProvider>(new CodeFixTestFixtureConfig
    {
        References = new[]
        {
            ReferenceSource.FromType<DataTypeAttribute>(),
            MetadataReference.CreateFromFile(Assembly.Load(new AssemblyName("netstandard")).Location)
        },
        AdditionalAnalyzers = new[] { new SyncValidEnumValueAttributeAnalyzer() }
    });
    fixture.TestCodeFix(code, code2, SyncValidEnumValueAttributeAnalyzer.DiagnosticId);
}

I originally had added the extra class at the end of both code variables because I didn't think it was going to compile without the namespace definition also being included so it is interesting it still compiles/works like this.

[broderickt2]

Closing this, I figured it out as explained above.