Potential bug with CSharpSymbolIsBannedAnalyzer when analyzing CSharpScript compilation

[haacked]

Analyzer

Diagnostic ID: RS0030: The symbol '{0}' is banned in this project

Analyzer source

NuGet Package: Microsoft.CodeAnalysis.BannedApiAnalyzers

Version: v3.3.1 (Latest)

Describe the bug

Attempting to ban a type doesn't catch usage of the type for CSharpScript compilations when calling a void static method of the type.

Steps To Reproduce

1. Create a CSharpScript with the code Console.WriteLine("test");.
2. Set up the CSharpSymbolIsBannedAnalyzer with the line T:System.Console;Don't use System.Console

Expected behavior

There should be a RS0030 diagnostic warning against using System.Console.

Actual behavior

No diagnostic.

Additional context

It might be easier if I just include the code I used to demonstrate this potential bug.

using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp.BannedApiAnalyzers;
using Microsoft.CodeAnalysis.CSharp.Scripting;
using Microsoft.CodeAnalysis.Diagnostics;
using Microsoft.CodeAnalysis.Scripting;
using Microsoft.CodeAnalysis.Text;

class Program
{

    static async Task Main(string[] args)
    {
        Console.WriteLine("CSharpSymbolIsBannedAnalyzer bug repro.");
        Console.WriteLine("---------------------------------------------");
        Console.WriteLine("Banned Symbols are: ");
        Console.WriteLine(BannedSymbolsAdditionalText.BannedSymbols);
        Console.WriteLine();

        await TestAsync(@"Console.WriteLine(""This should be banned!"");");
        Console.WriteLine();
        await TestAsync("var env = Environment.CommandLine;");
        Console.WriteLine();
    }

    static async Task TestAsync(string code)
    {

        var script = CreateScript(code);
        var diagnostics = await CompileCodeAsync(script);
        Console.WriteLine($"Compiling the code `{code}`");
        Console.WriteLine($"Resulted in {diagnostics.Length} diagnostics.");
        foreach (var diagnostic in diagnostics)
        {
            Console.WriteLine(diagnostic);
        }

        Console.WriteLine("Running the script produces the output:");
        await script.RunAsync(new object());
    }

    static Script CreateScript(string code)
    {
        static IEnumerable<string> GetSystemAssemblyPaths()
        {
            var assemblyPath = Path.GetDirectoryName(typeof(object).Assembly.Location)
                               ?? throw new InvalidOperationException("Could not find the assembly for object.");
            yield return typeof(object).Assembly.Location;
            yield return Path.Combine(assemblyPath, "mscorlib.dll");
            yield return Path.Combine(assemblyPath, "System.dll");
            yield return Path.Combine(assemblyPath, "System.Core.dll");
            yield return Path.Combine(assemblyPath, "System.Console.dll");
            yield return Path.Combine(assemblyPath, "System.Runtime.dll");
            yield return Path.Combine(assemblyPath, "System.Private.CoreLib.dll");
            yield return Path.Combine(assemblyPath, "System.Runtime.Extensions.dll");
        }

        var references = GetSystemAssemblyPaths()
            .Select(path => MetadataReference.CreateFromFile(path)).ToList();

        var options = ScriptOptions.Default
            .WithImports("System", "System.IO")
            .WithEmitDebugInformation(true)
            .WithReferences(references)
            .WithAllowUnsafe(false);

        return CSharpScript.Create(code, globalsType: typeof(object), options: options);
    }

    static async Task<ImmutableArray<Diagnostic>> CompileCodeAsync(Script script)
    {
        var compilation = script.GetCompilation();
        var analyzers = ImmutableArray.Create<DiagnosticAnalyzer>(
            new CSharpSymbolIsBannedAnalyzer()
        );

        var compilationWithAnalyzers = new CompilationWithAnalyzers(
            compilation,
            analyzers,
            new AnalyzerOptions(ImmutableArray.Create((AdditionalText)new BannedSymbolsAdditionalText())),
            CancellationToken.None);

        return await compilationWithAnalyzers.GetAllDiagnosticsAsync();
    }
}

public class BannedSymbolsAdditionalText : AdditionalText
{
    public const string BannedSymbols = @"T:System.Console;Don't use System.Console
T:System.Environment;Don't use System.Environment";
    public override SourceText GetText(CancellationToken cancellationToken = new CancellationToken())
    {
        return SourceText.From(BannedSymbols);
    }

    public override string Path { get; } = "BannedSymbols.txt";
}

And the output is:

CSharpSymbolIsBannedAnalyzer bug repro.
---------------------------------------------
Banned Symbols are: 
T:System.Console;Don't use System.Console
T:System.Environment;Don't use System.Environment

Compiling the code `Console.WriteLine("This should be banned!");`
Resulted in 0 diagnostics.
Running the script produces the output:
This should be banned!

Compiling the code `var env = Environment.CommandLine;`
Resulted in 1 diagnostics.
(1,11): warning RS0030: The symbol 'Environment' is banned in this project: Don't use System.Environment
Running the script produces the output:

As you can see, there is no warning for the code Console.WriteLine("This should be banned!"); but there is a warning for var env = Environment.CommandLine. Not only that, I was able to successfully run the Console.WriteLine script even though it should be banned.

[Evangelink]

Hey @haacked sorry for the delay in reviewing this issue. I am not really familiar with the use-case you are describing, I am just supposed to copy-paste the code in a new project?

Also, would you mind giving a shot with the latest version of the analyzer?

[haacked]

Hi @Evangelink, it's admittedly a slightly unusual use case.

I want to run the CSharpSymbolIsBannedAnalyzer against code that's created using CSharpScript.Create and then compiled script.CompileCodeAsync().

I couldn't find a way to programmatically reference CSharpSymbolIsBannedAnalyzer so I just cloned the dotnet/roslyn-analyzers repo and added my own Program.cs (which I included in the issue).

In the BannedSymbolsAdditionalText I'm attempting to ban System.Console and System.Environment.

And as you can see from my code, it fails to report a diagnostic for the following code:

Console.WriteLine("This should be banned!");

But it works fine and reports a diagnostic for:

var env = Environment.CommandLine;

I expected that a diagnostic would be reported for Console.WriteLine(...).

[junsan1]

Hi guys. Maybe someone can advice? How to use CSharpSymbolIsBannedAnalyzer in own project? I've put the package ref on https://www.nuget.org/packages/Microsoft.CodeAnalysis.BannedApiAnalyzers but the code (simple console app in where dynamic code compilation implemented). I've tried code like that

var analyzers = ImmutableArray.Create<DiagnosticAnalyzer>(
    new CSharpSymbolIsBannedAnalyzer());

var compilationWithAnalyzers = compilation.WithAnalyzers(
    analyzers,
    new AnalyzerOptions(ImmutableArray.Create<AdditionalText>(new BannedSymbolsAdditionalText(@"T:System.Diagnostics.Process;Don't use Process"))),
    cancellationToken);

But my code cannot see CSharpSymbolIsBannedAnalyzer. The package looks like contains some other structure (not as nuget regular reference). Maybe I should use other way?

[hmcguirk]

@junsan1 or @haacked did you work out how to reference the CSharpSymbolIsBannedAnalyzer for use on a C# script, like in your snipped? Im facing exactly the same issue, unsure what to do except perhaps to replicate the analyzer code in my script host.