CompilationAction not Triggering

[suprak]

I'm running into an issue with VS 2015 + Roslyn v1.

I read through the analyzer semantic documentation[1].

According to it, I expect that the compilation action I registered in the Initializer, to be executed once at the end of a compilation.

However I am not getting this behavior. The StartCompilation action gets called, but the End never does. Is this a bug? Am I doing it wrong?

        public override void Initialize(AnalysisContext context)
        {
            context.RegisterCompilationAction(this.OnCompilation);
            context.RegisterCompilationStartAction(this.OnStartCompilation);
        }

[1] https://github.com/dotnet/roslyn/blob/master/docs/analyzers/Analyzer%20Actions%20Semantics.md

[suprak]

So I did something crazy and changed the severity of the analyzer's supported diagnostic, from Hidden to Info.

Now the Compilation action is being triggered but not the Start Compilation action.

[mavasani]

@suprak Can you please give some more information here:

1. Are you seeing this behavior in command line builds OR IDE live analysis? Are you seeing this behavior consistently or only intermittently?
2. Are you running on VS2015 RTM, with your analyzer built against Microsoft.CodeAnalysis 1.0.0 nuget package?
3. Can you please share your analyzer code?

[suprak]

1. I have the issue with IDE live analysis (have not tried command line builds). The behavior is consistent.
2. I am running against VS2015 Enterprise RTM with analyzer built against nuget v1.0.0 of M.CodeAnalysis etc
3. Follows,

I set breakpoints in both methods, and only the Start Compilation one fires.

using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.Diagnostics;
using System.Collections.Immutable;

namespace Test
{
    [DiagnosticAnalyzer(LanguageNames.CSharp)]
    internal class TestAnalyzer : DiagnosticAnalyzer
    {
        private static readonly DiagnosticDescriptor Descriptor = new DiagnosticDescriptor(
            "ID 1000",
            "Description1",
            string.Empty,
            "Analysis",
            DiagnosticSeverity.Hidden,
            true,
            customTags: WellKnownDiagnosticTags.NotConfigurable);

        public TestAnalyzer()
        {
            this.SupportedDiagnostics = ImmutableArray.Create(TestAnalyzer.Descriptor);
        }

        public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics
        {
            get;
        }

        public override void Initialize(AnalysisContext context)
        {
            context.RegisterCompilationStartAction(this.OnStartCompilation);
            context.RegisterCompilationAction(this.OnCompilation);
        }

        private void OnStartCompilation(CompilationStartAnalysisContext context)
        {
        }

        private void OnCompilation(CompilationAnalysisContext context)
        {
        }
    }
}

[paul1956]

I am seeing the same thing on VSIX projects the second time I run a project with any edits between the two runs. The Breakpoint window says the break points will not be hit because no symbols are loaded.

Sent from my iPhone I apologize for any typos Siri might have made. (503) 803-6077

On Aug 5, 2015, at 1:02 AM, Alex notifications@github.com wrote:

I have the issue with IDE live analysis (have not tried command line builds). The behavior is consistent. I am running against VS2015 Enterprise RTM with analyzer built against nuget v1.0.0 of M.CodeAnalysis etc Follows, I set breakpoints in both methods, and only the Start Compilation one fires.

using Microsoft.CodeAnalysis; using Microsoft.CodeAnalysis.Diagnostics; using System.Collections.Immutable;

namespace Test { [DiagnosticAnalyzer(LanguageNames.CSharp)] internal class TestAnalyzer : DiagnosticAnalyzer { private static readonly DiagnosticDescriptor Descriptor = new DiagnosticDescriptor( "ID 1000", "Description1", string.Empty, "Analysis", DiagnosticSeverity.Hidden, true, customTags: WellKnownDiagnosticTags.NotConfigurable);

    public TestAnalyzer()
    {
        this.SupportedDiagnostics = ImmutableArray.Create(TestAnalyzer.Descriptor);
    }

    public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics
    {
        get;
    }

    public override void Initialize(AnalysisContext context)
    {
        context.RegisterCompilationStartAction(this.OnStartCompilation);
        context.RegisterCompilationAction(this.OnCompilation);
    }

    private void OnStartCompilation(CompilationStartAnalysisContext context)
    {
    }

    private void OnCompilation(CompilationAnalysisContext context)
    {
    }
}

} — Reply to this email directly or view it on GitHub.

[suprak]

@paul1956 Yours is a different issue than what I'm reporting here. In your case, what I found you have to do, is do a rebuild (menu Build > Rebuild) in order to have symbols line up on your next run.

[paul1956]

Rebuilt, cleaned and it still seems to not working. This would be a regression from VS2013. This only seems to happen on a VSIX project, other projects work as expected. I have also put Stop into project the first time it is run and it stops, then delete the stop and it still stops there on the next run (like it is running the older version). This happens on a Clean Azure VM, with only the default Code Refactoring (VSIX) project.

I can also confirmed with MS Support that CodeLens does not work on these projects either

https://connect.microsoft.com/VisualStudio/feedback/details/1628295/code-lens-doesnt-work-for-vsix-application-in-vs2015

From: Alex [mailto:notifications@github.com] Sent: Wednesday, August 5, 2015 9:12 AM To: dotnet/roslyn roslyn@noreply.github.com Cc: Paul M Cohen paul.m.cohen@comcast.net Subject: Re: [roslyn] CompilationAction not Triggering (#4068)

@paul1956 https://github.com/paul1956 Yours is a different issue than what I'm reporting here. In your case, what I found you have to do, is do a rebuild (menu Build > Rebuild) in order to have symbols line up on your next run.

— Reply to this email directly or view it on GitHub https://github.com/dotnet/roslyn/issues/4068#issuecomment-128053663 . https://github.com/notifications/beacon/AMMmedkq2XBNxKYJuK_DowwqWPyAgVMbks5oki1CgaJpZM4Fd9O6.gif

[mavasani]

@suprak I got a repro for your issue and am testing a fix. The underlying issue is that your analyzer only reports hidden diagnostics, and the IDE was skipping non-document actions for analyzer which report only hidden diagnostics. Skipping should only be done for document actions on closed documents (we don't want to compute SimplifyTypeName diagnostics on closed files), but not for compilation actions, as they can report these hidden diagnostics on open files.

I'll send a PR with the fix soon. Thanks for reporting this issue!

[mavasani]

@srivatsn can you verify or assign to someone to verify? I did add a unit test while fixing this.

[suprak]

Was this fixed in VS 2015 Update 2? I'm still seeing this issue, especially after applying a Code Fix. My compilation completion action is not being called.

[mavasani]

Tagging @heejaechang - do we now run analyzers reporting only hidden diagnostics when computing project diagnostics?

[heejaechang]

@suprak, for perf reason, we made decision not to support hidden diagnostics only analyzers from compilation end analyzers for live analysis.

hidden diagnostics reported from other actions such as symbol, Ioperation, syntax, semantic will work as long as diagnostic reported is on open file itself as it used to be. just compilation end analysis is disabled.

in future, we might be able to support those once we figure out a way to do it without affecting VS perf too much (such as moving diagnostic part out of proc), but right now, it is too much burden for VS to run all those hidden diagnostic analyzers in proc in VS. (since hidden diagnostics are not visible to users, people are abusing those)

until that happens, workaround will be making your analyzers not compilation end analyzer but other type of analyzer so that we can run it only on opened files (not every files in the solution)

other workaround is making diagnostic not hidden but info. info will not show up as squiggle in user's editor. so it won't distract users as much as warning or error. but also since it shows up in error list, one can't abuse it too much since user will complain if it creates thousands of unnecessary diagnostics.

if you really want it, another heck will be declaring your analyzer to not only report hidden diagnostics but also regular diagnostics but never actually report regular one.

this is heck since this useless regular diagnostic rule will show up in ruleset editor and other places. but it will trick engine to run your analyzers.

but again, the reason we have cut it for live analysis is perf so unless you are really sure it won't affect perf for your analyzer users in VS, I think you shouldn't try the heck.

these kinds of perf issue made us to decide to turn off full analysis by default, so even if it is supported, if perf is bad, people will most likely have full analysis turned off, so such diagnostics won't be reported anyway until user explicitly turn it on.

hope this help.

thank you.

[suprak]

I forgot to add, my diagnostics are no longer hidden. They are now Info. Still there will be times when compilation finished action is not called. ( I don't yet have a consistent repro, but am working on it).

[heejaechang]

@suprak can you try install this vsix. https://www.myget.org/F/roslyn-master-nightly/vsix/eb2680f2-4e63-44a8-adf6-2e667d9f689c-1.3.0.60526.vsix

and see whether it still repro? this is roslyn nightly build.

...

basically, just double click vsix to install and from admin VS developer command prompt do devenv /updateconfiguration

and then run devenv and see whether it repros.

if you want to uninstall the nightly, you go to tools->Extensions and uninstall Roslyn Insider extension and then do devenv /updateconfiguration from same prompt.

this should let you move back to what you have now.

the nightly is our candidate for update 3. so if it still shows problem, please let us know so that we can take a look before update 3 is released.

thank you

[paul1956]

I installed this and #9762 targeted at update 3 is still not fixed, should I expect it to be?

[walnutfalls]

I'm still having this issue:

  [DiagnosticAnalyzer(LanguageNames.CSharp, LanguageNames.VisualBasic)]
    public class CompilationAnalyzer : DiagnosticAnalyzer
    {
        public static readonly DiagnosticDescriptor SupportedDescriptor =
               new DiagnosticDescriptor("ID_1", "DummyTitle", "DummyMessage", "DummyCategory", DiagnosticSeverity.Warning, isEnabledByDefault: true);

        public static readonly DiagnosticDescriptor UnsupportedDescriptor =
            new DiagnosticDescriptor("ID_2", "DummyTitle", "DummyMessage", "DummyCategory", DiagnosticSeverity.Warning, isEnabledByDefault: true);

        public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics
        {
            get
            {
                return ImmutableArray.Create(SupportedDescriptor);
            }
        }

        public override void Initialize(AnalysisContext context)
        {
            context.RegisterCompilationAction(compilationContext => {
                compilationContext.ReportDiagnostic(Microsoft.CodeAnalysis.Diagnostic.Create(UnsupportedDescriptor, Location.None));
            });
        }
    }

[heejaechang]

@concubicycle even after "Full Solution Analysis" is on? from tools->options->C#->Advanced ?

[walnutfalls]

Hey, thanks for the response! I just tried that, still not hitting the breakpoint. I might just be missing something simple...

[heejaechang]

@concubicycle how are you putting this analyzer to VS? vsix, nuget, explicitly adding to project as analyzer reference?

[walnutfalls]

@heejaechang I right clicked the project (A VSIX project), and did Add New Item->Extensibility->Analyzer.

Then I went to the .vsixmanifest file, Added MefComponent and Analyzer.

I also tried to create a project from the Analyzer&CodeFix template, but it doesn't seem to trigger then either.

[heejaechang]

can you try to add the dll directly to a project's analyzer reference to see whether that works?

[walnutfalls]

Sorry, not sure what you mean

[heejaechang]

you can add your analyzer directly

[walnutfalls]

No luck =(

[heejaechang]

something is wrong with your VS.

I just tried your code and it works.

here is code.

I added analyzers directly through Analyzers node in solution explorer.

[heejaechang]

@concubicycle by the way, why are you reporting diagnostic that is not supported? this is not allowed.

[walnutfalls]

@heejaechang I copy pasted code from a unit test I found.. What steps did you take to get this to work? I still can't get it to work. I create a new VISX project, add an analyzer, and add an analyzer asset.

[heejaechang]

I didn't do anything special. I created class library, put your code, add code analysis nuget package, added the dll as analyzer in a project. that's it.

[rikimaru0345]

Does not work for me either. I'm trying

• RegisterCompilationStartAction - works
• RegisterCompilationAction (never called)
• RegisterCompilationEndAction (never called)

Severity is "Warning", full solution analysis is turned on.

I'm debugging the analyzer with visual studio. I started with the example project, then I press F5 to start a new instance of VS.

Diagnostics work, and codefixes work as well. But no matter what I do (rebuild, build, ...) the complication end actions are never called. VisualStudio 2017 Version: 15.2 26430.6

[Pilchie]

@heejaechang Isn't this the issue that CompilationEnd analyzers only run if Full Solution Analysis is enabled in Tools/Options?

Tagging @kuhlenh

[heejaechang]

@rikimaru0345 @Pilchie ya. it should run if FSA is on unless it has design time build error. with FSA on, do you see any error from closed files?

[rikimaru0345]

I have FSA on, so I assume it should run.

What do you mean by "error from closed files" and "design time build error"?

I get no errors in the ErrorList pane, the Debug output, or any other location I could think of that would report errors.

My "primary" visual studio instance would catch exceptions when/if they happen (and it doesn't so far).

My primary vs instance shows the debug output of the second one, and all I see are some trace-messages regarding web requests.

[heejaechang]

best way to see whether you have "design time build error" or not is whether you see any errors from files that are not opened in VS in error list (do not build) with FSA on.

when a user opens a solution/project in VS, it is not like VS can load all those solution or projects. some of them we can't load properly due to various reasons such as msbuild failure, missing dll references, cross language p2p reference issues, missing nuget packages and etc. and we call them "design time build errors". when this happen we turn off FSA internally since otherwise, user will/might get million errors since project is in broken/unknown state.

if you have built once VS is launched, please close VS and re-run VS and see whether you get any errors from closed files to see whether you have design time build errors.

thank you

[kentcb]

This is hella confusing. I arrived here with a similar problem: my RegisterSymbolAction is running, but not my RegisterCompilationEndAction. Moreover, the advice about enabling FSA does not seem relevant anymore because there's no such option in my VS2017 15.4.4 installation:

Can anyone clue me in?

[kentcb]

Here is the analyzer in question: https://github.com/code-heroes-pty-ltd/CodeHeroes.CodeAnalysis.Style/blob/feature/2358/Src/Analyzers/ClassAnalyzer.cs

I cannot for the life of me understand why my RegisterCompilationEndAction is not called. All other actions are called fine. And the unit tests all work fine, too, suggesting the problem is only within the context of a VS session.

[heejaechang]

@kentcb here is the location of the option.

since various analyzer has various perf characteristic, it is not authors decision what kind of analyzers runs, it is users decision. CompilationEndAction analyzer only run if user choose to opt in to use expensive analyzer with the above option

[kentcb]

@heejaechang Thanks for the reply, and enabling that option does indeed allow my compilation end action to trigger (though it's not hitting my breakpoint at the moment, which is strange).

However, I have comments/questions:

1. All the docs I could find point me to the wrong place in the options. Unless I missed it, it would be good to update this.
2. I've been using analyzers for years and had no idea this existed. It seems a really insidious default behavior to adopt. I understand the reasoning, of course, but the experience is very poor right now in my opinion. Ideally, I would think that always enabling full solution analysis is the way to go, with suppressions/rulesets used to disable any analyzers that the user doesn't want running. I'm sure achieving this would be difficult, but it seems a far more intuitive experience to me.
3. Can my analyzer detect whether full solution analysis is enabled/disabled? It seems kinda ironic that my analyzer is doing a whole bunch of work, but that work is useless because my compilation end action won't be called! If I could at least detect that the work is pointless, then my analyzer can just skip running altogether.
4. Further to the above, if my analyzer can detect that full solution analysis is disabled, I assume I can raise a warning diagnostic telling users to enable full solution analysis.

Thanks, Kent

[heejaechang]

tagging @jinujoseph @kuhlenh for @kentcb feedback.

"All the docs I could find point me to the wrong place in the options. Unless I missed it, it would be good to update this.", can you be more specific on "this"?

"I would think that always enabling full solution analysis is the way to go", we had that on by default and got a lot of feedbacks to turn it off. that is why it is off by default now. (main feedbacks being why VS is consuming 100% of CPU even if I left it idle for 5/10/30 mins - you put whatever time span there - ?)

so far what we are seeing is that, users want it to be off by default (I meant pure users who only consume analyzers but never create one), and analyzer authors want it to be on by default. (if not full solution, active projects or some other combinations)

once we made it off by default, only people who said or asked it should be on by default are analyzer authors, no users ever asked us to turn it back on by default.

"Can my analyzer detect whether full solution analysis is enabled/disabled? It seems kinda ironic that my analyzer is doing a whole bunch of work, but that work is useless because my compilation end action won't be called! If I could at least detect that the work is pointless, then my analyzer can just skip running altogether.",

No, author can't do that. analyzer can run not only in IDE but also in other places such as csc, and this option is host specific, meaning only matter to VS. and analyzer is supposed to be host agnostic.

...

I have this issue - https://github.com/dotnet/roslyn/issues/11750. It has been there for at least 1 year since no user ever asked this so low priority. only people who are making compilation end analyzers want it to be on (sure, they want their analyzers to run without users actions and want their analyzers to be discoverable) but so far, we never had a feedback from users to turn it back on since we have made it off by default. not a single feedback.

[CyrusNajmabadi]

but so far, we never had a feedback from users to turn it back on since we have made it off by default. not a single feedback.

That's because users, in general, do not know what options there are. And, more importantly, users can't identify that a feature is not there if it has been turned off by default. Say we added a bunch of refactorings, and then had them off by default. Would we be surprised that we heard almost no feedback from people saying "hey... you should turn these on!" Of course not. Users wouldn't even know the refactorings were there to ask for them to be enabled.

Discoverability is a large issue here. As is comprehension. Even if a user does discover the "Full solution analysis" option, they have no way to actually understand what this option does. While some might correctly surmise that this might provide errors for closed files, I'd wager a bet that almost no users would expect that this would cause analyzer to stop running for the files they did have open.

[heejaechang]

@CyrusNajmabadi but it as on by default for almost 1 year. and then turned off by default. if a refactoring was on by default for 1 year and then removed and if no body ask "hey where that refactoring go?", then that refactoring is probably not used much.

[mavasani]

How about we report a special IDE-only per-project Location.None diagnostic when the IDE driver notices that user has at least one enabled analyzer with a CompilationEndAction, but FSA is turned off? The hyperlink/description for the diagnostic can point to help URL to turn on FSA (as we cannot have a code fix for no-location diagnostic). It should have minimal perf-impact and makes FSA discoverable? @heejaechang would this be easy to implement?

[heejaechang]

I think if we decide to do this, I would just make active project to run compilation end analyzer as we have planned before. it is just getting delayed again and again since there is no big push from users. we do have BIG push from users to not turn FSA on.

[mavasani]

we do have BIG push from users to not turn FSA on

I think the push was only to not have it on by default. For users that are silently affected (projects with manually installed analyzers that have CompilationEndAction that is silently never executed), giving this diagnostic would be valuable and I think we have already got feedback for this multiple times. We have never got to implementing it because we have an easy workaround to share to the users.

[heejaechang]

but that diagnostic gives only option between FSA on or off. and it does more than compilation end analyzer. I think our previous plan is better. running compilation end analyer for active or opened file projects when FSA is off.

[heejaechang]

"I think we have already got feedback for this multiple times", all feedback I was involved in was from author not users. all start from I am implementing xxx, and it never get called.

only FSA related feedback I saw was, I only see errors from open files, which is different aspect of FSA than not running compilation end analyzers.

[mavasani]

running compilation end analyer for active or opened file projects.

However, it will have performance/RPS impact correct? If not, that option would be the best. @sharwell should probably be looped in.

all feedback I was involved in was from author not users. all start from I am implementing xxx, and it never get called

Yes, probably because the users felt the analyzer ran fully on their code, and their code is clean on all analysis and hence never reported issues.

[CyrusNajmabadi]

"I think we have already got feedback for this multiple times", all feedback I was involved in was from author not users. all start from I am implementing xxx, and it never get called.

Right. How are users going to complain if they've never had the features to begin with. And they don't have the features to begin with because authors are blocked from implementing them.

[CyrusNajmabadi]

Basically, imagine if we had 20 actual features using this. And we'd had it on for a year. Would we have seen people notice if it went away? Almost certainly. However, i think we only ever had one feature using this. And it was only on for a very short time (not even a year) before we disabled this again. So how many people would actually know that this had happened?

[heejaechang]

for this kind of thing, we have A/B testing. no reason to argue or speculate. we can start A/B testing on perf impact, usage increase and then decide what to do.

[kentcb]

I appreciate the discourse here, people, and can understand the competing desires.

It seems kinda ironic that my analyzer is doing a whole bunch of work, but that work is useless because my compilation end action won't be called!

This wasn't really addressed. I mean, given that the whole point of disabling FSA was to improve perf, isn't it kinda crazy that it actually results in more wasted work?

[heejaechang]

@kentcb yep. I think we should make compilation end analyzer on opened files should work even if FSA is off. tagging @jinujoseph

...

we thought if one implements Compilation End Analyzer, one would do most of heavy work in compilation end callback since it gets compilation and with compilation, one can do everything he wants. and doing everything there is more efficient than managing states in multiple callbacks which make it stateful and cause all the complexity coming from it such as concurrency, thread safe and etc.