Visual Studio does not reload source generator assemblies when they change on disk

[vatsan-madhavan]

I'm just started writing a source generator, and I'm finding that Visual Studio is caching source generators aggressively, and it's making it very hard to do iterative development.

This is what I'm having to do to make even small changes.

• Close Visual Studio (devenv.exe) instance in which the source generator project and its consumer (i.e., the project that ProjectReference's the source generator)
• pskill /t servicehub.roslyncodeanalysisservice
• del $env:TEMP\VS\AnalyzerAssemblyLoader -Recurse -Force <- This is where the source generator binaries seem to get cached. Both devenv and servicehub.roslyncodeanalysisservice seems to hold handles to files under this location

Am I missing something simple to get the development process working more seamlessly ?

I'm on 5.0.100-rc.2.20473.20 + Visual Studio 2019 Enterprise 16.8.0 Preview 4.0 [30517.14.main

[CyrusNajmabadi]

VS support for source-generators is pre-alpha. I would not expect a good VS experience here for a while. Certainly not prior to the 16.9 timeframe.

Am I missing something simple to get the development process working more seamlessly ?

Nope, you're not missing anything. It will just be a while until SGs are really supported in VS.

[vatsan-madhavan]

Is there a way to view the generated sources except through VS/intellisense ?

[Youssef1313]

@vatsan-madhavan

For the purpose of testing/debugging, I was viewing them with a simple hack (which isn't good, but did the purpose for me).

https://github.com/Youssef1313/PrintMembersGenerator/commit/31e31ce65803f3aa6055aa4f621216931f66c911

Then after things got stable with me, I moved to unit testing to confirm the correctness of the generated sources.

https://github.com/Youssef1313/PrintMembersGenerator/blob/master/src/PrintMembersGeneratorTests/PrintMembersGeneratorTest.cs

[CyrusNajmabadi]

Is there a way to view the generated sources except through VS/intellisense ?

As of yesterday, you can now emit generated files to disk so you can inspect them: https://github.com/dotnet/roslyn/pull/47047

[vatsan-madhavan]

This is awesome! This means I can get by with commandline builds while VS catches up!

[jasonmalinowski]

So the one problem @vatsan-madhavan you're probably running into is once we've loaded your assembly...the CLR doesn't give us sane ways to unload it or load a different version if the assembly version hasn't changed. @chsienki or @cartermp any chance somebody already has some MSBuild magic to work around this in some way?

[jaredpar]

Curious: if the generators in this case are strong name signed could we potentially manipulate the version here and load the new copy?

[jasonmalinowski]

@jaredpar Potentially. As crazy as the feature request is, I almost wish the compiler had a feature where it'd generate a (determinstic) but effectively random version and stuff that into the assembly version. :smile:

[jaredpar]

Honestly this could be done as a simple post-build sttep. Load the binary in memory, use the metadata writer to flip the strong name bit, change the version and then load that vs. the one on disk.

[jasonmalinowski]

@jaredpar Thanks for volunteering!

[jaredpar]

The best part about being a lead, maybe the only good part, is the ability to delegate ... @chsienki

😉

[PathogenDavid]

I almost wish the compiler had a feature where it'd generate a (determinstic) but effectively random version and stuff that into the assembly version. 😄

The module version ID would be good for this. Just kludge it into the version number. (It's been a hot minute since I've looked into it, but if I remember right with determinism enabled it's the SHA1/SHA256 of all the compiler inputs or something along those lines.)

Right now specifying a wildcard for the assembly version results in CS8357 if determinism is enabled. Maybe determinism + wildcard could mean "kludge MVID into version number".

[MisinformedDNA]

Is the Roslyn team not going through the same pains as us?

[jaredpar]

We use source generators inside our main solution hence we are dogfooding the experience every day. The source generators aren't iterated on as frequently though hence we don't hit the specific reload problem.

This is a problem we are taking a look at. It's existed since Roslyn 1.0 with analyzers (so roughly five years now) hence it's not a new problem, generators has just shined a new light on it. There are some ideas on how to work around this (see my comments above). At the moment though Roslyn does most of the evaluation in process and given we are still on .NET Desktop that limits our options a bit because of the inability to load multiple copies of a DLL into the same process space unless it's strong name signed + changes versions on every build.

[tmat]

@PathogenDavid Using hash for version is not viable since it's not monotonic and versions are expected to be.

I'd suggest that instead of running source generator in VS when developing it it's better to run it in a unit test. Write a unit test that runs the generator and produces output. In that setting you can iterate fast - even using Edit and Continue to modify the generator code as you are debugging it. There is no need to mess with versions/reloading/VS complexity etc.

[tmat]

@jasonmalinowski

So the one problem @vatsan-madhavan you're probably running into is once we've loaded your assembly...the CLR doesn't give us sane ways to unload it or load a different version if the assembly version hasn't changed. @chsienki or @cartermp any chance somebody already has some MSBuild magic to work around this in some way?

The magic is called Core CLR ;-).

[PathogenDavid]

Using hash for version is not viable since it's not monotonic and versions are expected to be.

Fair enough for making that the default behavior of *, but for the purposes of this discussion the version number being monotonic doesn't matter.

I'd suggest that instead of running source generator in VS when developing it it's better to run it in a unit test.

This is essentially what I've been doing. (Except running dotnet build outside of Visual Studio with EmitCompilerGeneratedFiles enabled.)

The main frustration for me has been that once I have finished working on my source generator, convincing Visual Studio to relinquish whatever hidden cached source generator it is seemingly impossible to do reliably. At the very least I'd expect restarting Visual Studio to fix things, but even stopping all instances of Visual Studio, killing any lingering service hubs, deleting the cache folder named in the main issue, deleting all folders starting with vs in my temp folder, making a blood sacrifice, deleting my .vs folder, and finally restarting Visual Studio: My old generator still sometimes (somehow) sticks around. (But only sometimes.)

[PathogenDavid]

any chance somebody already has some MSBuild magic to work around this in some way?

I was going to sit on this a little bit longer to make sure it doesn't have problems, but since I'm here and thinking about it...

Edit: This is NOT a good idea anymore. Simply restarting Visual Studio is enough these days.

This solution has not been reliable in the long term, so I no longer recommend it. (Click here if you want to see it anyway.)

I've been using this workaround for about a week without issues. In short: I keep the emitted code around during normal builds. During design-time builds the in-solution source generators are skipped and the previously-emitted code is used instead. This means the IDE experience does not update without manually building, which obviously undermines a lot of the benefits of source generators, but it's a much smoother experience until the Visual Studio issues are resolved. First, enable `EmitCompilerGeneratedFiles`. (I do this in `Directory.Biuld.props` so it applies to my entire solution.) ```xml true ``` Then, in `Directory.Build.targets`: (Should work in the project file too, did not test.) ```xml <_AnalyzerProjectReferencesToRemove Include="@(ProjectReference)" Condition="'%(OutputItemType)' == 'Analyzer'" /> <_GeneratedSourceFileToRemove Include="$(CompilerGeneratedFilesOutputPath)/**/*.cs" /> ``` Since this leverages `EmitCompilerGeneratedFiles`, you do not need to modify your source generator in any way. This fix intentionally only applies to in-solution generators, so NuGet generators are unaffected. (Note the included workaround for https://github.com/dotnet/roslyn/issues/47966 and https://github.com/dotnet/roslyn/issues/49125: It's pretty aggressive, so don't use your `CompilerGeneratedFilesOutputPath` for unrelated files for whatever reason.) Edit: It's also worth pointing out this is slightly over-zealous and ends up removing any in-solution analyzers from design-time builds. You can tweak `_AnalyzerProjectReferencesToRemove` to be less aggressive if that causes you issues.

[palpha]

Pretty aggressive is accurate – trying to implement your fix, @PathogenDavid, all my .cs files on my entire C: drive were deleted. I probably did something wrong, but perhaps you want to do some sort of check on CompilerGeneratedFilesOutputPath in _GeneratedSourceFileToRemove... Now, time to dig through my latest backup and hope for the best.

[PathogenDavid]

@palpha 😱 That's horrifying! Sorry you're having to deal with that, glad to hear you have backups at least. I added a gate to skip that target if CompilerGeneratedFilesOutputPath is missing for whatever reason.

[PathogenDavid]

It might be too late by now, but you might see if Windows File Recovery can help get some of them back.

[palpha]

Yeah, I hadn't installed the May feature update required for the tool, so I'm currently waiting for that update and hoping it doesn't overwrite the deleted data in the file system.

Any important files should have been in a repository and any changes should have been committed, and I should have checked the fix more carefully – it's definitely my fault if I lose anything of value.

Hopefully the end result is just a bit of tedious work and yet another war story of how I've failed, with which I can bore my niece and nephews when I retire.

[thomasclaudiushuber]

Hey friends of .NET, I'm working on a NuGet package (not listed yet), and I have a library and a source generator in that NuGet package. I came across this caching problem when testing the package, and I wonder what the recommendation for NuGet package authors is. Should I add the package version to the .dll file name of the generator .dll to avoid caching?

I mean, if users reference the latest NuGet package version, it's not good that the generator from that NuGet package is not used, but the old generator from the Visual Studio cache.

I'm using Visual Studio 16.9.4, and the issue still exists.

[thomasclaudiushuber]

Ok, my fault, I had AssemblyVersion on the source generator .dll set to 1.0.0.0. I aligned that version with the NuGet package version by using a Directory.Build.props file, now as far as I can see, the issue is gone. This means when the assembly version is set correctly, it works fine for me in Visual Studio 16.9.4. So far, I don't see the caching problems anymore.

Anyway, because of this, I generate the source generator's assembly version into the header of the generated source files. Makes it easier to find out problems. I should have done this from the beginning, would have saved me a few hours of confusion. 🙂

[KhaledSamir]

Any solution to the issue where we have to close and reopen VS instance? @thomasclaudiushuber

[thomasclaudiushuber]

Hi @KhaledSamir , I recommend to use Unit Tests to build the generator. You find an example in this project: https://github.com/thomasclaudiushuber/mvvmgen

[Eli-Black-Work]

@jaredpar,

At the moment though Roslyn does most of the evaluation in process and given we are still on .NET Desktop that limits our options a bit because of the inability to load multiple copies of a DLL into the same process space unless it's strong name signed + changes versions on every build.

Currently, it seems that even when the Source Generator is signed and the version number incremented, Rosyln won't reload the generator unless Visual Studio is reloaded.

What I tried:

1. Create a MyGenerator project with "Package Version" 1.0.0.0 and "Sign the assembly -> Sign the output assembly to give it a strong name" checked.
2. Create an Example project that references MyGenerator.
3. Build Example. MyGenerator is built and runs.
4. Change MyGenerator to generate slightly different output, and bump "Package version" to 1.0.0.1.
5. Build Example again.
6. The old version of MyGenerator is used.

Restarting VS fixes the issue, but I was hoping that strong signing the Source Generator and changing the version would also cause VS to reload the source generator 🙂

[Eli-Black-Work]

I've filed a Visual Studio bug for this, since I suppose that this is technically a VS issue 🙂 Feel free to upvote! https://developercommunity.visualstudio.com/t/changes-to-source-generator-arent-applied-without/1698836

[khaled-saleh-sf]

@Bosch-Eli-Black Thanks for creating it as a bug, I think it would take time till we get this changed. Thankfully, we need to restart VS only while developing but once it's stable, this shouldn't be a problem.

[sharwell]

@Bosch-Eli-Black I believe the ability for Roslyn to distinguish between different versions of an assembly was removed in https://github.com/dotnet/roslyn/pull/56432

[Eli-Black-Work]

Thanks, @sharwell 🙂

Would it make sense for code generators be run OOP (Out Of Process)? It seems like that might solve several issues:

• Code generators could use .NET 6, since they'd be running in a different process from VS.
• VS could easily load new versions of a code generator (Just kill the old code generator process and start a new one).
• Slow code generators wouldn't cause hangs in the UI.

I'm sure that's oversimplifying things, but it sure sounds like a good idea, haha. Not sure if interprocess communication would be too slow, though?

[Eli-Black-Work]

@Bosch-Eli-Black Thanks for creating it as a bug, I think it would take time till we get this changed. Thankfully, we need to restart VS only while developing but once it's stable, this shouldn't be a problem.

@khaled-saleh-sf No problem! 🙂

I think that for our team, the main issue is more that if someone on the team updates the code generator, the other team members won't know that they need to rebuild and then restart VS the next time that they git pull.

[sharwell]

Code generators could use .NET 6, since they'd be running in a different process from VS.

Code generators can still run under MSBuild on .NET Framework, so the restriction to netstandard2.0 wouldn't change from this. Also, even when a process runs separately from VS, it's not specified whether that process is using .NET Core or .NET Framework.

VS could easily load new versions of a code generator (Just kill the old code generator process and start a new one).

This could also be done with AssemblyLoadContext on .NET Core, or by updating the .NET Framework code to allow side-by-side loading of analyzers with different strong names.

Slow code generators wouldn't cause hangs in the UI.

This has been largely resolved already by decoupling source generator execution from the typing loop.

[Eli-Black-Work]

Code generators could use .NET 6, since they'd be running in a different process from VS.

Code generators can still run under MSBuild on .NET Framework, so the restriction to netstandard2.0 wouldn't change from this. Also, even when a process runs separately from VS, it's not specified whether that process is using .NET Core or .NET Framework.

Ah, right 🙂

VS could easily load new versions of a code generator (Just kill the old code generator process and start a new one).

This could also be done with AssemblyLoadContext on .NET Core, or by updating the .NET Framework code to allow side-by-side loading of analyzers with different strong names.

May I go ahead and create an issue for updating the .NET Framework code to allow side-by-side loading of analyzers with different strong names?

[jasonmalinowski]

@Bosch-Eli-Black Yeah, failure to handle higher versions is a newer regression, so we should be able to fix that.

[Eli-Black-Work]

@jasonmalinowski Yay! 🙂 Should I file a new issue for that?

[jasonmalinowski]

@Bosch-Eli-Black I don't think we have a bug on that, so feel free to file!

[Eli-Black-Work]

@jasonmalinowski Thanks! Filed as https://github.com/dotnet/roslyn/issues/60446 🙂

[Phylum123]

Going on 2 years and no resolution? Do you guys care about SourceGenerators? I am confused...

[CyrusNajmabadi]

@Phylum123 We are working on source generators constantly. However, they are extremely complex with many parts of their design requiring an enormous amount of effort across our entire stack. Performance alone has occupied many devs worth of time for years at this point.

Source Generators and IDEs will likely continue to take many more releases to get around to everything that people want.

[cmfrydos]

Isn't this one solved? I am using VS22, and I can iteratively work on my Roslyn based generator without needing to restart VS every time. I only must remember to do a full rebuild every time, and I am fine.

Only once I ran into a problem where two versions of the generator were executed:

• VS found a bug in the newest generated code (showing there was a compile error in a .g.cs). Sadly you cannot click on those errors or investigate the generated files directly (before runtime). That's why:
• I was caching my generated output on the same time to a different directory to be able to investigate my generated code. It was strange, since here I clearly found the code of my old generator version, which did not match up with the VS errors.
• Changing this separate output directory inside my generator.cs produced the new code in the new directory hurray!

So, what was going on: At some point both generators were run: First the new one (which VS used to generate the code for my other project), then the old one, which only overwrote my files in the separate folder. Restarting VS fixed this strange behavior though.

You see, I am an absolute beginner with Roslyn Analyzer, but it seems to me that this restarting issue mostly does not exist anymore? Or am I getting it wrong?

---

Microsoft Visual Studio Community 2022 (64-bit) - Current Version 17.1.6 Microsoft.CodeAnalysis.CSharp Version 4.0.1 Microsoft.CodeAnalysis.Analyzers Version 3.3.3

[weltkante]

@cmfrydos I just tested this again, using this old blog post to write an example. I refer to the source generator via ProjectReference and I have to restart VS after every change I make to how the source is generated. As far as I can tell nothing changed. (VS 17.3.0 Preview 1)

[cmfrydos]

Mhh, this is strange. For me this example is working without needing to restart VS. I only changed a little bit, making the generated class and function partial, such that I can call them without using reflection.

small video proof 😉:

[PathogenDavid]

@cmfrydos The issue has never been with building/running, it's been with the IDE experience. If you make changes to the generator which affects things outside of generated code, those changes won't be reflected in the IDE. (EG: If you change the generator to emit add a new method, that method won't appear in Intellisense until you restart Visual Studio.)

[cmfrydos]

@PathogenDavid Ok, I see. So, I totally misunderstood the issue, sorry. For me this is not really a bug though. If you want to generate code only once, and then use it as part of your codebase, you can simply copy that generated code into your project, and IntelliJ will find it.

But since I want the generator to freshly generate the code on every build, I see the generated files as part of the build, but not as part of my codebase.

But I see why you would want the IDE to find and manage the generated code like it was manually written. Maybe some sort of generator would be nice, which is not called during build, but every time you change and save some code. The output files would then be directly written into your source folder. This would also allow the generator to then inspect the generated files, and even generate more stuff out of that (if you don't run into an endless recursion ;D )

[canton7]

Maybe some sort of generator would be nice, which is not called during build, but every time you change and save some code.

This is how SGs currently work, no?

[cmfrydos]

Mhh I thought it works like illustrated in the blog post, welkante mentioned:

[canton7]

Yeah, the compilation is generated on every (or almost every?) change. That's how intellisense can suggest things based on code you've written but not compiled.

[cmfrydos]

Oh, you're right. I didn't notice that before. But if I change, save (and compile) the generator (not manually building it), it will not 'replace' the generator used in the compilation of the other project.

[cmfrydos]

Actually, even after a rebuild IntelliJ (compilation on save) still uses the old generator.. This also explains the bug I mentioned above, having an old and a new generator at the same time (the old one triggered by saving my file, the new one by calling Build ). If IntelliJ would use the new generator too, maybe there is even a way to generate only code for the files IntelliSense just compiled, and not always for every file / class in my codebase, which would help with performance a bit.