[Questions] Several questions about source generators

[QuantumDeveloper]

Sorry, but I didnt find any other suitable place for asking this questions. I have several question, which I cannot find answer for this time:

1. Is there a way to subscribe to AdditionalFiles changes? I mean can I detect that text in any of additional file was changed and automatically invoke generator to update sources?
2. Is there a way to access AdditionalFiles from IncrementSourceGenerator? I didnt find such property there.
3. Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

[Eli-Black-Work]

I suspect that you may have already found the answers to these questions, but in case not:

1. If your generator implements ISourceGenerator, I believe that ISourceGenerator::Execute() will be called every time the contents of one of the files in AdditionalFiles changes.

2. Access AdditionalFiles from IIncrementalGenerator like so:

   
   public void Initialize(IncrementalGeneratorInitializationContext initializationContent)
   {
   // Some of this code is from https://github.com/dotnet/roslyn/blob/main/docs/features/incremental-generators.md
   var files = initializationContent.AdditionalTextsProvider.Where(/*static*/ file => file.Path.EndsWith(".json"));
   
   var namesAndContents = files.Select((file, cancellationToken) => (Name: Path.GetFileNameWithoutExtension(file.Path), Content: file.GetText(cancellationToken).ToString(), Path: file.Path));
   
   initializationContent.RegisterSourceOutput(namesAndContents, AddSource);
   }

private void AddSource(SourceProductionContext context, (string Name, string Content, string Path) file) { string fileName = $"{file.Name}.g.cs";

try
{
    if (file.Content == null)
        throw new Exception("Failed to read file \"" + file.Path + "\"");

    string sourceCode = .....

    context.AddSource(fileName, sourceCode);
}
catch (Exception e)
{
   string errorMessage = $"Error: {e.Message}\n\nStrack trace: {e.StackTrace}";

   context.AddSource(fileName, sourceCode);
}

}

3. Let me know if you still need answer to this question 🙂

[QuantumDeveloper]

@Bosch-Eli-Black Thanks for your answers. I suspect, that answer for my 3rd question is 'Yes', but just to be sure would like to know your answer :)

And one more question appears when I start working with incremental generator:

Does updates in Additional files will be immediately reflected in AdditionalTextsProvider or I need to do some extra job for this? I mean if I will add/remove some files or update files content.

What I want to achieve is when content one of additional files is changed or if file was added, I want my generator to run and update sources correspondinly. Currently I can update source only after the whole build, which is not what I want, unfortunately

[Eli-Black-Work]

@QuantumDeveloper No problem! 🙂

We're actually trying to accomplish the exact same thing as you, I think 🙂 We have some .json files that we generate C# from, and we'd like for that to happen whenever one of the .json files changes.

In our project, we specify AdditionalFiles like so:

<AdditionalFiles Include="$(ProjectDir)examplesubfolder\*.json" />

• If I change one of the examplesubfolder\*.json files, the source generator automatically runs and generates new C# code. I don't need to rebuild the project.
• Similarly, if I put a new .json file in examplesubfolder\, the source generator will automatically run and generate C# code for the new file. I don't need to rebuild the project. I do need to wait about 10 seconds while all of the analyzers and the generator in my project run.
• If I delete a .json file in examplesubfolder\, I have to restart Visual Studio before the generated C# code will go away.

Note that we're using ISourceGenerator instead of incremental generators, so this may be different for your project 🙂

In case it helps, here's what our project files look like:

MyProject.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">

    <PropertyGroup>
        <TargetFramework>net6.0</TargetFramework>
    </PropertyGroup>

    <ItemGroup>
        <ProjectReference Include="..\MyGenerator\MyGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" SetTargetFramework="TargetFramework=netstandard2.0" />
    </ItemGroup>

    <ItemGroup>
        <AdditionalFiles Include="$(ProjectDir)examplesubfolder\*.json" />
    </ItemGroup>

    <PropertyGroup>
        <Example_Variable>in_case_you_need_to_pass_variables_to_your_generator</Example_Variable>
    </PropertyGroup>
    <ItemGroup>
        <CompilerVisibleProperty Include="Example_Variable" />
    </ItemGroup>
</Project>

MyGenerator.csproj

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>netstandard2.0</TargetFramework>
        <LangVersion>8.0</LangVersion>
        <!-- Setting `IsRoslynComponent` to `true` allows us to debug the source generator.
             See https://dominikjeske.github.io/source-generators/#comment-5804484397 for debugging instructions. -->
        <IsRoslynComponent>true</IsRoslynComponent>
    </PropertyGroup>
</Project>

[Eli-Black-Work]

I've filed an issue for the bug where deleting an input file doesn't cause the analyzer to be run again: https://github.com/dotnet/roslyn/issues/60455

[Eli-Black-Work]

Ah, I forgot to reply to your original question! 🙂

3. Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

Do you mean that you have ProjectA that depends on ProjectB, and you want both projects to use MyGenerator?

[QuantumDeveloper]

Ah, I forgot to reply to your original question! 🙂

3. Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

Do you mean that you have ProjectA that depends on ProjectB, and you want both projects to use MyGenerator?

No, I mean that I have ProjectA that references ProjectB. I attach Generator only to ProjectA and generate sources for ProjectB and ProjectA as ProjectB is referenced from ProjectA

[QuantumDeveloper]

@QuantumDeveloper No problem! 🙂

We're actually trying to accomplish the exact same thing as you, I think 🙂 We have some .json files that we generate C# from, and we'd like for that to happen whenever one of the .json files changes.

In our project, we specify AdditionalFiles like so:

<AdditionalFiles Include="$(ProjectDir)examplesubfolder\*.json" />

• If I change one of the examplesubfolder\*.json files, the source generator automatically runs and generates new C# code. I don't need to rebuild the project.
• Similarly, if I put a new .json file in examplesubfolder\, the source generator will automatically run and generate C# code for the new file. I don't need to rebuild the project. I do need to wait about 10 seconds while all of the analyzers and the generator in my project run.
• If I delete a .json file in examplesubfolder\, I have to restart Visual Studio before the generated C# code will go away.

Note that we're using ISourceGenerator instead of incremental generators, so this may be different for your project 🙂

In case it helps, here's what our project files look like:

MyProject.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
      <TargetFramework>net6.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
      <ProjectReference Include="..\MyGenerator\MyGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" SetTargetFramework="TargetFramework=netstandard2.0" />
  </ItemGroup>

  <ItemGroup>
      <AdditionalFiles Include="$(ProjectDir)examplesubfolder\*.json" />
  </ItemGroup>

  <PropertyGroup>
      <Example_Variable>in_case_you_need_to_pass_variables_to_your_generator</Example_Variable>
  </PropertyGroup>
  <ItemGroup>
      <CompilerVisibleProperty Include="Example_Variable" />
  </ItemGroup>
</Project>

MyGenerator.csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
      <TargetFramework>netstandard2.0</TargetFramework>
      <LangVersion>8.0</LangVersion>
      <!-- Setting `IsRoslynComponent` to `true` allows us to debug the source generator.
           See https://dominikjeske.github.io/source-generators/#comment-5804484397 for debugging instructions. -->
      <IsRoslynComponent>true</IsRoslynComponent>
  </PropertyGroup>
</Project>

I tried to replicate described behavior in my demo project. I even use SourceGenerator instead Incremental and it still does not work as I expect. It creates sources only if I rebuild main project and not when I change Additional files. I will attach my small project. Maybe you can find want I am doing wrong here. CodeGenerationDemo.zip

[Eli-Black-Work]

@QuantumDeveloper Sorry, forgot to reply!

I downloaded your project and gave it a try, and it seems to work on my computer 🙂

Here's what I did:

1. Opened MainWindow.xml, changed the contents from <Window59></Window59> to <Window59>asdfasdf</Window59>, and saved the file. The auto-generated code immediately changed to be this:

   //Auto-generated code
   namespace CodeGenerationDemo.Window;
   public partial class MainWindow
   {
   public string text = "<Window59></Window59>";
   }

2. Copied and pasted the file MainWindow.xml, so that I got a file named MainWindow - Copy.xml. I immediately saw a new file named MainWindow - Copy.g.cs be created by the generator.

Is that the same as what you're seeing? If not, I'm running Visual Studio 17.2.0 Preview 2.1; maybe something got fixed in the newer preview versions 🙂

[Eli-Black-Work]

Ah, I forgot to reply to your original question! 🙂

3. Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

Do you mean that you have ProjectA that depends on ProjectB, and you want both projects to use MyGenerator?

No, I mean that I have ProjectA that references ProjectB. I attach Generator only to ProjectA and generate sources for ProjectB and ProjectA as ProjectB is referenced from ProjectA

If ProjectA references ProjectB, then I believe you'll want to generate your sources in ProjectB. That way, they're visible to both ProjectA and ProjectB 🙂

[QuantumDeveloper]

Ah, I forgot to reply to your original question! 🙂

3. Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

Do you mean that you have ProjectA that depends on ProjectB, and you want both projects to use MyGenerator?

No, I mean that I have ProjectA that references ProjectB. I attach Generator only to ProjectA and generate sources for ProjectB and ProjectA as ProjectB is referenced from ProjectA

If ProjectA references ProjectB, then I believe you'll want to generate your sources in ProjectB. That way, they're visible to both ProjectA and ProjectB 🙂

Ok, I was talking just about that :)

[QuantumDeveloper]

@QuantumDeveloper Sorry, forgot to reply!

I downloaded your project and gave it a try, and it seems to work on my computer 🙂

Here's what I did:

1. Opened MainWindow.xml, changed the contents from <Window59></Window59> to <Window59>asdfasdf</Window59>, and saved the file. The auto-generated code immediately changed to be this:

//Auto-generated code
namespace CodeGenerationDemo.Window;
public partial class MainWindow
{
public string text = "<Window59></Window59>";
}

2. Copied and pasted the file MainWindow.xml, so that I got a file named MainWindow - Copy.xml. I immediately saw a new file named MainWindow - Copy.g.cs be created by the generator.

Is that the same as what you're seeing? If not, I'm running Visual Studio 17.2.0 Preview 2.1; maybe something got fixed in the newer preview versions 🙂

No, thats the issue. You must see <Window59>asdfasdf</Window59> instead of <Window59></Window59>. And I am sure you will see it after rebuild, but not in runtime unfortunately. This issue both in Visual Studio and in Rider.

[Eli-Black-Work]

@QuantumDeveloper Whoops, sorry, I mistyped!

Here's what I did:

1. Opened MainWindow.xml in Visual Studio.
2. Changed the contents to <Window59>testing</Window59>
3. MainWindow.g.cs immediately changed to be this:

   //Auto-generated code
   namespace CodeGenerationDemo.Window;
   public partial class MainWindow
   {
   public string text = "<Window59>testing</Window59>";
   }

I didn't need to rebuild; as soon as I changed the .xml file, it updated the .g.cs file 🙂

Which version of Visual Studio are you using?

[QuantumDeveloper]

@Bosch-Eli-Black I am using version 17.1.3

[Eli-Black-Work]

@QuantumDeveloper If you have a chance, you could try with 17.2.0 Preview 3.0, which is what I'm using 🙂

[QuantumDeveloper]

@Bosch-Eli-Black I installed latest preview and I dont see much difference. Still no auto generation after I change file content and I need to build project each time I want to generate updated sources. First screen show no update when I changed file: Second scrren is after I press "build" option:

I have no idea why your version of VS is working fine and mine is not :(

[Eli-Black-Work]

@QuantumDeveloper Sorry for the late reply! Just got back from vacation 🙂

Here's a GIF of what I'm seeing:

Is the behaviour that you're seeing different, or are we doing different things?

[Eli-Black-Work]

P.S.

I used Screen to Gif for this 🙂

[QuantumDeveloper]

@Bosch-Eli-Black Sorry for long delay. When you showed me where to find generated code, I realized that my generator also regenerates code just after I change my xml, but it does not update cs file on disk until I build assembly. This is good news.

Bad news is that seems generators have issues with referencing another projects. In my real project I have local reference inside my generator via <ProjectReference>. I found that I cannot simply reference it, because in that case generator does not generate anything. After some googling, I found https://stackoverflow.com/questions/69764185/c-sharp-source-generator-not-including-results-from-project-reference and when I add OutputItemType="Analyzer" to my refenece project, It starts working, BUT after some testing found out that at some random point it just stopps working. After 10-20 builds it just stop emitting output. but at the same time it continue updating cs file in runtime (via Solution Explorer).

Then I decided to attach all code from the refeneced project to generator and seems that it works perfectly. At least I didnt faced with described issue. But I would like to make it work with <ProjectReference> because it`s quite stupid to copy all code from other project to make generator work as expected.

Do you know how to workaround such issue?

[Eli-Black-Work]

@QuantumDeveloper Np! 🙂

Does this help with the dependency issue: https://github.com/dotnet/roslyn/discussions/47517#discussioncomment-64145 ?

[QuantumDeveloper]

@Bosch-Eli-Black didnt try that. I am wondering does local project can be referenced via <PackageReference> instead of <ProjectReference>? <PackageReference> its about nugets, not local projects

[Eli-Black-Work]

@QuantumDeveloper I would read through the replies to Sharwell's comment; there's some good information there about how to link to NuGet packages from an analyzer, how to link to transient NuGet packages from an analyzer, and how to link to other projects from within an analyzer.

I think that specifically what you're looking for is this comment: https://github.com/dotnet/roslyn/discussions/47517#discussioncomment-580567 🙂

[QuantumDeveloper]

@Bosch-Eli-Black After whole day of testing seems one of comments helped. My generator now working quite stable. I am talking about this comment: https://github.com/dotnet/roslyn/discussions/47517#discussioncomment-1355614

But anyway I am still thinking that such tricky and not obvious way of linking local project to generator is a bad design and should be improved and simplified.

[Eli-Black-Work]

@QuantumDeveloper Ya, I've been wishing for Microsoft to simplify this, too 🙂

[QuantumDeveloper]

@Bosch-Eli-Black Hi. Working recently on new generator I found that it does not want to work with multitarget assemblies. If I remove multitargeting and leave only netstandard2.0, it starts working correctly, but I need multitargeting for my assemblies.

Is there a way to make generator work with such assemblies? Did you faced with such issue?

[Eli-Black-Work]

@QuantumDeveloper I'm afraid I don't have any experience there 🙂 I'd recommend filing a new bug.