Any plan for using dotnet 5.0 ISourceGenerator for C# target?

[Thaina]

https://devblogs.microsoft.com/dotnet/using-c-source-generators-to-create-an-external-dsl/

As of dotnet 5 there is an official support for source generator system ISourceGenerator which I think would be cleaner method to generate lexer and parser for antlr, include referencing antlr itself into any project

Is it possible and would you like to have official support for it?

[ericvergnaud]

The generation is done by the Tool, which is written in Java, so I don't think this would be possible.

[Thaina]

@ericvergnaud Even then I think it would still possible if we using java tool to generate C# code as a string text. Then inject those text with ISourceGenerator instead of actual file in project. Or something like this, maybe?

[ericvergnaud]

ISourceGenerator is about generating text, adding the file to the project is out of scope.

[sharwell]

I might update Antlr4.Generator in the future to support source generators, but given it already does exactly that today even in old compilers that don't support the new API, I'm not sure what new value it would be adding.

[Thaina]

@sharwell As for me I think it would make our life easier when we want to integrate antlr into our project. We shouldn't need to install any tools or rely on IDE tool system. instead we have C# process the source generator under dotnet pipeline just by reference antlr with nuget

And if I understand it correctly, we also wouldn't really have generated source under project, or at least, can be ignored from repo as it would be regenerated on C# compiling

This would be convenient and could increase adoption of using antlr for DSL in C#

[sharwell]

@Thaina the C# package I created already works the way you describe (starting with the 4.6.5 release of Antlr4.Generator, Java is no longer required for the code generation step). In practice is almost no observable difference by moving to a source generator.

[Thaina]

@sharwell The document I have seen since my first time using antlr is still telling me to install java and all and doesn't mention that I could use dotnet and what process I can do https://github.com/antlr/antlr4/tree/master/runtime/CSharp/src

Follow a link to seeing your tool https://github.com/tunnelvisionlabs/antlr4cs have more elaborate documentation but I am still not sure it was the same tool and could use the same config

Also is this can work with vscode? Currently now I use this tool https://github.com/mike-lischke/vscode-antlr4 because I could get it work on vscode

And if it using ISourceGenerator which is injecting source code at conpile time, would it able to provide intellisense too ?

[sharwell]

Follow a link to seeing your tool https://github.com/tunnelvisionlabs/antlr4cs

That link points to the repository where my fork of the C# target is developed. One of the packages produced from that repository is Antlr4.CodeGenerator, which has two copies of my fork ANTLR 4 tool: one in Java and one in C#. You can add the following to your project file to use the C# generator:

<PropertyGroup>
  <Antlr4UseCSharpGenerator>True</Antlr4UseCSharpGenerator>
</PropertyGroup>

And if it using ISourceGenerator which is injecting source code at conpile time, would it able to provide intellisense too ?

ISourceGenerator can provide IntelliSense, but my Antlr4.CodeGeneration also provides IntelliSense using the older mechanism that tools like XAML have used for many years. There are many subtleties to handle slight differences in IDE and MSBuild versions over time, but it's all implemented in Antlr4.CodeGenerator.targets.

[Thaina]

@sharwell Thank you ver much

In that repo there was a warnin here so it seem like this was outdated?

⚠️ Starting with release 4.5.0-alpha003, users are no longer required to install the Java Runtime in order to compile .NET applications using ANTLR 4. However, installing Java will dramatically improve the performance of the code generation process. It is highly recommended, especially on developer machines where background code generation is used for IntelliSense functionality.

[sharwell]

The IKVM toolchain that led to that performance note was replaced in v4.6.5-beta001 by the new C# toolchain that is as fast as the Java toolchain, if not faster. The only reason it's not enabled by default is I need to do a bunch of binary diffs of the output comparing the Java and C# tool to make sure they always produce exactly the same code.

[Thaina]

@sharwell Thanks again. So did the Antlr4UseCSharpGenerator documented anywhere?

[sharwell]

Currently it's only documented in the release notes where it was introduced (the link in comment https://github.com/antlr/antlr4/issues/3068#issuecomment-774859825 above). The readme should probably be updated to highlight it as the intended replacement for the Java toolchain when generating code for the optimized C# target.