Better support for single file generators

[sayedihashimi]

We have implemented basic support for .t4 and .resx but the implementation is limited and not meeting all the needs. We need to come with a better story there.

Open items

• For .resx how to select internal versus public
• How to enable/disable generators on a per file basis
• How to change the generator used
• Is this going to be a VS specific set of features? It would be better to have support in the dotnet runtime instead of just inside of Visual Studio.

I've heard some rumors that we may get support for .resx files from the runtime. If that happens we should design it such that we can leverage the same support for other generators.

Right now we are busy converting dnx to dotnet so we are not really implementing many features like this. I'd like to see if there is going to be a better story from the runtime before we implement something specific in tooling.

Related issues

• https://github.com/aspnet/Tooling/issues/124
• https://github.com/aspnet/Home/issues/272
• https://github.com/aspnet/Tooling/issues/339

[mcquiggd]

With regards to T4, here is a brief(!) summary of my (often stated) thoughts on this:

In T4, we have a tool that is already cross platform, i.e it has been available for over 5 years on Mono, including integration with Mono Develop, Sharp Develop etc, it can be run (through non-Microsoft efforts) on OSX, Linux, Windows. We have a large number of community projects and frameworks in circulation that use T4. As far as I am aware, the Entity Framework team continue to use T4.

Current strengths of T4:

1. That it can be used to output almost any type of file, from C#, VB.Net, JavaScript, HTML, XAML, SQL, PHP etc. etc. etc.
2. It can work with metadata from a database, a Visual Studio project, CodeDom, Roslyn among others (for example I have used HTMLAgility Pack within T4 to parse HTML, and perform replacement of strings with Razor syntax and constants, located within the same Project, or an external assembly).
3. T4 brings us the power of a language such as C#, along with a scripting (ASP.Net) syntax, to allow complex or simple output, and is executed in compiled form.
4. The ability to import any .Net Framework or custom assembly and make use of that functionality.
5. Importantly, it can be hosted within an IDE, and interact with that IDE (e.g. Env.DTE).

Current weaknesses of T4

1. Lack of official Microsoft support would have to be my first point, which leads to:
2. Poor support in Visual Studio; lack of built in syntax completion, lack of syntax highlighting (especially .ttinclude files), lack of auto run templates, lack of functioning 'Transform All Templates.
3. Lack of inherent multiple file output (and I do realise this topic is regarding Single File Generators).
4. Reliance on third party tools to fill in such gaps, e,g ReSharpers ForTea extension, Tangible T4 editor, T4 Toolbox. I do not believe Clarius T4 editor is available now; a lot of this declining support is due to lack of any official Microsoft story on Code Generation - people are wary of investing time and effort in any Microsoft technology that seems to have 'gone dark'.
5. More recently, with ASP.Net 5 projects, the inability to set properties, such as custom tool, custom tool namespace, ability to run automically on build.
6. No template debugging support in ASP.Net 5 projects.

Possible Approaches

Scrap T4 and come up with another technology

1. Breaks backward compatibility.
2. Will probably take a long time.
3. Might be cleaner architecture?
4. Might open up new feature possibilities (I personally cannot think of any at the moment).

Enhance T4

1. Support Razor syntax.
2. Perhaps rebase it on Roslyn (better Runtime templating support?)
3. Support syntax highlighting and syntax completion (maybe a fair comparison is the Omnisharp project? maybe this is already almost supported due to Omnisharp?, as it is basically C# and Razor).
4. Support it in Visual Studio, and Visual Studio Code, with features such as 'auto run on build', 'build all templates' in project / solution.
5. Distribute it via NuGet, as an extension to the .Net Framework.
6. State that this is the preferred approach for .Net Code Generation to encourage third party and community support.

As I say, just my initial personal thoughts.

In another thread, it was asked if there were other projects to make T4 cross platform; the good news is yes.

There is a project here by a member of the Entity Framework team and updated to RC1 here byTibor @totht91)

That would indicate it would be possible to incorporate T4 into an Asp.Net 5 pipeline via Gulp, and a command, working cross platform in VS Code, or indeed anything compatible with the dnx.

It also supports Razor syntax

Another obvious example: https://github.com/mono/monodevelop/tree/master/main/src/addins/TextTemplating

Which is published as a NuGet, Copyright 2009-2011 Novell, Inc. 2011-2014 Xamarin Inc. https://www.nuget.org/packages/Mono.TextTemplating/

https://github.com/giacomelli/MonoDevelop.ThoseMissingFeatures

[kevinkuszyk]

Some more on template runtime errors here: T4MVC/T4MVC#68.

[michael-x]

We are using a custom tool to create XXX.Designer.cs from XXX.resx files. For resources whose comments contain an appropriate parameter list, the tool creates type-safe methods instead of properties. In order not to have to revert to old behavior, we would need one of the following:

1. Full support for custom tools, letting us choose the old custom tool settings per item.
2. Let us turn off resx file code generation per item and instead use something like T4 to generate the output, including full design time support (e.g. when changing the resx file, the generator needs to run automatically; running at build time is too late as you can then change and commit a resx file without an updated cs file).
3. Let us customize the template of whatever tool you would like to integrate into the runtime, although I've got a hard time thinking why you would want to do something like this at runtime, instead of at design time, which would be even before compile time.

[sayedihashimi]

We will have better support for this in the future. Closing this for now.

[daveaglick]

@sayedihashimi Are any details about forthcoming single file generator support available? Primarily I'd like to know if the general concept of single file generators will end up being supported or will the support focus exclusively on enabling the common generation scenarios like T4. I'd note that there are some (though admittedly not many) libraries in the community that use the single file generator plumbing to do stuff other than T4/resources (I.e., Scripty).

[drauch]

We're also using a SingleFileGenerator custom tool to generate source code on each save of the file. Is there any workaround to run custom tools on a per-file basis for now? If no, we have to step back to the old project format which is a real downer.

@sayedihashimi : Please, some workaround advice :-) And even better: a roadmap on when we can expect our SFG to work again.

[Inspyro]

@drauch @all I actually found out that it already works as expected for .NET Core projects. You just have to add another CodeGeneratorRegistration for the new project type (new guid). For the old projects you can use the constant from VSLangProj80.vsContextGuids.vsContextGuidVCSProject ("{FAE04EC1-301F-11D3-BF4B-00C04F79EFBC}") For the new projects you need to use the constant "{9A19103F-16F7-4668-BE54-9A1E7A4F7556}" (this is the guid for the new project type, which you can see in the .sln files - you need to add the {} as well!)

You can either enter the code generator in the compile item: <Generator>CustomToolName</Generator> or you can use the properties dialog (not page) from Visual Studio - however this is for some reason glitched: a.) in the properties page the CustomTool setting is not avialable anymore b.) in the properties dialog of the file you can find the CustomTool in Advanced - however you need to add a namespace before adding the custom tool and remove it afterwards again (as you get an exception else and the setting is not stored)

@sayedihashimi / dotnet / aspnet team: Why is the CustomTool settting not displayed in Visual Studio 2017 for the new project format? (Only in the properties dialog)

[danielcrabtree]

@Inspyro Thanks for the tip.

Adding a GeneratorRegistration using the new project type constant "{9A19103F-16F7-4668-BE54-9A1E7A4F7556}" worked for me. I can now use single file generators in .NET Core projects in Visual Studio 2017.

[bradphelan]

@danielcrabtree Exactly where and how did you add this GUID?

[bradphelan]

I figured it out. I added the GUID to my generator

[ComVisible(true)]
[Guid(GuidList.GuidI18NReactivetring)]
[ProvideObject(typeof(I18NReactive))]
[CodeGeneratorRegistration(typeof(I18NReactive), "I18N.Reactive", vsContextGuids.vsContextGuidVCSProject, GeneratesDesignTimeSource = true)]
[CodeGeneratorRegistration(typeof(I18NReactive), "I18N.Reactive", "{9A19103F-16F7-4668-BE54-9A1E7A4F7556}", GeneratesDesignTimeSource = true)]
public class I18NReactive : IVsSingleFileGenerator, IObjectWithSite
{

[KZumbusch]

While custom tools can be started in VS 2017 15.8.1 on .NET Core 2.1 projects without adding the second registration for us, we are still having issues to use our custom tool. Our custom tool is used for localization and takes a simple text file and converts it to a C# class and two resource files with English and German string resources.

To generate the code and manage the files in the solution our custom tool implements the IObjectWithSite interface. Unfortunately the SetSite method of that interface is never called if the custom tool is started from a .NET Core project and we are not able to access neither the CodeDomGenerator nor the project's items. Adding the GUID to the registration does not help.

Is there a replacement for IObjectWithSite that works with .NET core projects?

[powermetal63]

We have the same issue as @KZumbusch Even with the latest VS 2017 15.8.3, the custom tool is instantiated, but neither SetSite nor Generate is called for multi target project using the new csproj format (class library, <TargetFrameworks>net461;netcoreapp2.0</TargetFrameworks>). The custom tool works in old style full framework projects inside the same solution. What should we do?

[oising]

Any movement on this, @sayedihashimi ? I think three years counts for "later." ;)