[DISCUSS] Update code generation to use Roslyn over Reflection

[jeffcampbellmakesgames]

Is your feature request related to a problem? Please describe.

Several folks have asked me recently about why EntitasRedux uses reflection-based code generation versus Roslyn like the Entitas Asset Store version does. In short:

• I have more familiarity with reflection over Roslyn, particularly for code-generation and this made forking Entitas originally easier to do.
• The open-source version of Entitas is reflection-based and it would have been a significant effort undertaken to refactor this to be Roslyn based at the same time as the forking effort.
• Previously I had written a Roslyn-based code generator and was unhappy with the amount of effort it took to construct code using Roslyn syntax, though analysis takes significantly less. In hindsight, I would have been better off using it for code-analysis only and used more traditional text templating for the code-generation itself.

Reflection-based code analysis for data providers has a few disadvantages related to Roslyn-based code-analysis.

• Can only inspect code in compiled assemblies; compiler errors will prevent discovery of new code changes.
• Requires extra work to discover members/objects based on access modifiers, i.e. public versus private.

Roslyn on the other hand has disadvantages of it's own for Unity, though several have become less relevant over time as Unity has changed.

• Generally folks are less familiar with Roslyn versus reflection, smaller knowledgeable userbase
• The ability to use Roslyn in Unity directly has been hampered with the older compiler (mcs) and 3.5 Net version. This has been largely mitigated as Unity has made several upgrades and now in 2019 and later the default compiler is Roslyn and the .Net version is 4.6 equivalent.
• Requires a bit more complex setup to be able to discover all csharp source files, add references to assemblies, in order to inspect code. It also can't as far as I know inspect code directly in assemblies as its parsing ability is limited to source files.

Describe the solution you'd like I'd like for EntitasRedux to be able to gain the same or similar features as its Roslyn-powered equivalent, and potentially share some of those same capabilities with Genesis as well so that other developers can use Roslyn-based code analysis easily in Unity for other purposes unrelated to EntitasRedux.

I'd like to hear more from others with regards to their experience using the open-source reflection-based Entitas or EntitasRedux frameworks and how that contrasts with their experience using the Roslyn-based closed-source Entitas. In general, I'd also like to hear more about developers experience with Roslyn and any other pro/cons between reflection and Roslyn that I don't have listed above.

[fahall]

For me, the biggest down side to reflection based code generation is that automated refactoring can be difficult. I end up spending orders of magnitude more time fiddling with generated code and getting things to compile when using Reflection. This could just be because I don't have a good process.

Lets say I want to change the name of a component.

My ideal workflow would be:

1. ctrl-shift-r (rename refactor hotkey)
2. Type in new name
3. All generated code updates to reflect new name (including functions like ReplaceComponentName.

I don't get this with either Reflection or Roslyn, but with Roslyn it's not much worse:

1. ctrl-shift-r (rename refactor hotkey)
2. Type in new name
3. Run Roslyn generator
4. Jump to new compile errors and use AutoComplete to find the new function names.

With reflection, my experience was closer to:

1. ctrl-shift-r (rename refactor hotkey)
2. Oh crap, it won't compile
3. Hunt for all the references to that component in systems and generated code
4. Update generated code until we compile again.
5. Run generator
6. Uncomment systems
7. Repeat (because I probably messed up somewhere).

With Roslyn, if my generated code gets in a weird state, I can always just delete it and regenerate. I'd much rather kill and regenerate the code than try to fix it.

[JesseTG]

Roslyn alone can't scan compiled assemblies, but Microsoft provides Roslyn-backed assemblies that can. You'll want to use Microsoft.CodeAnalysis.Workspaces.MSBuild, just as vanilla Entitas' codegen does. Make sure to scan the whole solution, not just a single project.

[jeffcampbellmakesgames]

Roslyn alone can't scan compiled assemblies, but Microsoft provides Roslyn-backed assemblies that can. You'll want to use Microsoft.CodeAnalysis.Workspaces.MSBuild, just as vanilla Entitas' codegen does. Make sure to scan the whole solution, not just a single project.

I'm not sure if this is an option if this code-gen process takes place within Unity though versus an external app. Based on the nuget package here, MSBuild is limited to .Net 4.7 and higher. When I had looked at the unofficial Roslyn package from Unity and elsewhere, this was one of the assemblies that wasn't included.

Even still, does the MSBuild workspace enable you to search through the source of those already-compiled assemblies the same way as if you were inspecting a C# source file (Syntax trees and the like) or does it simply add additional type/reference information a developer wouldn't have if they weren't loaded? I had thought it was the latter.

[JesseTG]

I'm not sure if this is an option if this code-gen process takes place within Unity though versus an external app.

I've been writing my codegen plugins (for vanilla Jenny, not for Genesis) with external use in mind.

Based on the nuget package here, MSBuild is limited to .Net 4.7 and higher.

I've been using custom Jenny plugins on Linux, where .NET Framework 4.7 is not available. I've been using the .NET DLLs (and .NET executable for Jenny itself) that are included with the asset store package. However, I don't know enough about .NET or Mono to be able to tell you why this works. As far as I'm concerned, it's extremely fussy black magic. This does suggest that it's possible to put together a special build in a UPM package, though. It would definitely be easier to use than what I'm doing now.

Even still, does the MSBuild workspace enable you to search through the source of those already-compiled assemblies the same way as if you were inspecting a C# source file (Syntax trees and the like) or does it simply add additional type/reference information a developer wouldn't have if they weren't loaded? I had thought it was the latter.

Definitely the latter. Is there a reason you'd need the former in this case?

[jeffcampbellmakesgames]

I'm not sure if this is an option if this code-gen process takes place within Unity though versus an external app.

I've been writing my codegen plugins (for vanilla Jenny, not for Genesis) with external use in mind.

Based on the nuget package here, MSBuild is limited to .Net 4.7 and higher.

I've been using custom Jenny plugins on Linux, where .NET Framework 4.7 is not available. I've been using the .NET DLLs (and .NET executable for Jenny itself) that are included with the asset store package. However, I don't know enough about .NET or Mono to be able to tell you why this works. As far as I'm concerned, it's extremely fussy black magic. This does suggest that it's possible to put together a special build in a UPM package, though. It would definitely be easier to use than what I'm doing now.

Jenny might be a .Net Core app, which means it can run on Windows, Linux, and Mac. There is some amount of interoperability between the two frameworks, which means that a .Net core app can do things like load .Net Standard assemblies and use types from it as long as it can resolve all of the assembly dependencies those types might be using.

This article here has some good info around this topic, particularly in that it uses a plugin architecture where plugins are loaded from .Net Standard assemblies into a .Net core app.

Even still, does the MSBuild workspace enable you to search through the source of those already-compiled assemblies the same way as if you were inspecting a C# source file (Syntax trees and the like) or does it simply add additional type/reference information a developer wouldn't have if they weren't loaded? I had thought it was the latter.

Definitely the latter. Is there a reason you'd need the former in this case?

The one nice bit reflection has is that it can inspect types in already-compiled assemblies whereas Roslyn code-analysis depends on the source files being present. This would lead me to believe that all of the code that would result in code generation (that would be inspected using code-analysis) like components for example would need to be present as C# source files for Roslyn to be able to inspect. This is different from code-gen plugins which would only be in loaded assemblies.

The overall result of what I'm trying to say is that I wouldn't necessarily consider getting access to a MSBuild workspace as being required for EntitasRedux to use this approach, although it would be simpler. Being able to directly load the solution into a Roslyn workspace is great and saves a fair amount of work, but it is possible to approximate similar results using an AdHocWorkspace. It just takes a bit more to find and load all of the source files, assemblies and the resultant workspace doesn't have to resemble something that would compile into a DLL or executable.

I had a fair amount of success with that here where I loaded every C# source file and added all loaded assemblies into a single project and in 99% of cases was able to get the basic type information we'd want (in addition to all of the syntax trees to search).

[fahall]

I'm not sure if this is an option if this code-gen process takes place within Unity though versus an external app.

To clarify: is the goal that the code gen run within Unity or be runnable from a Unity menu/hotkey?

If it’s the latter, I think that there are ways to run external applications from within Unity.

[JesseTG]

Personally, I'd go for supporting both, just as Jenny does. There's ups and downs to each. When running codegen in the editor:

• You have access to the AssetDatabase, so your code generators can use information derived from your assets and project configuration.
• You can't run the codegen while code is being compiled or the domain is being reloaded, so this increases iteration time.
• You're confined to Unity's environment, so it's hard to use NuGet packages or unsupported .NET APIs
• It's difficult to cache state between codegen runs, as assembly/domain reloads throws out all static members and it's difficult to serialize Roslyn/MSBuild's internal caches to ScriptableObjects
• It's easier to set up, as you can provide custom codegen plugins through Unity packages. You probably don't even have to compile them in advance, codegen plugins are discovered through reflection as a matter of course.

But when running codegen in an external process (even if launched through Unity):

• You can easily reuse Roslyn's parsed state between codegen runs; just keep the process open and have it communicate with the Unity editor through a local socket.
• You can use any APIs or plugins you want, whether they come from UPM, NuGet, or somewhere else.
• You're responsible for installing and referencing custom plugins, and for making sure the rest of your team can do so.
• You don't have access to the AssetDatabase.
• You can invoke code generation whenever you want, even if Unity isn't actually running.

[jeffcampbellmakesgames]

I'm not sure if this is an option if this code-gen process takes place within Unity though versus an external app.

To clarify: is the goal that the code gen run within Unity or be runnable from a Unity menu/hotkey?

The over-arching problem from my perspective is that discovering code via reflection has several disadvantages versus roslyn, in particular that it has to be compiled beforehand. There are levels of solutions to this problem, some more complex and time-consuming than others and I'd like to enumerate these just so we're all on the same page for the level of effort.

At minimum, the goal would be to convert all data providers for EntitasRedux (the main mechanisms for how app-specific code is discovered and which serves as the source for code-generation) to use Roslyn-code analysis versus reflection. This would likely require some tweaks to the code-generators for any reflection bits that it might rely on from data-provider output, but largely refactoring/supplementing the data-providers would be the bulk of the work. This would render the bulk of the value from using Roslyn code analysis to be able to regenerate code, even if it doesn't compile.

If it’s the latter, I think that there are ways to run external applications from within Unity.

Definitely! Easy to call CLI or terminal commands from Unity via creating a process with the appropriate arguments. More on the benefits of an external app below.

But when running codegen in an external process (even if launched through Unity):

• You can easily reuse Roslyn's parsed state between codegen runs; just keep the process open and have it communicate with the Unity editor through a local socket.
• You can use any APIs or plugins you want, whether they come from UPM, NuGet, or somewhere else.
• You're responsible for installing and referencing custom plugins, and for making sure the rest of your team can do so.
• You don't have access to the AssetDatabase.
• You can invoke code generation whenever you want, even if Unity isn't actually running.

Another goal which could also help to solve this solution, but also add more complexity and time is to create an external application which Unity would call in order to facilitate code-generation. This would likely begin as a v2 Genesis that would run as a CLI app first, using almost all of the existing code with some slight refactoring. After that point, a v2 EntitasRedux would start that would convert all the plugins to be compatible with any changes in v2 Genesis, then do the above described work of rewriting the data providers. This adds some of the benefits described above, but I don't think is a critical path to being able to use Roslyn code-analysis for the data providers.

A third-goal on top of that might be to make that same CLI app a server so that it could be constantly running and increase performance on the code-generation itself. This isn't personally a goal at the moment as anecdotally the biggest cost for time in writing code in Unity isn't the code-generation itself, but the time Unity takes to compile and reload the app domain as a result of the newer changes. I could also see accomplishing the goal of modifying Genesis to run as an external app as an intermediary step that could be used to more easily make it run as a continuous server/process.

Just to put it out there, I'm not opposed to the second path where Genesis becomes an external app; it does have some advantages @JesseTG outlined for opening up a lot more flexibility in what libraries are available, decoupling the code-gen from Unity, but I'm not certain that its required to be able to adapt EntitasRedux to use Roslyn code analysis.

What are your thoughts around these descriptions?

[JesseTG]

Just to put it out there, I'm not opposed to the second path where Genesis becomes an external app; it does have some advantages @JesseTG outlined for opening up a lot more flexibility in what libraries are available, decoupling the code-gen from Unity, but I'm not certain that its required to be able to adapt EntitasRedux to use Roslyn code analysis.

Oh, it's not a requirement at all. Roslyn-powered codegen and Unity decoupling are two orthogonal goals; both can be developed independently. Roslyn and its accompanying tools already don't know or care about Unity. If you minimize your use of UnityEngine or UnityEditor APIs, that will simplify a command-line port later. In fact, ticking the "No Engine References" box on the appropriate assembly definitions will go a long way.

Just note that if you use Unity's undocumented Roslyn package, any Roslyn or MSBuild assemblies you need that it doesn't already provide would have to be carefully selected and packaged, or possibly even compiled specially.

[fahall]

Just to put it out there, I'm not opposed to the second path where Genesis becomes an external app; it does have some advantages @JesseTG outlined for opening up a lot more flexibility in what libraries are available, decoupling the code-gen from Unity, but I'm not certain that its required to be able to adapt EntitasRedux to use Roslyn code analysis

Required or not, my tendency is to avoid coupling to Unity unless there is a very strong reason to do so. Having genesis as an external app (with whatever is necessary to make it callable from Unity) seems a big improvement both in terms of workflow and available tooling as outlined above.

That said, I think it would need to be a .NET Core or Standard app to support developers across platforms, so there may be a significant development cost there(?)

I’m not clear on the use case for having access to the AssetDatabase during codegen. @JesseTG do you have an example?

I recall there being some discussion (here or in Discord) re it being easier to do code gen (e.g. custom plugins) with reflection over Roslyn and other devs being more familiar with it. I submit that MS is pushing Roslyn as the go-to solution for these problems, therefore it’s reasonable to expect future devs (e.g. future us) to be more and more comfortable with Roslyn.

TL;DR, I support the plan outlined for Genesis v2 followed by Entitas-Redux v2.

[JesseTG]

That said, I think it would need to be a .NET Core or Standard app to support developers across platforms, so there may be a significant development cost there(?)

Microsoft is unifying .NET around .NET Core. In fact, they're dropping all suffixes and just calling it .NET. As far as I'm concerned, .NET Standard and Framework are legacy APIs. So anything you do is gonna have to support .NET Core whether you like it or not.

I’m not clear on the use case for having access to the AssetDatabase during codegen. @JesseTG do you have an example?

No specific Genesis or Jenny plugins come to mind, but you can find a whole bunch of use cases on the Code Generation section of OpenUPM.

I recall there being some discussion (here or in Discord) re it being easier to do code gen (e.g. custom plugins) with reflection over Roslyn and other devs being more familiar with it. I submit that MS is pushing Roslyn as the go-to solution for these problems, therefore it’s reasonable to expect future devs (e.g. future us) to be more and more comfortable with Roslyn.

Yes, especially once source generators become more popular.

[jeffcampbellmakesgames]

Another goal which could also help to solve this solution, but also add more complexity and time is to create an external application which Unity would call in order to facilitate code-generation. This would likely begin as a v2 Genesis that would run as a CLI app first, using almost all of the existing code with some slight refactoring. After that point, a v2 EntitasRedux would start that would convert all the plugins to be compatible with any changes in v2 Genesis, then do the above described work of rewriting the data providers. This adds some of the benefits described above, but I don't think is a critical path to being able to use Roslyn code-analysis for the data providers.

Based on your feedback, I'd like to go with this second option. Thankfully the original version of Genesis I iterated on before I reworked and released it was a .Net Core app that ran external to Unity, so I did some work yesterday afternoon to drag some of that previous work in on a new branch and start the process of migrating the foundational genesis code over. I currently have it setup to run as a standalone CLI .Net Core portable app (so that it can run on multiple OSs) and it uses the existing menu item and shortcut to be able to run from Unity.

Once I get to a point where I have that foundational work done, I will create a project on the Genesis repository here and start tasking that work out at which point if you either of you would like to contribute that would be greatly appreciated! I will comment back on here once that's available.

At the point where Genesis v2 is stable, I'd start a feat/v2 branch for EntitasRedux where that refactoring/rewrite work could begin and we can write tasks for that as well.

[jeffcampbellmakesgames]

I have pushed up a branch feat/v2 onto Genesis with the foundational work, setup, created a project here to track issues related to it's development, and created several tasks and a few issues to start with (specifically configuration and an initial data provider to create a Roslyn workspace). I'll close this issue shortly as this work will be covered elsewhere.

If you need more details on how to get started, hit me up on discord and we can chat.

[jeffcampbellmakesgames]

Genesis v2 has officially been released, although more documentation needs to be created on how to implement code-generation plugins (installation docs and other info are available). I've done some initial work on ER v2 on branch feat/v2 to create an external code-gen plugins solution for ER, configure the Unity project to be able to leverage Genesis v2, etc... I'll be creating stories to document the work to port ER's code gen to take advantage of Genesis v2 and other related work over the next few days.