Dynamic C# code loading

[CptMoore]

being able to load .cs files without having to precompile them allows for the following use cases:

1. mods can be developed without VS (still need dnSpy though) I still want .cs projects to be compatible with .dll project structure wise (so one can leverage from VS and resharper tooling)
2. mods can expose certain code that mod users can modify without much VS or even coding experience. so basically an advanced configuration file.

I'm right now finishing up a new mod for BTML called DynTechMod that should allow .cs based mods. I'm also splitting up my VisualHardpointLimits mod into versious submods based on the .cs idea.

Once I'm finished we can discuss how to integrate that into ModTek of even if.

[mpstark]

An interesting idea -- I would be concerned about a couple things:

• Assembly compatibility with BattleTech's included assemblies -- Unity and Mono are not one-to-one with .net
• Feature-creep into ModTek's already fairly large feature-set
• Extending BattleTech's already giant load times
• Encouraging users to edit code files that they don't understand

[CptMoore]

Assembly compatibility with BattleTech's included assemblies -- Unity and Mono are not one-to-one with .net

That is no issue, when compiling one can just reference the unity/mono assemblies. doing that already in my prototype.

Feature-creep into ModTek's already fairly large feature-set

Feature creep is a none issue, it would just precompile a dll before loading (from a code view and a modders view, just one additional feature/step). There is an issue with release size and maintenance.

Alternatives would be to modify ModTek to just optionally integrate such a solution, but that would even further confuse the users installation procedures.

Extending BattleTech's already giant load times

Solved since one just compiles an assembly to disk and does not recompile unless sources change.

Encouraging users to edit code files that they don't understand

Well that's a risk with all modding, but cooler would be if users start learning stuff step by step, this can really help with that.

So my current list of issues:

• Large distribution size, it will probably bloat up to 10MB because of that. One has to include a minimal compiler suite.
• Only windows currently (mono.exe ..) I could test on a mac later, but maybe will introduce more bloat.
• C# language version and features are limited by the distributed compiler suite (none issue now, just take the latest, but might be problematic later)
• how to properly integrate into ModTek within the constraints you have given, not a big issue but it does change my prototype in key ways (e.g. right now I allow .cs files to depend on the DynTekMod assembly..)

I also evaluated other ways of scripting:

• scripting languages like ironpython, ironruby, unity script (based on js) all still require a compilation step and usually a compiler that has to be bundled, so even worse than C# since you have to learn more languages and have another layer in between.
• other light weight scripting languages are nice for just exposing algorithms, but they interop poorly with C# (and harmony) or not at all. They are all interpreted too, so they would slow down the engine considerably in certain use cases.

[CptMoore]

Size of the Mono-Compiler Suite zipped is 17 MB, windows only

[CptMoore]

I have a working prototyp

BTML mod that dynamically compiles other mods https://github.com/CptMoore/DynTechMod

DynTechMod mod that uses dynamic compilation: https://github.com/CptMoore/VisualHardpointLimits

Right now the example mod is directly initialized from DynTechMod. Once ModTek is out, DynTechMod will only be a compiler, compiling various ModTek mod in the BTML init phase and ModTek will initialize those once that the BTML init phase is finished.

Status:

• allows for dynamic compilation of whole mods
• soon allows for dynamic compilation of code snippets in C# (useful for scripting, e.g. when testing gameplay algorithms)
• can be compatible with mods that are developed using visual studio (example mod can be both compiled with VS or using DynTechmod)
• requires additional 17MB to download (nothing in todays world though)
• dynamic compilation not a substitute for a real IDE with static analysis features

[Tyler-IN]

I've done C# scripting (e.g. compiling snippets) before in UE games, but compiling entire projects at runtime .... is a bit of a huge, enormous ... overcomplication.

Mono provides a light-weight example of a REPL project that could be used as an example; http://www.mono-project.com/docs/tools+libraries/tools/repl/

Pretty sure all you need to include is Cecil and it's friends, not the entire framework...

Unity even bakes in support for compiling UnityScript for this kind of usage... but gah holy cheesy rice on a cracker Batman...

You may have over-scoped by ambition here.

Where exactly is VisualHardpointLimits using dynamic compilation? I guess I'll go looking. edit; Oh it literally builds the entire project. Yeah I don't think that's cool...

edit2; Also, I think that instead of dynamic compilation, if you have something that needs to be compiled at runtime, you might be better off looking at combining System.Linq.Expressions's C# language integrated Expression<Func<...>> parameters with Expression trees and "Lambda" (compiled) expressions, if you can't just get by with generics and/or file parsing.

[CptMoore]

Unity even bakes in support for compiling UnityScript for this kind of usage

Based on my research, Unit Editor has scripting support which is not available in the unity 3d engine runtime. Meaning you can't compile on runtime or let the script run on runtime. Please tell me if that's different, I really could use information on that.

System.Linq.Expressions's C# language integrated Expression<Func<...>> parameters with Expression trees and "Lambda" (compiled) expressions.

Don't know how far that can get me, but a query based language doesn't seem suited for the use case I envision.

In order for this feature to be really useful, I would like to have this class as a script: https://github.com/CptMoore/VisualHardpointLimits/blob/master/source/stats/MechStatisticsRulesCalculateHeatEfficiencyStatPatch.cs#L14

Oh it literally builds the entire project. Yeah I don't think that's cool...

One file, every file, best to have the options. In any case, another pro would be easy accessibility for people to look at code and modify it if they want. There is no alternative that makes it that easily unless the ecosystem forces it (e.g. Cocoapods forces it for iOS development).

REPL

Since mono is implementing CSharpCodeProvider by calling csc.exe I always assumed it doesn't have easy accessible compilation features, but REPL might be it. Gonna test that now. In the worst case modern language features might not be available, but that should be enough for most use cases where scripting is required.

[CptMoore]

REPL

Ok, this took me a while, but I have made it. Found out that in later mono version the compiler functionality is exposed and directly called for REPL functionality. Found out I can do the same with an older Mono.CSharp.dll thats compatible with unity, just need a bit of reflection. Was happy when I saw that instead of mono.exe I can just call Mono.CSharp.CompilerCallableEntryPoin.t

Prototypes:

• DynTechMod
• VisualHardpointLimits

So new status:

• allows for dynamic compilation of whole mods
• soon allows for dynamic compilation of code snippets in C# (useful for scripting, e.g. when testing gameplay algorithms)
• can be compatible with mods that are developed using visual studio (example mod can be both compiled with VS or using DynTechmod) (requires C# 5.0 language version)
• requires additional 351kb to download (nothing in todays world though) dynamic compilation not a substitute for a real IDE with static analysis features should be independent on platform now, so mac and linux support included

[CptMoore]

The csharp compiler thats compatible with unity mono is buggy. SelectMany crashes hard, not even an exception to catch. The same code compiled with the newest VS (or newest mono) works perfectly.

[mpstark]

I don't think that this ultimately should be in ModTek and you've got development rolled off into DynTech, so I'm closing the issue.

[CptMoore]

I agree, since ModTek provides a nice dependency management system, DynModLib seems to work fine.