C#/.NET support

[itsamelambda]

Newbie question, any reason why C# and .NET is not supported? Would it be possible technically? Apologies in advance if this is out of scope of the project.

Thanks

[chumer]

Thank you for asking.

A C# and .NET interpreter on top of Truffle would be absolutely possible. You can either implement it as a full AST interpreter specialized for C# (similar to Graal.js) or you could implement a Bytecode Interpreter for CLR bytecode (similar to Sulong). The second approach would probably be simpler and would also get you support for other CLR languages.

Currently we have nobody working on it. But we are an open platform ;-).

[thomaswue]

We would actively support such an effort. Also, we could create internships for students with .NET assembly experience for getting it started.

[sheevy]

Would it be possible, in principle, to use something like lilc, which translates CIL to LLVM IR (https://github.com/dotnet/llilc however it seems to be dead).

[chrisseaton]

I think it's generally best to write Truffle interpreters as high-level as you can, as this means that tools understand and report on the actual source language, not a lower-level representation. You may also lose useful information about the program which could be used for optimisation when you lower it to another representation.

So the gold standard would be an AST interpreter for C# and the other .NET languages themselves.

A CIL interpreter would also be very good.

As you get to the point of using LLVM IR translated from CIL, compiled from C#, you might find things start to get a bit strained and tools don't work as well as you'd like and some opportunity for optimisation will be missed.

One of the reasons that TruffleRuby, for example, is faster than something like JRuby is that the Truffle representation of the program works at a higher level of abstraction than their emitted JVM bytecode.

[vmutafov]

Happy to see there are fellow developers interested in this idea. :)

Writing an interpreter for the LLVM bitcode generated by LLILC would probably be harder than interpreting C#/CIL. The reason for this is that the LLILC compiler "changes" your C# code in a way to enable features like garbage collection, exception handling and memory management as a whole, in the C# runtime (e.g. CoreRT). A disadvantage of using LLILC to compile C# to LLVM IR and use it on GraalVM is also the fact LLILC is not yet production ready and not all C#/CIL features work. This of course will change in the future I guess. Working on a lower level, as mentioned in the previous comment, would make you lose some program metadata too.

However, writing an interpreter for C#/CIL should be easier. I can't say for sure if writing an interpreter for C# would be easier than writing one for the CIL. In both cases there would be "hard" moments like supporting C# unsafe code, generics, exception handling, lambdas, etc. We should not forget the support for debugging C#. At least parsing either C# or CIL could probably be done by the Roslyn compiler(for C#) or the LLILC Reader(for CIL) thus not needing to write a C# parser.

But why do all this work? In my oppinion, this would be quite beneficial. This would finally provide a high performance Java(and other GraalVM languages) connection to the C# world, many libraries can then be reused without(or with less) effort and with no need of RMIs, language runtime bridges, type marshalling, etc. For me, this seems like a really interesting project with big value to software development.

[v6ak]

I think it's generally best to write Truffle interpreters as high-level as you can, as this means that tools understand and report on the actual source language, not a lower-level representation. You may also lose useful information about the program which could be used for optimisation when you lower it to another representation.

From performance PoV, this is true, unless the bytecode reflects the language enough. For Java, it might be the case, as bytecode was originally rather a postfix notation to the source code. (It is less the case today, but still, it is often pretty close.) For .NET and C#, I am quite less experienced than for Java and its bytecode, but I believe this might be still the case. Maybe CIL will be closer to C# than Java bytecode to Java, because it seems it has reflected the language changes (e.g., generics) more than Java did.

From usability PoV, CIL support will allow more than just C# support. Even for libraries written in C#, it seems to be easier to just pick a NuGet package than managing dependencies through cloning their source code. The only potential drawback I see there might be debugging (there is more complex mapping to the actual source code), but I believe this can be still handled.

Don't get me wrong, I believe direct language support can be useful for some other languages for .NET/JVM, but not much for C#.

[pladynski]

For C# and .NET based languages it seems to be better idea to use existing languages runtime bridges like our www.javonet.com which today out of the box let you access entire .NET world from Java even from GraalVM. Whereas implementation of Java based runtime for .NET seems to pose multiple risks and challenges like usage of Win32 dependent libraries, performance, support for all the features of .NET etc..

What do you think?

[vmutafov]

Hey! javonet looks interesting. :) How do you share the state between two runtimes - CLR and JVM? Is there detailed information about the architecture of the project?

[pladynski]

Javonet works both directions calling .NET modules from Java and Java for .NET. So to generalize lets assume calling language and foreign language. We manage the lifecycle of the objects so the GC in foreign language is waiting until all references are release on calling side and also we keep equal number of managed threads on both sides so cross-thread calls are made in corresponding thread. Therefore the state of foreign modules live as long as the calling side keeps any reference with respect to threads. We do not provide too much details of architecture on our website but we do answer all the questions on introduction demos, so you can request such meeting contacting our sales team.

Thanks for good feedback!

[vmutafov]

Thanks for the info :)

[chumer]

@pladynski running .NET and Java bytecodes as a Truffle language would have several advantages (please correct me if I am wrong):

1) Truffle languages use a shared GC (with your approach you need to make the Java and .NET GC work together, that can expensive and complicated, see literature on distributed GCs) 2) Zero overhead when calling from Java to .NET and vice versa with Truffle interop. (your approach at least requires a compilation boundary, e.g it cannot be compiled in one JIT compilation unit) 4) Shared tooling (debug and profile Java and .NET together) 5) Multiple instances of Truffle Java/.NET can run in the same process. 6) TruffleLanguages are sandboxable. (File system and network access is virtualizable, some features for that are still under development) 7) Interoperable with all other Truffle languages (Python, Ruby, JavaScript, R, ...)

[pladynski]

@chumer thanks for that feedback!. We are excited about those directions and look forward to position ourselves correctly to keep offering highest value for our customers in our field. Below some of my comments to your reply:

1. That could be true, indeed we need to get both GCs in sync however we do not see performance issues there.
2. We do need to pass the call between the two platforms however for many of call types Javonet pass them directly through references therefore the observable overhead is in fact only on types conversion or marshalling and if present averages around 0,000009s/call today.
3. Indeed shared debugging is needed but convenient from other perspective due to years devoted to adjust Visual Studio for debugging .NET and IntelliJ / Eclipse for debugging Java
4. We do work on supporting other languages as well, we do already support some other JVM based languages out of the box like Groovy

The key benefit is that .NET is interpreted by .NET, so for every existing library, developers have guaranteed the same performance and reliability of internal processing and no compatibility issues possible.

Correct me if I am wrong but with Truffle processing the C# CIL would be interpreted by Truffle and executed by your runtime which would be heading to replicate all features of standard CLR over coming time is that correct?

[svick]

@pladynski 9 µs (or 9000 ns) per call sounds like quite a lot to me. And writing it using confusing units to make the number look smaller does not inspire confidence either.

[chumer]

@pladynski thanks for responding. (Please keep marketing here minimal, this is a technical thread)

Correct me if I am wrong but with Truffle processing the C# CIL would be interpreted by Truffle and executed by your runtime which would be heading to replicate all features of standard CLR over coming time is that correct?

Correct. Not exactly a small undertaking ;-).

[NickAcPT]

Sorry for the ping, but have there been any updates on this? Couldn't Mono be used to pull this off? They do some magic already with https://github.com/mono/Embeddinator-4000 where they turn .NET libraries into native libraries that can be consumed by non .NET environments.

[brunoborges]

For reference:

• https://github.com/Microsoft/LLVMSharp
  • LLVM bindings for .NET Standard written in C# using ClangSharp
• https://github.com/dotnet/llilc
  • LLVM based compiler for .NET Core. It includes a set of cross-platform .NET code generation tools that enables compilation of MSIL byte code to LLVM supported platforms.

// @shelajev @thomaswue @chrisseaton

[ringods]

Could this be built by leveraging the Roslyn compiler framework?

https://github.com/dotnet/roslyn

[chrisseaton]

I'm not an expert on the .NET system, but my understanding is that Roslyn just generates CIL. That bit isn't the problem - we can use the existing compilers to do that. The bit we need to implement would replace the .NET VM.

[ringods]

@chrisseaton if you use the full package, then it generates CIL. However Roslyn offers you the full spectrum of source code parsing, analysis, compilation in an API. While I don't know a lot about Roslyn, I do know that we use Roslyn to perform static quality analysis rules on our customer projects.

So if you stick to the parsing functionality, would it be possible to generate GraalVM specific intermediate code?

[nafg]

I'm surprised that such a potential game changer is not being worked on by anyone

[brunoborges]

You probably want to get the .NET team's attention :-)

[ShalokShalom]

Yuhuu!

[schmik]

Yuhuu!

Indeed :-) Just in case Google invalidates the aforementioned link:

"Truffle CIL Interpreter", Master Thesis, submitted by Hagmüller, Patrick. Persistent URL: https://resolver.obvsg.at/urn:nbn:at:at-ubl:1-38220

Unlike the discussed implementations [...], which contain a runtime that executes CIL code, our presented approach compiles CIL code to Java bytecode. This Java bytecode in turn will be executed in the Java runtime. The possibility to execute .NET code in the Java runtime makes our approach novel.

[ShalokShalom]

So can we do this semi-official?

Is there any approach sensible, to put this in a repo, possibly oracle/trufflecil?

[bencz]

https://github.com/alex4o/truffleclr

https://github.com/gavioto/xmlvm

https://github.com/jagotu/BACIL

[laeubi]

Couldn't Mono be used to pull this off? They do some magic already with https://github.com/mono/Embeddinator-4000 where they turn .NET libraries into native libraries that can be consumed by non .NET environments.

Just for completeness mono has support for IKVM a JVM that can bind .NET assemblies to java: https://www.mono-project.com/docs/about-mono/languages/java/

it also describes how to compile everything to native code.

[NCLnclNCL]

Use nativeaot

[steve-s]

https://github.com/Softwarovy-projekt/Cilostazol is successor of https://github.com/jagotu/BACIL

It is more complete CIL spec implementation (but still far from fully complete). Especially the CIL binary format parser was improved significantly and could be useful as a separate component. However, the overall project's focus on completeness meant that the team wasn't so focused on ensuring good quality Truffle partial evaluation results during the development process (but that's fixable!), hence not so great performance numbers (fow now).