Run Julia under the GraalVM

[KenziTrader]

The GraalVM allows to combine code written in JVM-based languages such as Java, Scala, and Kotlin, but also other programming languages such as JavaScript, Ruby, Python, and R. I would like to be able to use Julia as well.

We have a Java based codebase. I develop my algorithms in Julia and then convert them to Scala for our production system. I prefer to add my Julia code directly in our production code.

I use the great JavaCall package for testing but its overhead is too big for our realtime system.

GraalVM can run LLVM bitcode and Julia is based on LLVM so it seems a plausible project.

[timholy]

Sounds pretty cool. Assuming "we" refers to a for-profit company, this might be a great opportunity to show your love of Julia by sponsoring someone at JuliaComputing to work on this feature. You might want to contact them and see what it would involve. (Non-disclaimer: this is not a self-serving comment as I do not work for JuliaComputing.)

[Keno]

This kind of thing is definitely possible, but it's a very big project and of course also is an ongoing maintenance burden. It's unlikely to happen unless somebody with serious long-term interest shows up and implements it.

[ViralBShah]

Perhaps more of a discourse discussion than an issue here. Requesting to move the discussion to discourse.

[StefanKarpinski]

Some non-Julia-specific work that would help towards this is porting libuv to GraalVM.

[Keno]

Not sure that's needed. You may be able to get away with just using Graal as the JIT and probably the GC, keeping the rest of the runtime native.

[ihnorton]

The Graal sub-project Sulong can execute LLVM IR on the JVM, however it does not support dynamic linking:

The interaction with the LLVM bitcode dynamic linker is not supported, e.g., dlsym/dlopen cannot be used. This does not allow to load native code.

The expectation is that everything is compiled with Sulong (and I believe this is the approach the FastR people are taking: compile R extensions w/ Sulong and distribute as LLVM bytecode).

[rschatz]

The interaction with the LLVM bitcode dynamic linker is not supported, e.g., dlsym/dlopen cannot be used. This does not allow to load native code.

This quote is from the "managed execution" section. Obviously you can't use dlsym/dlopen if you are running in a mode where you actively want to prevent running native code at all.

The expectation is that everything is compiled with Sulong (and I believe this is the approach the FastR people are taking: compile R extensions w/ Sulong and distribute as LLVM bytecode).

You can use native libraries from LLVM bitcode in the default mode. Of course you'll have a compilation boundary there. Calls across that boundary are going to be relatively expensive (but not significantly worse than random native interfaces like JNI or so).

GraalVM languages like FastR are usually taking the approach that the language extension code itself is compiled to LLVM bitcode, but it might still call out to actual native code for common system libraries.

[ViralBShah]

GSoC project - if the right mentor is available?

[el-oso]

@ViralBShah Where can I find the Discourse thread?

[ViralBShah]

What I meant to say is - the github issues are not the right place to have this discussion. But start the same discussion on a new thread in discourse (if there isn't one already).

[el-oso]

Sure, no problem. I guessed that much, I was just hoping that the conversation didn't stop here and that you might had the corresponding thread.

[ShalokShalom]

Julia should run out of the box on GraalVM.

Sulong does all for you, and if you encounter any issues, just raise an issue.

GraalVM gives you access to the ecosystem of the JVM (Java, Clojure, Scala, Kotlin) and also all the Truffle languages (Python, Ruby, R, Javascript, PHP, etc.)

https://www.graalvm.org/latest/reference-manual/llvm/