[proposal] JIT compiler

[peter7891]

I hereby propose, at some point, to make a JIT compiler for gnolang. By monitoring what the hot path in a contract/library is, we can JIT compile those functions and have x20-50 speedups.

The most sensible thing to do is use LLVM as a backend.

A possbly optional feature is allowing users to dictate when a function is JIT-ed. We can do this with a similar feature as the Go build tags. Something like

// +JIT(always)
func expensive_function {}

[moul]

This sounds very promising. However, we must ensure that both the JITification and the JIT runtime are deterministic. It may be necessary to blacklist certain operations from being jitted, and we may need to establish a consensus process for adding the package to ensure that the JITification is seamless.

Do you have any insights into the complexity and timeline for developing a proof of concept that we can analyze?

[peter7891]

This sounds very promising. However, we must ensure that both the JITification and the JIT runtime are deterministic. It may be necessary to blacklist certain operations from being jitted, and we may need to establish a consensus process for adding the package to ensure that the JITification is seamless.

Do you have any insights into the complexity and timeline for developing a proof of concept that we can analyze?

Since we have full control over the LLVM IR(intermediate representation) generated, i don't see determinism being an issue. We just need to be mindful of what we tell LLVM to do, basically.

The JIT compiler will be entirely integrated into the VM and do compilation on demand. As the LLVM ORC api suggests https://llvm.org/docs/ORCv2.html This means that we will evaluate code, as we do now and whenever we think a function is too expensive and it is being called many times (It is a hot function), we will compile it, save the compiled code (in memory) and further calls to that function will be done on that. That gives us the flexibility to do JIT-ing as wide or narrow as we want. It also will allow us to incrementally work on it, while still being useful for the code it can optimize.

As far as a timeline for POC, it is hard to give anything other than a guestimation. The reason is, there aren't any well maintained LLVM libraries in Go. I will need to investigate if one of the two LLVM wrappers has a newer enough version of LLVM. If we have to write an LLVM wrapper, it would take longer. POC that can illustrate the possible gains will take me 2-4 weeks ~

[moul]

Why ask developers to specify which parts should be Jitted instead of auto-Jitting?

Couldn’t this be merged with #671, Jitting once at AddPkg?

[peter7891]

Why ask developers to specify which parts should be Jitted instead of auto-Jitting?

The idea behind this is the same as many languages allow users to say whether functions are inlined or not. It's mainly to add the ability of users to determine that they know for a fact something should be inlined and not rely on the analysis of the compiler to decide, if it can't figure it out. Or override them for a single function. In our case, the analysis we write may not compile everything that should be, because of prudency. But the user might know his application better than us and thus explicitly flag it. Couldn’t this be merged with #671, Jitting once at AddPkg?

JIT compiling is only worth doing for functions that are called a lot or do a lot of computation. This is because if you JIT compiled functions that don't do a lot of computation, the compilation process time, because it is done during runtime, will end up making it slower than not compiling it. This fundamentally relies on running the code as is now and then determining through some metrics that it is worth to JIT compile function X or Y.

Some of the things we will have to do are.

• Runtime statistics (how many times called and or how long a CPU bound function ran)
• Analysis of the AST (if we spot a lot of constants, we can pass it to LLVM to compile it and it will do things like constant folding)

[moul]

JIT compiling is only worth doing for functions that are called a lot or do a lot of computation.

What about caching JIT objects to make them more worthwhile?

[peter7891]

JIT compiling is only worth doing for functions that are called a lot or do a lot of computation.

What about caching JIT objects to make them more worthwhile?

Caching it would amortize the compilation costs, in theory. If by caching, you mean writing it on disk, i would need to see what that would involve. But it sounds like a good idea.

[peter7891]

@moul maintained LLVM bindings can be found https://github.com/tinygo-org/go-llvm