GDScript performance improvement via Just-in-time (JIT) compilation

[ghost]

What about adopting just-in-time (JIT) compilation for GDscripts?

JIT compilation has no disadvantages over interpreted code (aside from slower startup, in some cases) and many advantages, mainly in terms of performance.

For JIT compilation, there are at least two popular ways to implement it in a cross-platform way without writing any code from scratch:

• LLVM Intermediate Representation
• DynASM, used in LuaJIT

There is also the option of using PyPy, the JVM or the CLR, but those are quite heavier, using GNU Lightning or libjit (not sure if the license is compatible with Godot's) and of course, writing a code generator from scratch (which could take years).

So, the choice is pretty much between LLVM IR and DynASM. Both have very good performance, but DynASM is written in Lua and LLVM has a fairly large footprint (~20MB), but also offers other features.

Of course, compiling GDScript directly to C++ could also be a valid alternative - the only game engine which does it that I know of (if I understood it correctly) is Enigma, which compiles its own scripting language (EDL) to C++.

What do you think about this?

[Calinou]

JIT compilation has no disadvantages over interpreted code

There is one: portability…

But wouldn't implementing static typing be better and easier at least for now? Not to mention C# support which may eventually happen?

[bojidar-bg]

@Calinou Portability isn't a big concern, since we could just run the GDScript as we do now on unsupported platforms.

[reduz]

static typing can be optionally jitted too

[ghost]

@Calinou portability isn't a big concern if those libraries are used - they support plenty of architectures (LLVM probably supports more architectures than Godot) and besides, the interpreter could always be used as a fallback (as @bojidar-bg said).

LLVM IR would actually make static typing easier to adopt (on the back-end, of course GDScript would still need either type inference or explicit type signatures).

If C# support is implemented by linking either against Mono or CoreCLR, those both include JITs so this issue will be solved automatically. I can understand why C# (it's a better language than Java and Unity already uses it) but for those purposes, the JVM would probably bring better performance than either Mono or CoreCLR (especially because of the JVM garbage collector, which is well suited for games since it doesn't freeze anything). After Java 8, I personally don't think most people would object to it instead of C# (as the advantages brought by C# at this point mostly come down to Generics and LINQ, both not as useful in game development, which is mostly imperative), and using the JVM gives access to plenty of neater functional languages than C# like Clojure, Scala and Kotlin anyway.

Also LLVM makes it possible to use C++ in a JITted fashion. Unreal Engine also uses C++ for scripting, so I don't think that would be too weird (provided segfaults will be more common, but GDScript could still be offered to less experienced programmers). C++14 (supported by LLVM for the most part) is quite a small language even compared to Java and C#, if used properly (of course, compile times are long and error messages are still not as good).

[Ace-Dragon]

I would support JIT compiling for GDscript as long as it doesn't mean the end of the language (or its respective built-in editor) as it is now,

My proposal.

• Add a new static variable type, which in turn can be created by way of doing something like
  static var float(5.5) or something (the core difference would be higher performance and better parsing due to the expectation of it being a specific type. This method should allow the use of static typing while keeping the advantages of dynamic typing and eliminate any need for rewriting scripts if you want to use them
• After that, add a JIT compiling routine that converts GDscript to C++ behind the scenes (ie. invisible to the user), but visible in the form of performance when the game is run. The idea of using C++ here would mean that the Godot developers would not have to add a bunch of new libraries to the source.

[reduz]

JIT in dynamically typed languages like Lua is done by doing type inference from the entry point through the call tree, going as deep as possible. I'm not saying this wouldn't be possible in Godot, as pretty much every entry point is typed. The code completion in Godot does something similar, which is why it's able to guess so much type information.

However there are many cases where the type is not obvious, and while the code completion works great by guessing the get_node() calls on the tree being edited, this information might be different or change in run-time. I'm honestly not sure how efficient type inference might be.

To help on this, it would be easy to allow GDScript to allow the user to "force" the type of a variable (something not possible in lua). But if you are going to do this, then you might as well just go full static typing.

I'm not convinced of any approach at the moment.

On Sun, Jun 5, 2016 at 5:37 PM, Ace-Dragon notifications@github.com wrote:

I would support JIT compiling for GDscript as long as it doesn't mean the end of the language (or its respective built-in editor) as it is now,

My proposal.

• Add a new static variable type, which in turn can be created by way of doing something like static var float(5.5) or something (the core difference would be higher performance and better parsing due to the expectation of it being a specific type. This method should allow the use of static typing while keeping the advantages of dynamic typing and eliminate any need for rewriting scripts if you want to use them
• After that, add a JIT compiling routine that converts GDscript to C++ behind the scenes (ie. invisible to the user), but visible in the form of performance when the game is run. The idea of using C++ here would mean that the Godot developers would not have to add a bunch of new libraries to the source.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/5049#issuecomment-223836149, or mute the thread https://github.com/notifications/unsubscribe/AF-Z2znHs58L-KDgtIjHYYjgHVUGXgwpks5qIzOngaJpZM4IuWZe .

[Warlaan]

Haxe uses type inference even though it is type safe under the hood. Personally I like the fact that I don't have to define the type of every variable but that you can do so if you need it. That way you have the best of both worlds: you don't need to specify types (as long as the compiler can figure them out) and writing "generic"/"template" methods results in a much cleaner syntax as in statically typed languages like C++ or C#, but effectively the code is type safe and you can specify the requested type whenever you want to use it to make the code more readable.

In Haxe it's done using the syntax var <name> [: <Type>] [=<initial value>] which looks very weird coming from typed languages but makes perfect sense coming from a language like GDScript.

But I have no idea how this affects complexity and effectiveness of JIT compilation, I merely wanted to point out that this would be kind of a middle ground between dynamic typing as it is now and full static typing as it is in languages like C++.

[volzhs]

I used Haxe for 1~2 years long time ago. I like it to be able to define type optionally for variable and function return as @Warlaan said.

[bojidar-bg]

I'd prefer the c++ type declaration style before the typescript/haxe one:

var x = 50
int y = 60
float z =70

Or maybe we could mirror export's style:

type(int) var y = 60
type(float) z =70

[kubecz3k]

@bojidar-bg Personally I prefer c++ style more (a lot less typing).

[ghost]

There are a lot of issues that can arise from putting the type before the variable. Go was written by some of the most experienced C implementers ever (including Ken Thompson) and they went with Postfix notation (like Pascal and Haxe).

I think this discussion belongs in a separate discussion. I'm not proposing a vote because of course most people are used to typed languages like C++, C# and Java and could vote the familiar syntax but merely to gather research materials and opinions to find the solution that is best suited for a language like GDScript (which is clearly not C-like in any other aspect).

[Warlaan]

I agree in all four aspects:

1. The C++-syntax is nicer to read, but
2. imho the postfix notation makes more sense here
3. which I would prefer to discuss (not decide by majority)
4. in another thread. Sorry, I didn't mean to hijack the thread when I mentioned the topic. Let's return to the issue of JIT compilation.

[ghost]

Would implementing JIT via LLVM mean Clang would be preffered over GCC . Is it possible to compile LLVM JIT using GCC?

Unlike Java , Python, C# etc, GD Script works as part of a bigger C++ native code engine. In fact both GD Script & the rest of the engine are one unit. Some game components are C++ and some are GD Script. Overall the game performance depends on creating a balance between the two. Writing a Godot game solely in script would make it slower but writing it mostly via nodes with minimal gd script would make it faster.

So my question is . Is this small boost in performance really worth the effort?

I think C# addition may in fact be better for the engine than implementing JIT in GD Script. C# is probably much more optimized and efficient language thus probably as fast as GD Script + JIT but not as integrated as GD Script.

To answer my own question I personally don't see this being an important issue right now. Maybe in the future when Godot has all the desired features and devs would be simply optimizing those features.

[ghost]

Would implementing JIT via LLVM mean Clang would be preffered over GCC . Is it possible to compile LLVM JIT using GCC?

Yes, but then LLVM would become a dependency so it would make more sense to use LLVM for everything.

I don't think the boost in performance would be small for larger games. Maybe for arcade games it would not be noticeable, but in games with a lot of loops and function calls it definitely would.

Also, is there a discussion about why C# over Java or even C++? I think C# is a very good language, but the CLR and tooling is definitely inferior to the JVM (especially on other platforms than Windows).

[ghost]

@paper-pauper you should make a forum post about Java, JIT to continue this discussion. I'd actually prefer Java > C# tbh. I think C# was a not a choice but more of an opportunity that devs decided to take.

[ret80]

so that in the end we decided? will JIT or not?

[ghost]

How about the ability to compile GDScript to static C++ code? Would that even be possible?

[SuperUserNameMan]

@trollworkout : it would probably be simpler and more optimized to write your game logic directly in C++ then, wouldn't it ?

[ghost]

@SuperUserNameMan Not necessarily. First you would be using C++ not GDScript and second you still need to figure out a way to load a binary object as a dll and plug it into the engine without having to recompile the engine with ur C++ changes.

[ghost]

@trollworkout Thanks to #3936 it could be possible to compile GDScript to C and load it dynamically, eventually.

[ret80]

it is not necessary to translate the script in c ++. @SuperUserNameMan rights

[ret80]

or need JIT or c# and all will be happy

[ghost]

nah the other topic @paper-pauper linked is actually much more interesting. The idea is to use C binary as a type of scripting language and call code directly via reflection rather than being interpreted making it as fast as C++ native code I think that would solve the ABI issue . In fact with this you don't need ABI anymore.

[brakhane]

I am currently experimenting with implementing a GDScript JIT using RPython. The first (very preliminary) results are promising, but I don't know how hard it will be to integrate it into the engine.

Is there still interest in a JITed GDScript or are people more interesting in c# or static GDScript?

[Ace-Dragon]

Considering that you're actually trying out a possible solution, I say go for it and see if there's enough performance gain in games to be worth it.

What I mean by that is such a thing would only be worth pursuing if there's a rather large performance gain to be had for complex logic (or otherwise such gains might be better obtained from just doing general optimization work on the language itself).

[qiqian]

Performance is crucial for me, I just can't choose Godot for the script Performance. But I think a faster language(Lua, Java, c#) is better than Jit, since Jit doesn't work on iOS. And compiling to c++ loses the possibility to update game logic on iOS

[vnen]

Isn't Lua faster just because of JIT? Or does it have some kind of AOT compilation too?

I still prefer a statically typed GDScript (maybe an optional typing system, like TypeScript is for JavaScript). Also having GDScript -> C++ transpilation would solve a lot of performance issues, but I can see this is far from trivial.

[punto-]

How do you update the game logic in ios? you can't download scripts from your games and run them on ios, everything has to be installed from your original ipa that is published in the app store. Either way you have to update the app to get new logic, event if it's new scripts or a new runtime with the logic compiled in.

On 20 August 2016 at 05:05, George Marques notifications@github.com wrote:

Isn't Lua faster just because of JIT? Or does it have some kind of AOT compilation too?

I still prefer a statically typed GDScript (maybe an optional typing system, like TypeScript is for JavaScript). Also having GDScript -> C++ transpilation would solve a lot of performance issues, but I can see this is far from trivial.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/5049#issuecomment-241175124, or mute the thread https://github.com/notifications/unsubscribe-auth/AGVmPZDOcIkO1-6ayDySyPnCfJdRk5Saks5qhm76gaJpZM4IuWZe .

[qiqian]

Statically typed GDScript with optimized VM would be also be good, is there any plan for this ?

[Shin-NiL]

Optional typing system for GDScript as @vnen suggested would be perfect :)

[RandomShaper]

While researching about this, I've found the great SLJIT. I have been trying to figure out how could I integrate it to allow JIT compilation at least at the GDScript function level.

After some hours of reading/thinking I've come to the conclusion that it would be of little benefit because the biggest bottle neck, now I have studied the code deeplly, is probably all the Variant stuff.

Finishing my task for a test would be overkill so I've abandoned it. Anyway a branch with my initial work is in my repository in case someone wishes to play with SLJIT and have it already integrated into the build system.

So I sum myself to the group that believes that static typing would remove a lot of calling and branching thus improving performance notably.

[brakhane]

@RandomShaper

I agree that the JIT is not feasible currently, but the problem is Variant and not dynamic typing.

I've experimented with writing GDScript JIT using RPython, which is the base for PyPy and specifically made to create JITs for dynamic languages.

I disagree that the problem is static typing, it would be possible to have a fast GDScript JIT if we wouldn't have to work with Variant. Variant does replicate some of the functionality of dynamic languages, and unfortunately that is the part where RPython's JIT gets its speed boost. RPython needs to know what types it currently has and can then speed up those parts.

Variant is so deeply embedded into Godot however, that it's really hard to go without it. I would have to basically reimplement all of GDScripts data types and transform to Variants afterwards.

If anybody has an idea how we can get by without using Variants, a fast GDScript would be possible (a proof of concept GDScript JIT of mine that didn't use Variant was promising. A (silly) loop that in GDscript took 20 seconds was done in under a second when JITted)

[RandomShaper]

Well, I was treating Variant and dynamic typing as if both were the same thing or as if one couldn't be without the other.

But probably I'm missing something on language design.

[vnen]

AIUI Variant make possible to use C++ in Godot as if it were a dynamically typed language. Especially in the return type for some methods. I'm not that acquainted with the source, but I'm not sure if there's a clean way around it.

[ghost]

@brakhane Care to share your RPython experiment? I think everybody here would benefit from taking a look at it :smile:

Variant is so deeply embedded into Godot however, that it's really hard to go without it. I would have to basically reimplement all of GDScripts data types and transform to Variants afterwards.

In theory, what would be the drawback to this (aside from the time needed to implement it)? This kind of casting is not that expensive, as far as I understand it.

[ghost]

@paper-pauper look at PyPy and RPython - Python is strictly dynamically typed, and PyPy can speed-up Python code very well thought... RPython is very good for development of interpreters (with almost-free JIT)

[touilleMan]

A really interesting article has been published about (currently in development) new Ruby JIT.

The key part here is the Ruby choose to implement their JIT by generating C code and calling an external C compiler to build a .so then load it. Needless to say this has created a lot of interrogations and this article starts to give some answers.

Basically the advantage to use and external compiler (as opposed to integrate a JIT framework) are:

• independence to a single compiler (can use GCC or LLVM depending on what's available/best)
• JIT frameworks like libGCCJit are young so their API can be much more fragile that C

Of course this needs to be taken with a lot of precautions (the article benchmarks compare a prototype with production ready projects such as JRuby), but it shows that, provided you use clever optimizations such as precompiled headers, using an external compiler can be a viable solution.

From the Godot point of view, this technique could be useful to also provide a static compilation of GDscript.

[DoctorAlpaca]

I'm working on JIT compilation for my master's thesis and am planning on implementing something suited for Godot. So far I've mostly just read a lot of papers on the topic, specifically on how to deal with the dynamically typed nature of GDScript. I've now started working on a prototype, and hopefully have something to show relatively soonish.

[vnen]

@DoctorAlpaca I'm almost done with the static type checks for GDScript, I believe this might help you at least a bit (you can look my fork if you're interested, and also #10630). I was planning to look into JIT eventually, but I have zero experience with it.

[RandomShaper]

Some time ago I started an experiment about JIT-ting GDScript. I did setup some foudation for it. In case you want to have a look... https://github.com/RandomShaper/godot/tree/exp-gdscript-jit-master

[DoctorAlpaca]

@vnen I'm actually very excited to see how much of a difference in performance static typing can bring, as opposed to the type profiling I'm planning on doing. Since I'll also have to produce some actual text for my master's, I don't think I'll get much more done than a prototype, on which there'll be a lot of room for improvement.

@RandomShaper That setup looks nice, and after looking at the available options I'll probably go with SLJIT as well. Thanks for that!

[touilleMan]

May 2018 poll results have arrived, and this proposition appeared to be on the roadmap priority.

I personally don't think JIT GDScript would be good idea:

• JIT is not allowed on mobile platforms (at least on IOS, not sure about Android), which are the one needing the most this kind of optimisation...
• JIT is a big mess. There is multiple approaches (tracing vs method JIT), some of them not even deterministic :'-(
• JIT is a BIG BIG mess. Debugging is complex (involving dumps of jited code and memory...), so is profiling, so is maintaining the code :'-(
• JIT is a special kind of mess. JIT programmers is a scarce resource so maintaining such a thing would be tricky. This could lead to making GDScript evolve becoming harder since the interpreter and the JIT would be tightly coupled.

The luaJIT project illustrate well those points: the project is simply brilliant (code base really good, performances crazy good), it solve a real usecase, it is used by commercial companies. And yet it is slowly dying since it main author has step back, unable to keep up with the new lua version (luaJIT stopped at 5.1, arguing the add of integer on top of float in the language made by the 5.2 would be really complex to add), even if the lua community has a medium size and luaJIT it best known project.

Obviously GDScript is order of magnitude simpler than Python or even lua, but the burden of making a JIT stays an important one, especially considering it is not a core feature of the Godot project.

The only acceptable solution here would be to rely on llvm which would take care of all the crazy parts and let us just output llvm-ir code to it. However, as said before, llvm is a big dependency (at this point we could almost ship v8 with Godot and write a GDScript to Javascript transpiler instead :trollface: ), so additional research to understand how much would be added is required here before going anywhere. And again this wouldn't be a viable solution for at least IOS platform (damn you Apple !).

I'm sorry to sound so gloomy here, but I wouldn't post this without a proposition ;-) I guess what poll voters understand by "JIT GDScript" is "GDScript, but fast as native code". So in the end it doesn't really matter what it the mean to reach this goal right (AOT, JIT, transpiling + compilation etc.) ?

From what I saw of GDScript, a really cool (lack of) feature is it is not a dynamic language. There is no exec keyword, no monkey patching, you cannot modify variable during debugging etc. This makes the language suitable for static compilation (and in fact it is again one more reason not to go with JIT). On top of that the new static typing system should allow easy optimisation (compare to having to do type inference).

So my idea would be to start by making a simple (no optimisations, just use Variant everywhere) GDScript-to-C transpiler, that would available as godot --gdcc <myfile.gd>. Using the existing GDScript compiler and GDNative api should make this relatively easy. The users are then left to deal with the rest of the toolchain (for example compiling each C file as a .so and replace the original .gd file by a .gdns file pointing on the .so).

In a second time, we could start wondering about optimizations and, more importantly, a way to ship a compiler into the Godot. Considering this second point, tinycc seems a really interesting solution given it low weight, it number of platforms supported (x86, amd64, arm, arm64 for linux, win32 and osx), it libtcc.h api that allow to integrate it into a program and it cross compilation mode. On top of that version 0.9.27 has been released last December (so the project seems stile alive ^^).

Finally the same way we currently turn .gd files into .gdc or .gde at export time, we could create a new type (.gdso ?) that would be nothing but a regular nativescript (so the GDNative resource loader would call Nativescript on it automatically).

[neikeq]

I agree it would be overkill to write a JIT compiler for GDScript. As you mention, JIT compilation is not allowed on iOS (Android does allow it), so you would either have to fallback to the slow interpreted on that platform or write an AOT compiler (look! now you have to maintain two compilers as well as the interpreter).

I said many times that I consider the GDScript2C transpiler to be the best option now that we have GDNative.

For binaries built with the mono module, you could write a GDScript2IL compiler and then you would have both JIT and AOT compilation.

[neikeq]

From what I saw of GDScript, a really cool (lack of) feature is it is not a dynamic language. There is no exec keyword, no monkey patching, you cannot modify variable during debugging etc. This makes the language suitable for static compilation (and in fact it is again one more reason not to go with JIT).

Actually, in GDScript you can modify the source code for a script at runtime.

[touilleMan]

Actually, in GDScript you can modify the source code for a script at runtime.

@neikeq I looked for this before writting my post, guess I didn't look close enough ^^ can you point me to an example ? (or where it is implemented in the codebase)

[vnen]

Actually, in GDScript you can modify the source code for a script at runtime.

Not as much as "modify". You can set the source_code property of Script with a whole new code and then call reload() to recompile it. Technically this is implemented for C# as well, since it's part of the Script API.

@touilleMan your idea is similar to what's discussed in #11068.

[DoctorAlpaca]

Problem with transpiling GDScript, or compiling it ahead of time, is that even with the static type system all method calls still have to be virtual. Anything could hide behind a Node parameter. So you'd run into similar performance problems as Java, which uses JIT on its bytecode for (in part) the same reasons.

In any case, since I need a practical part for my master's thesis I'm going to implement a JIT for GDScript anyway, so after that we can see how well it fares and how much it destroys the code base. I honestly wouldn't mind even if it stays a prototype forever and never gets merged.

Also, for the dynamic things, you can load script files from anywhere on the disk and use them for anything you can use scripts in your project for. Which is a very nice feature for DLC and/or user created content (though I don't know if anyone is actually using it).

[neikeq]

@vnen The source code in C# is only used for scripting. It's not compiled on reload.

[ret80]

@touilleMan

Considering this second point, tinycc seems a really interesting solution given it low weight, it number of platforms supported (x86, amd64, arm, arm64 for linux, win32 and osx), it libtcc.h api that allow to integrate it into a program and it cross compilation mode. On top of that version 0.9.27 has been released last December (so the project seems stile alive ^^).

Bad idea, the project is more likely dead than alive!

@neikeq

For binaries built with the mono module, you could write a GDScript2IL compiler and then you would have both JIT and AOT compilation.

But this is an interesting option. And perhaps comparable to the labor costs with the addition of JIT.

In general, if there is already a person who will be engaged in this in any case, then let him do it. The team does not lose from this. I propose to relax and wait, suddenly something will turn out (although I do not count on it)

[touilleMan]

@DoctorAlpaca considering the virtual method dispatching, I guess things are already this way when using the C++ API. So your goal here is to make GDScript faster than C++ ? ;-) Anyway, I'm sorry if my previous post had appeared to you in a "it's useless, you shouldn't do it" style. Experience is immensely more valuable than supposition so I can only support you no this, on top of that there is great chances your work will bring you to improve various part of the engine (bug fixing, cleaning, or even finding optimizations for the too-dynamic-to-be-fast Variant system) so nothing is never lost ! BTW if you're really interested in JIT, I suggest you come to the Europython (end of July in Edinburgh, I'll be there ^^), the Pypy team is there every year and you can do sprint coding on Pypy with them during the whole weekend. There is usually not much people and they are really kind and pedagogic so it's basically like taking a 2 days class on JIT with world class experts ;-)

@ret80 I agree the tinycc project is not really active. On the other hand we are talking about a C compiler, the most used standard is still C99 (with C89 still really present) so there is not much evolutions to do once the project is stable enough (I guess we could say evolutions could be done in optimization, but this would likely add to the size and the speed of the compiler, so not the goal of this project). My biggest concern for this compiler is more the support of arm architectures and cross compiling .dll and .dynlib because they are more exotic feature (so potentially less tested). Finally given we would use this compiler with autogenerated code (always same constructions and gdnative as single dependency), we should be relatively safe not to hit hidden bugs once the project is on track.

@neikeq I find really elegant the idea of using the mono runtime for the JIT ! This would keep low the number of dependencies (and improve the number of people involved in the mono module which is always better) and they would save us from making yet another release.