Porting to QuickJS

[Geequlim]

I will wait until quickjs is more stable at least the debugger is implemented.

Export godot symbols to script

• [x] Refactor the ECMAScriptGCHandler to improve the binding process
• [x] Export godot Object types godot.Object godot.Reference
  • [x] Constants and enumerations
  • [x] Methods
  • [x] Properties
  • [x] Indexed properties and theme properties
  • [x] Signals
• [x] Export global symbols of godot
  • [x] Builtin functions godot.sin godot.randi
  • [x] Builtin constants and enumerations
• [x] Export builtin types godot.Vector2 godot.Color
  • [x] Implement the way to bind builtin types efficently
  • [x] Script to generate binding code
  • [x] Operator override
  • [x] Properties and constants
  • [x] Mannually bind overrided methods
  • [x] Mannually bind constructor initializers
  • [x] Mannually bind virtual properties of builtin types Rect2.end, Color.r8

Register script classes to godot

• [x] Implement godot.register_class to register classes form JS
• [x] Modules
  • [x] ES6 module support
  • [x] Register ECMAClass with export default ClassFunction syntax
  • [x] Preload resources with module
• [x] Export properties from JS to godot inspector
• [x] Register signals from JS
• [x] Update the plugin to dump api
• [x] Tooled script support
• [x] Class icon support
• [ ] Named class to allow create customer node write in javascript in the editor create dialog

Exporting

• [x] Allow exporting javascript scripts
• [x] Support compile script to bytecode when exporting
• [x] Support export script with encryption key
• [ ] Implement script dependence dectection to allow only export required scripts

Optional Plans

• [x] Refactor the abstract binding helper to make it easier to port to other script engines
• [x] Refactor the ECMAScript resource workflow to allow attach javascript files to nodes
• [x] Allow passing function object for Object.connect, Object.disconnect and Object.is_connected
• [ ] Override more methods which using method name as paramaters to using function object
• [x] Add await support for signals
• [x] Add plugin for better using of TypeScript
• [x] Add multi-thread support with Worker API
• [x] Add api to convert data between godot pool array with javascript typed arrays
• [x] Operator override for builtin classes
• [x] Add debugger support
• [x] Optimize the Performance
  • [x] Profile from the fork of bunnymark
  • [x] Optimize the performance for builtin types
  • Add allocator for binding object ECMAScriptGCHandler
  • More work to improve the performance for container types Array Dictionary

[rosshadden]

This looks awesome!

[erodozer]

I was just about to ask for this as it appears to be a huuuuge improvement over duktape and even chakra if you want something small to embed with high ECMA compliance.

[Geequlim]

Test case for QuickJS to make sure we can use Godot Object type correctly in javascript

(function() {
    try {
        console.log("---------------------------");
        var btn = new godot.Button();
        console.log(btn, btn instanceof godot.Object); // <--- Variant toString test should print `[Button:XXX]`
        // prototype chian check
        console.log(btn instanceof godot.Object, btn instanceof godot.Control, btn instanceof godot.Resource); // true, true, false

        (function () { // C++ take references of JS objects
            var theme = new godot.Theme();
            theme.aaa = 'AAA'; // <-- script properties
            // UTF8 character & properties
            theme.set_name('资源名');
            theme.resource_path = '资源路径';
            btn.set_theme(theme); // <-- Add reference count of `theme` object in C++
            console.log(theme, theme.resource_name, theme.get_path());

            // `o` is Object type should not be auto freed by GC
            var o = new godot.Node();
            o.ooo = 'OOO'; // <-- script properties
            btn.set_meta('obj', o); 
            console.log(o);

            // `res` should be freed by GC as it was not used in C++
            var res = new godot.Resource();
            console.log(res);
        })(); // <--- Avoid `theme` and `o` be freed by the GC

        var theme = btn.get_theme();
        console.log(theme, theme.aaa); // <--- Get script property to make sure the object is the one we created before

        var o = btn.get_meta('obj');
        console.log(o, o.ooo);
        btn.set_meta('obj', null);
        // Object have to be freed manually as GDScript
        o.free();
        btn.free();
        // theme.free() // <--- Throw exception when try to free Reference

        console.log("---------------------------");
        // constants
        console.log(godot.Control.CURSOR_HELP);
        // enumerations
        console.log(godot.Control.CursorShape.CURSOR_HELP);
        console.log(JSON.stringify(godot.Control.CursorShape));
        // signal
        console.log(JSON.stringify(godot.Control.focus_exited));
        console.log("---------------------------")
    } catch (error) {
        console.log("ERROR:", error);
    }
})();

[Geequlim]

Fork of QuickJS with the necessary changes: https://github.com/koush/quickjs VSCode Extension: https://github.com/koush/vscode-quickjs-debug VSCode Marketplace Link: https://marketplace.visualstudio.com/items?itemName=koush.quickjs-debug

↑ It's time to start the work :)

[Geequlim]

Bunnymark compare to GDScript 2019-11-17:

2020-01-01

[erodozer]

In addition to porting to quickjs, are there any plans to investigate bindings using pluginscript instead of a godot module. It would be nice to have this supported as something that can be downloaded through the asset library instead of having to compile it into godot. Quickjs is rather portable, so distributing the addon would be conceivably small and quick to plug in for projects thinking of exploring the option.

[Geequlim]

@nhydock No such plan in the near feature.

[ChaosWitchNikol]

Hello, I've encountered this error when trying to compile: (on Ubuntu 18.04.3)

modules/libmodules.x11.tools.64.a(quickjs_builtin_binder.x11.tools.64.o): In function 'QuickJSBuiltinBinder::initialize(JSContext*, QuickJSBinder*)': /home/cupi/Make/godotJS/godot/modules/ECMAScript/quickjs/quickjs_builtin_binder.cpp:160: undefined reference to 'QuickJSBuiltinBinder::bind_builtin_classes_gen()'

[Geequlim]

@cupiniki comment this line https://github.com/Geequlim/ECMAScript/blob/706d39af38e313170ffdbed6f94ee65ad9ec0cd2/generate_builtin_api.py#L6

[ChaosWitchNikol]

@Geequlim Thank you

[Geequlim]

@nhydock Why not using GDNative/ PluginScript ?

1. I'm not sure the gdnative and the plugin script related stuff is stable enough to finish the work.
2. The performance does matter for script binding. I don't want more extral cost for data converting.
3. I want to publish games for iOS to App Store I don't think Apple allow dynamic link

[erodozer]

Just an idea something that might be interesting to explore wrt getting as much performance out of this as possible...

A lot of the godot built-in methods are there because for the convenience of GDScript, which is developed entirely within the closed ecosystem of Godot. Javascript has a lot of similar math functions as part of its language, or you can get them as npm modules. Because of the overhead of c interop when calling any godot bindings from javascript and adapting data to javascript data structures, in some cases it can be more performant to use an equivalent function that are plain javascript. This can be especially true if the functions are going to be called frequently, such as using Rand functions for procgen.

It could be interesting to see the performance difference of rebuilding parts of the godot API in equivalent native javascript versus their c function counterparts.

[Geequlim]

@nhydock I have thought about reducing the allocation by implementing the builtin type in JS. But there is an important problem. Can the performance of the QuickJS vm achieve the expectations? quickjs does not support JIT, and there are many computationally intensive functions in the built-in APIs, which may not achieve the ultimate goal. Doing so may perform well in some simple use case like acquiring members of vector2 such as bunnymark, but not as expected in real projects. So I think the huge workload of maintaining built-in types with JS is not worth it.

[erodozer]

a lot of the built-in types such as Vector2 and Basis I wouldn't touch, because those would need to always be converted back regardless since the engine heavily relies on those structs. I was more interested in a lot of the pure math related functions, namely everything that's just convenience methods in https://docs.godotengine.org/en/stable/classes/class_@gdscript.html

Since they're purely computational and for convenience, they may either be redundant to JS or suboptimal to use frequently with their interop overhead.

If someone did want to see if they could optimize their own project by rewriting the same functions directly in javascript there's nothing stopping them from making their own lib instead of calling to godot.

[Geequlim]

A new benchmark result after opimize performance (In both binding code and script code)

As the result shows in V1DrawTexture we got more bunnies in JavaScript than GDScript.

[fire]

Is this completed?

[fire]

As far as I know, quickjs has been implemented. Please open new issues for bugs and improvements.