GDNative support on HTML5 export

[kkimdev]

Please correct me if I'm wrong, but to my knowledge GDNative is not supported on html5 export(emscripten).

[Zylann]

This would require GDNative libraries to be compiled to WebAssembly, I guess?

[kkimdev]

I believe so. Also probably need some nontrivial work on actually hooking the wasm correctly.

[leonkrause]

Had a first look at this a few days ago. Emscripten has several relevant linker flags, including -s MAIN_MODULE=x for the main build and -s SIDE_MODULE=x for libs. Flags are documented in Emscripten's settings.js. Loading WASM modules can be done using loadWebAssemblyModule with a module buffer or loadDynamicLibrary with a URL.

There's a kanban-board on the Emscripten repo for dynamic linking related issues: https://github.com/kripken/emscripten/projects/2

Emscripten performs dead code elimination that we'll have to disable for this, I've yet to check what impact this will have on file size.

[matty]

Is this going to be worked on at all or planned?

[leonkrause]

Had a prototype working, but the WebAssembly binary file size grows by an unreasonable amount due to kripken/emscripten#5586. We're also hitting kripken/emscripten#5608.

In general WA bin growth should be small, since we export the GDNAPI explicitly and thus can build with -s MAIN_MODULE=2 (that is, with normal DCE)

[Type1J]

I'd really like this feature, but until it's available, it looks like a custom built engine is the only option.

[Zireael07]

Any progress/ideas on this?

[Type1J]

I'm not sure if there's any progress here. Until it's supported, GDNative can't be first class.

[bojidar-bg]

kripken/emscripten#5586 was closed around a month ago, maybe a better prototype might be doable now?

[Type1J]

Maybe so.

[garrett-hopper]

Does the web export work with GDNative now?

[Calinou]

@Jumblemuddle Not yet :slightly_smiling_face:

[Zireael07]

Is anyone working on it, seeing as the Emscripten backend issues are solved?

[Type1J]

I'd like to encourage this issue to be solved. Anyone else that wants to do the same, look here: https://www.bountysource.com/issues/50530202-gdnative-support-on-html5-export

[NHodgesVFX]

This would definitely be a great addition to gdnative.

[Calinou]

@NHodgesVFX Please don't bump issues without contributing significant new information; use the :+1: reaction button on the first post instead.

[ludamad]

Is there any current documentation for building a GDNative project for HTML5 export with access to the Godot source? I feel like this will remain a sensible option for a long time for performance reasons

[Waridley]

Are https://github.com/emscripten-core/emscripten/issues/5607 and https://github.com/emscripten-core/emscripten/issues/5608 blocking this, or is it possible to work around them? It's not encouraging that they are closed as wontfix due to being stale for over a year.

[neikeq]

That bot says you can re-open them if the issues are still relevant.

[gamedevsam]

I want to remind everyone that if you compile the engine yourself you can write native C++ modules to do whatever you want: https://docs.godotengine.org/en/latest/development/cpp/custom_modules_in_cpp.html

You'd then have to compile your own export templates that include your modified godot: https://docs.godotengine.org/en/3.2/development/compiling/compiling_for_web.html

Just because you can't use GDNative doesn't mean you can't write C++ and target web assembly.

[Waridley]

GDNative != C++

There are GDNative bindings for Rust, but no tutorials for creating custom Godot modules in Rust, for example. If I knew more about C++, I wouldn't be asking if this was possible to implement, I'd be trying to see if I could get it to work myself.

If it is a non-goal of GDNative to support WASM, that's fine, but then this issue should be closed and that made clear. Otherwise, I think it's worth discussing what it would actually take to get this working.

I'm just curious whether the work can even be done in Godot alone, or if it needs to be fixed in Emscripten.

[Type1J]

I had said above that a custom build is our only option at this time, and our additional code is in Rust.

It'a true that there's no tutorials on making that happen, and we have to have C++ code to glue our Rust code to the engine, and the Rust bindings for GDNative do take care of that, if we could use them.

The basic idea is that your main code needs to have a C interface in a static library:

extern "C" {
    fn some_function() {
        // ...
    }
}

From there, create a native module to be compiled with scons that calls those functions and links with the static library cargo creates in the targets directory, and have scons call cargo build or cargo build --release depending on the config given to scons. Also, you'll need to give a target triplet based on the platform given to scons, it would be more like cargo build --release --target=wasm32-unknown-emscripten for a WASM build, but the other platforms need their triplet explicitly specified for each value of platform that you are supporting.

I know this is just an outline for a tutorial, but, hopefully, it helps.

[Frontrider]

I'm not too well versed in those tools. Is this one of those "it's a pain in your butt and you'll spend more time fixing it than using it" or "set it up once and forget"?

[neikeq]

I'm not sure what the status is. As far as I see the intention is to do dynamic loading and that's blocking the progress for some reason. For those looking for alternatives, IMO there's a better option than writing an engine module, which is to link the GDNative libraries statically, albeit this would require extra steps after exporting the game. This is already done for iOS AFAIK, so with a few changes to the GDNative module it should work for WASM as well. I think the only catch is to make sure to use the same emsdk backend and version that Godot is built with. This is a good option while waiting for dynamic loading, and some will still prefer it after the latter is working as well.

[Type1J]

Iteration time is a big problem when GDNative isn't involved.

With GDNative I can make changes, cargo build, and switch (alt-tab) over to the Godot editor to press play.

Without GDNative, I have to quit the editor, scons platform="<whatever platform I'm using that day>", bin\godot (I ussually press tab to fill out the rest of the executable name here.), Load up my project again, then press play.

Also, bindings for GDNative are very idiomatic in Rust and C++. When making an engine module, Rust must drop to the C interface, which I haven't used. C++ is fine, and I have used that interface, but there are still differences from GDNative, but probably nothing a few #ifdefs couldn't solve. That being said, the build times are longer due to linking all Godot deps, plus your module, and the requirement to shutdown the engine for every small change just takes the fun out of it.

Getting GDNative working in WASM would make iteration faster for those "web only" problems.

[Type1J]

@Frontrider Once you get your build setup, you're "on rails" (It's "set it up once and forget".)

[Calinou]

Without GDNative, I have to quit the editor, scons platform="<whatever platform I'm using that day>", bin\godot (I ussually press tab to fill out the rest of the executable name here.), Load up my project again, then press play.

On Linux/macOS, if your module only has "runtime" functionality (i.e. it doesn't add nodes in the editor or add editor plugins), you don't need to do this as running the project will start a new process.

PS: You don't need to specify the scons platform since Godot 3.2, as it's automatically detected.

[Waridley]

@Calinou Except on Windows scons fails to write the executable unless you close the editor.

[Type1J]

As @Waridley said, Windows locks the file so it can't be overwritten when its a .exe file and currently being run.

[Calinou]

@Type1J I edited my comment since they replied :stuck_out_tongue:

[Type1J]

@Calinou I'd rather develop on Linux, anyway. 😄 [offtopic]I've been fighting vcpkg all morning (on a non-Godot C++ project), and that's supposed to be the "easy way" to get libraries on Windows.[/offtopic] From what I understood, dlopen() should work with Emscripten, now. I haven't tried in a while, but we may be able to open a .wasm file from GDScript in HTML5 at this point. If that's true, GDNative won't be far away, since I believe that a GDNative file is only a dynamic library with a few entry points assumed to exist.

[KoBeWi]

We are moving proposals to the Godot proposals repository, thus closing this. AFAIK this feature is already planned, so it's probably not necessary to create a new proposal.

[agausmann]

Is there another tracking issue for this that I can subscribe to, then?

EDIT: godotengine/godot-proposals#147 seems to be related