How to include data (webassembly binary) in a gdevelop extension?

[robehickman]

I am converting my existing Javascript pitch detection system to a gdevelop extension for a friend's project. This code depends on a webassembly module that does the actual pitch detection calculations (for best performance).

I've read the documentation and can't find anything about how to include a binary with an extension, or load it from the JavaScript side. I also don't see anything like that being done in any of the existing extensions. (I assume this would need special handling to work on different build targets).

I could base64 encode the file and inline it in the wasm setup code as a data url, but want to know if there is a better way of handling it.

https://github.com/4ian/GDevelop/tree/master/Extensions

I see that some extensions are written in C++, which I presume are compiled to webassembly automatically. However that doesn't help in my case as my wasm module was compiled from Rust code.

A second question I have relates to the AudioContext. Is there a way to obtain gdevelop's existing one, or is the preferred approach to create another one?

[4ian]

I see you just closed the issue - did you find the solution or opened the discussion somewhere else (forum/GitHub discussions?). Thank you!

[robehickman]

@4ian I also asked on the forum and someone there said that GD does not have a system for including data in extensions, and that it needs to be encoded into the source code.

https://forum.gdevelop.io/t/loading-webassembly-module-from-javascript-extension/39699/4

My question relating to the audio context is still unanswered

[4ian]

Yep there is for now no way to include files directly in extensions made in the editor. We could think of allowing arbitrary files, though I want to be sure not to give too much freedom here and end up with extension abusing it/making them hard to maintain.

For the audio context, I think you need to dig into what Howler.js, which we use for audio, is exposing. I think Howler.ctx might be what you look for. If useful, the sound manager wrapping Howler is accessible from the runtime game: https://docs.gdevelop.io/GDJS%20Runtime%20Documentation/classes/gdjs.RuntimeGame.html#getSoundManager

EDIT: seems you need to run at least a sound to have this context set up or do something to force its setup (https://stackoverflow.com/questions/46690346/react-and-howlerjs-howler-ctx-is-null). EDIT2: Also, this is not officially supported, so you might need to adapt how you get the context if this changes in a future version of GDevelop... though Howler and our usage of it has been pretty stable so far.

[robehickman]

@4ian Does this also apply to engine extensions in the 'Extensions' directory with JsExtension.js etc? That's what I was planning on doing, and I've been referencing https://github.com/4ian/GDevelop/tree/master/Extensions/ExampleJsExtension

I don't know how these differ from ones made in the IDE, are they effectively two different extension systems?

In that case for reliability creating another audio context may be the most reliable option. One is needed for reading from the microphone api.

You have a global 'Howler' singleton in GD, so that:

Howler.volume(1);

would work?

[4ian]

Does this also apply to engine extensions in the 'Extensions' directory with JsExtension.js etc? That's what I was planning on doing, and I've been referencing https://github.com/4ian/GDevelop/tree/master/Extensions/ExampleJsExtension

Well, extensions in the Extensions directory can specify any file to be imported when a game using one of the extension action/condition/object is exported - so it could work, but the file will be considered as a JavaScript file and put in a Githubissues.

Githubissues is a development platform for aggregating issues.