List pending todos

[fire]

Do you have some pending todos I can try fixing?

[fwsGonzo]

Yes, absolutely. I don't understand how to translate (let's say) a GDscript function call into a RISC-V vmcall. I'm also unsure what is the best way to go about it.

Is it possible to avoid GDscript, or is calling RISC-V from GDscript the way forward? I think the basics are already done, and as you can see it's very little code.

What's missing is:

• An interface back to Godot that will grow over time, eg. how to access and safely modify engine resources from the script
  • This can be implemented using system calls
  • It can also be implemented using custom instructions (faster, but requires build-time processing)
  • More dynamic approaches also exist but are hard to implement, and often require the RISC-V program to be written in C++ due to compile-time features
  • You can have a look at how WASM does it for a very slow and time-consuming, but fully dynamic way of doing it
• A plan to forward GDscript function calls into a RISC-V function
• What else am I missing?

Since I know very little about Godot, I can't make the overall design decisions well, simply because I don't understand how things are done!

[fire]

Godot Engine has laboriously construct a c-abi called gdextension so we can avoid the entire gdscript api and is generalizable across many c-abi languages.

A lot of work has been done to expose all scripting methods https://github.com/godotengine/godot-cpp/blob/master/gdextension/extension_api.json. At least all the documented menthods in the godot engine documentation. Even the non documented ones that are stubs.

https://github.com/godotengine/godot-cpp/blob/master/gdextension/gdextension_interface.h

I will post how Godot Engine "calls the shared library via the c symbol" in the next posts.

Edited:

Note that there's two apis. The precision=single and the precision=double. I use the precision=double api!

[fire]

We can avoid construction of a scripting language in gdextension via using something like https://github.com/Vahera/godot-orchestrator. I suggest delaying implementing gdscript in gdextension to a later date.

These are independent problems.

[fire]

https://github.com/godotengine/godot/blob/25de53e147a04ba15afc461b3ad4aa1884ff927d/core/extension/gdextension.cpp#L774

Error GDExtension::open_library(const String &p_path, const String &p_entry_symbol, Vector<SharedObject> *p_dependencies) { is the entry point into the gdextension system.

Note that it calls Error err = OS::get_singleton()->open_dynamic_library(abs_path, library, &data); which is directly equivalent to vmcalls I think.

[fwsGonzo]

https://github.com/godotengine/godot/blob/25de53e147a04ba15afc461b3ad4aa1884ff927d/core/extension/gdextension.cpp#L774

Error GDExtension::open_library(const String &p_path, const String &p_entry_symbol, Vector<SharedObject> *p_dependencies) { is the entry point into the gdextension system.

Note that it calls Error err = OS::get_singleton()->open_dynamic_library(abs_path, library, &data); which is directly equivalent to vmcalls I think.

No, that is dynamic linker loading - but that is also supported, although not as fast as a static executable. Once the loading has happened, vmcall can be used on the loaded binary (just like any other binary), but there is a need to calculate where the functions are.

[fire]

Here is the text book example of a gdextension addon. https://docs.godotengine.org/en/latest/tutorials/scripting/gdextension/gdextension_cpp_example.html

Here is a github using cmake https://github.com/asmaloney/GDExtensionTemplate

[fire]

err = OS::get_singleton()->get_dynamic_library_symbol_handle(library, p_entry_symbol, entry_funcptr, false);

I think the hardest part is defining what the api is, and not the conversion from dynamic linker loading or vmcalling.

Which the list is here https://github.com/godotengine/godot-cpp/blob/master/gdextension/gdextension_interface.h

[fire]

No, that is dynamic linker loading - but that is also supported, although not as fast as a static executable.

I'm not sure if we can arbitrary add a main() to the shared library to convert it to an executable. There's a separate project called libgodot, but I want to avoid that since it's not been stable yet.

Do you think making it a dynamic linker loading of a riscv shared library is ok? I think there's room to optimize architecturally here though.

[fire]

No, that is dynamic linker loading - but that is also supported, although not as fast as a static executable. Once the loading has happened, vmcall can be used on the loaded binary (just like any other binary), but there is a need to calculate where the functions are.

I think the gdextension api/abi, caches the location of all the functions after loading the gdextension shared library. How bad is copying that approach?

Like it loads based on the general kind of api and not like per function call in the api.json.

[fwsGonzo]

I see. Thinking about this, it seems doable, but the whole point of GDExtension is the native performance. libriscv has good performance when full binary translation is active, which requires compiling (or pre-compiling) a shared object (a .dll if you will) using a system compiler (usually on the system where its going to be used).

Without full binary translation, there's JIT which is not close to native - rather more like 2x interpreted. But it only works on some platforms. Linux is the easiest to get it to work on.

And finally there's interpreted. If we imagine that the functions we call are tiny and require low latency, then libriscv is the best choice, currently, but I imagine that GDExtension code can do some heavy processing. Not ideal work for an interpreter.

libriscv is, at least in its current form, mostly a replacement for GDScript itself.

[fire]

My problem is trying to compile gdextension for like 8 platform variants and some of them require code signing.

libriscv has good performance when full binary translation is active, which requires compiling (or pre-compiling) a shared object (a .dll if you will) using a system compiler (usually on the system where its going to be used).

My idea was we use an external system to compile the riscv binary -- Let's say github actions.

Then we store it on the cdn as a binary blob that players can fetch (think in spirit to the godot asset library, can be the literal godot asset library.)

[fire]

If we imagine that the functions we call are tiny and require low latency, then libriscv is the best choice, currently, but I imagine that GDExtension code can do some heavy processing. Not ideal work for an interpreter.

This is a great usecase, the idea we can use librisc to compile small functions into native code. https://www.erlang.org/doc/system/nif.html Like erlang nifs with a fallback in gdscript. I think this is 100% doable, but I don't think I can fully utilize it yet in my Social VR Networking platform (FOSS).

[fire]

I see. Thinking about this, it seems doable, but the whole point of GDExtension is the native performance. libriscv has good performance when full binary translation is active, which requires compiling (or pre-compiling) a shared object (a .dll if you will) using a system compiler (usually on the system where its going to be used).

Point of confusion.

Did you mean compiling riscv or compiling riscv as native for the platform?

[fwsGonzo]

Well, binary translation means taking the RISC-V instructions and generating compileable code (eg. an intermediary representation, or a full-blown language). My binary translator generates freestanding C code. That C code has to be compiled to a shared object using a system compiler, like GCC. Once that is done, you can have up to ~80% native performance in safely sandboxed code. More with some experimental features.

Without full binary translation, there is libtcc which I have only gotten to really work well on Linux so far. It is sometimes 2x faster than interpreted mode. CoreMark reached 7k vs 41k native (~17% native). But like I said, it will take some effort to support other platforms, eg. Windows. Other platforms don't allow JIT anyway, like Switch.

With interpreted mode the latency of vmcalls remain the same, but the performance suffers, and it can randomly suffer harder just by things being moved around in memory. Interpreters are infamously prone to slowdowns due to random changes in code, changing alignments.

[fwsGonzo]

I guess I should add one last thing: In the future you will be able to embed binary translated RISC-V into your project directly as code, instead of having to load it as a shared library. It makes it possible to use it on consoles, among other things. But not possible to load dynamically, then, as it's something you use for shipping / final build.

[fire]

So I could begin by using TCC on Mac, Windows and Linux. Then at some point run the RISCV -> C code -> operating system binary.

Do you know how difficult it is to make the TCC part better?

And like the console strategy, I can bundle common user-generated codes as translated binaries that are cached.

[fwsGonzo]

The way that I work on my game:

1. I work with TCC enabled, as it doesn't take any time to generate the faster code.
2. When I build the Windows client, I enable full-binary translation, but it can only load a .dll (it can't compile anything by itself). If it has no .dll it just uses interpreter mode instead. So it's not an error.
3. When the Windows client connects to my server, the server sends a cross-compiled .dll to the client, which loads it and enables the 8-9x faster RISC-V emulation
4. For systems where we can't use binary translation at all, the interpreter would make the most sense - although in the future, we will probably be able to embed the binary translation code directly, and just vmcall functions as regular functions. Since I already have working whole binary translation, this is not much work, but I still don't know all the details yet. I'm ~100% sure I can make it happen though.

That's the direction I've been going with my game.

[fwsGonzo]

So, when it comes to GD extensions that we can pre-build, perhaps it's good enough to transpile a GD extension to a RISC-V shared object, and then convert that to binary translated code that can be embedded in the game project? It would if nothing else be truly portable. It can be used on every system. You can't load a GD extension dynamically on Switch simply because it's not allowed. But if you can embed the fully binary translated GD extension in the project, then that solves a lot of problems.

So what's the catch? Well, I have never tried to load a shared object in libriscv before. Not sure what's supposed to happen. But it's definitely possible.

[fire]

My friend @shakesoda mentioned there has to be complications, but transpiling a GD Extension to a RISC-V shared object then converting that to a binary translate code embeddable everywhere Godot Engine lives sounds like an amazing dream.

[fwsGonzo]

Yes, complications are everywhere:

• Some extensions love to access filesystem
• OS resources has to be faked, like /proc/self/exe
• Some extensions are not written in easy-to-handle languages like C/C++
• Any extension that tries to load other DLLs is dead in the water - it needs to be rebuilt to not have dependencies other than linker-loader (ld64) and perhaps maybe libc.so. We can settle on a list of shared objects that is acceptable.

Of course, all the necessary support to load these shared objects still has to be implemented. libriscv is mostly a safe sandbox. It's not made to emulate full Linux programs, but at the same time I can make HTTPS requests in a Zig RISC-V executable, so it's also not impossible either. It just means that all parts have to be extended enough to support enough programs. It will never be 100%, and that's OK. The simplest method is to take things on a case-by-case basis.

For example, an order of things:

• Implement shared object loading, verify able to fetch address of dynamically loaded functions
• Verify shared object works with binary translation
• Output embeddable C code for shared object
• Verify embeddable C code works. Eg. run it on Windows.

Getting all the way to the end for this basic proof-of-concept would validate the general idea.

[fire]

One of my friend was asked me:

My main concern about RISC-V is performance

Any idea how to think about the RISC-V emulation performance?

[fire]

My thought-experiment was trying to make https://github.com/Vahera/godot-orchestrator work with emulated RISCV shared gdextension.

• riscv compiled on github actions for orchestrator
• Godot 4.3-beta2 support is broken, but the maintainer is working on it
• as far as I can tell it doesn’t load other dlls
• it doesn’t seem to call outside http calls outside of the godot engine apis
• Written in c++

[fire]

Implement shared object loading, verify able to fetch address of dynamically loaded functions

Would an echo command work here as the hello world sample?

I think listing all the symbols in the riscv shared library would be even easier first step.

[fwsGonzo]

Initial support for embedding programs pre-translated to sandboxed C/C++ code here: https://github.com/fwsGonzo/libriscv/pull/174

How to test:

• Execute ./rvlinux -o mycode.cpp
• bash build.sh --embed mycode.cpp
• Execute ./rvlinux

It won't work with dynamic executables right now because they have several execute segments, but it's just some missing typewriter work.

[fire]

Analysis from the gdextension team quoted:

Hm, well, GDExtension will only dlsym() the entry point funtion. However, the GDExtension API is full of function pointers that the GDExtension registers with Godot

So, for example, when a GDExtension registers a class, it calls classdb_register_extension_class3() passing in a GDExtensionClassCreationInfo3 struct, which the GDExtension needs to fill with (among other things) 18 different function pointers

Which Godot will then just call whenever it needs to, for example, in Object::get() it'll call the _extension->get function pointer, which is one of the ones passed on GDExtensionClassCreationInfo3

So, if the extension implemented in RISC-V can get function pointers and pass them to Godot that we can just call like normal function pointers, that should be fairly straight-forward to add to Godot

If they need to be called in a special way, though, then it'll require deeper changes, and it'd be tricky to make those changes in such a way that could be merged upstream