Scripting API

[erlend-sh]

Description

Lots of higher level coders won’t be interested in modding unless we support a higher level scripting language, like Lua.

Scripts don’t even need to be able to run in production (although we will eventually figure that out); their primary function is iterative prototyping. A rust implementation might still be required for anything running in production.

Alternatives & Prior Art

• https://github.com/fishfight/FishFight/pull/394
• https://github.com/makspll/bevy_mod_scripting

hey all! The creator of bevy_mod_scripting here. Very happy to see interest in my baby 😂 The current state of the crate is fairly good, tealr support is already there with on the fly compiling to lua and hot-reloading + documentation generation. The biggest piece of functionality missing is currently the bevy API (i.e. get_component Vec2/Vec3 etc etc), and I am currently working on it full-time, I think bevy 0.8 (mid july) is a fairly realistic deadline for that feature and I'd recommend waiting for then to start using the package unless you do not need this and are rather looking for simple scripts which do not directly interact with the bevy World. Let me know if you have any questions, happy to help!

While still being a very limited language, Mun would also be super exciting to start playing with:

• https://github.com/mun-lang/mun/pull/416

---

Tasks

• [x] #140
• [x] #253

[virgilhuxley]

It doesn't look like Mun is ready for wasm targets at this moment: see gitlab issue https://github.com/mun-lang/mun/issues/215

I tried it anyways.

I inserted a basic mun example into the punchy world and it resulted in the following.

$ just build-web
cargo build --target wasm32-unknown-unknown
warning: unused manifest key: dependencies.mun_runtime.id
   Compiling mun_libloader v0.1.0 (https://github.com/mun-lang/mun#c5a9a259)
   Compiling bevy_asset v0.7.0
error[E0432]: unresolved import `libloading::Library`
 --> /home/virgil/.cargo/git/checkouts/mun-c902bbae864708cf/c5a9a25/crates/mun_libloader/src/temp_library.rs:5:5
  |
5 | use libloading::Library;
  |     ^^^^^^^^^^^^^^^^^^^ no `Library` in the root

error[E0412]: cannot find type `Symbol` in crate `libloading`
  --> /home/virgil/.cargo/git/checkouts/mun-c902bbae864708cf/c5a9a25/crates/mun_libloader/src/lib.rs:32:46
   |
32 |         let _get_abi_version_fn: libloading::Symbol<'_, extern "C" fn() -> u32> =
   |                                              ^^^^^^ not found in `libloading`

error[E0412]: cannot find type `Symbol` in crate `libloading`
  --> /home/virgil/.cargo/git/checkouts/mun-c902bbae864708cf/c5a9a25/crates/mun_libloader/src/lib.rs:35:39
   |
35 |         let _get_info_fn: libloading::Symbol<'_, extern "C" fn() -> abi::AssemblyInfo<'static>> =
   |                                       ^^^^^^ not found in `libloading`

error[E0412]: cannot find type `Symbol` in crate `libloading`
  --> /home/virgil/.cargo/git/checkouts/mun-c902bbae864708cf/c5a9a25/crates/mun_libloader/src/lib.rs:38:51
   |
38 |         let _set_allocator_handle_fn: libloading::Symbol<'_, extern "C" fn(*mut c_void)> = library
   |                                                   ^^^^^^ not found in `libloading`

error[E0412]: cannot find type `Symbol` in crate `libloading`
  --> /home/virgil/.cargo/git/checkouts/mun-c902bbae864708cf/c5a9a25/crates/mun_libloader/src/lib.rs:56:45
   |
56 |         let get_abi_version_fn: libloading::Symbol<'_, extern "C" fn() -> u32> = self
   |                                             ^^^^^^ not found in `libloading`

error[E0412]: cannot find type `Symbol` in crate `libloading`
  --> /home/virgil/.cargo/git/checkouts/mun-c902bbae864708cf/c5a9a25/crates/mun_libloader/src/lib.rs:72:38
   |
72 |         let get_info_fn: libloading::Symbol<extern "C" fn() -> abi::AssemblyInfo<'static>> = self
   |                                      ^^^^^^ not found in `libloading`

error[E0412]: cannot find type `Symbol` in crate `libloading`
  --> /home/virgil/.cargo/git/checkouts/mun-c902bbae864708cf/c5a9a25/crates/mun_libloader/src/lib.rs:89:50
   |
89 |         let set_allocator_handle_fn: libloading::Symbol<'_, extern "C" fn(*mut c_void)> = self
   |                                                  ^^^^^^ not found in `libloading`

Some errors have detailed explanations: E0412, E0432.
For more information about an error, try `rustc --explain E0412`.
error: could not compile `mun_libloader` due to 7 previous errors
warning: build failed, waiting for other jobs to finish...
error: Recipe `build-web` failed on line 17 with exit code 101

See the branch of "example/mun-example" in the following repo for a reference example of what it would look like to add a mun runtime to punchy. (reminder: This will not build to wasm32-unknown-unknown but it will build to x86) https://github.com/virgilhuxley/punchy/tree/example/mun-example

This only adds the runtime to the world. It does not invoke anything from mun. Update 2022-07-21: repo now has a very basic example of a working system that invokes a Mun function.

[erlend-sh]

Thanks!

We don’t actually need to support scripting for the web build, so making scripting exclusive to native builds is acceptable.

Scripts don’t even need to be able to run in production (although we will eventually figure that out); their primary function is iterative prototyping. A rust implementation might still be required for anything running in production.

[zicklag]

I'm going to try an experiment with scripting to see if we can have scripted systems that are able to modify any components in the world that implement Reflect.

I haven't looked super deep into bevy_mod_scripting yet but I want to see if we can get away with something super minimal that could still accomplish what we need today. This will just help me get familiar with what's possible and what's not, and we could still use bevy_mod_scripting once it allows interaction with the Bevy world. This is just some exploration.

As for language, I'm going to try Rust Python because you can create intuitive bindings to Rust objects with it and it will work in WASM out of the box. It's just the easiest commonly known language to experiment with that runs on native and web right now.

Another possible language that could be good would be JavaScript. We might be able to use Deno Core to support JavaScript on native platforms, and it would be awesome if we could get away with using the browser's native JavaScript engine for web support.

@erlend-sh, you said we don't need WASM support for plugins, and I know this is supposed to be more for quick prototyping, but I do want to at least explore what it could be if we used scripts possibly to implement even core items.

If it wasn't a performance or other kind of problem, I imagine that scripts could be a great way to implement items and maybe fighter moves, even in production. This would make it incredibly easy for people to mod the game, and they wouldn't have to think about this division between Rust core plugins and their mods so much.

Again, this is just a quick experiment and it might quickly come to nothing, but I think it's worth entertaining, as long as we put a limit on how long we spend on it before moving to more feasible options, if this turns out difficult.

[zicklag]

I haven't gotten super deep into this yet, but I think I'm leaning towards JavaScript/TypeScript for scripting.

I'm not super comfortable with the development status of RustPython. It seem like a great project but without a lot of developer bandwidth. Besides, JavaScript is probably way faster, especially if we re-use the browser's built-in engine instead of compiling Python to run in WASM, and it is arguably one of the most well-known languages of all time, for better or worse.

Also, we might get some fairly strong security features from embedding the V8 JavaScript engine on native platforms, which is a fairly big deal if we want to have a mod store one day where you might otherwise need to beware of malware in mods. Deno Deploy runs V8 Isolates for hundreds of users on shared hardware if I understand correctly. If V8 isolates can be secure enough to keep malicious code from impacting the server and other isolates, then I think that's good enough to keep mods from trying to break the user's computer. Though there could absolutely be things I'm unaware of.

That could be a big advantage, though, over a language like Lua which I believe usually has un-fettered access to your system.

Finally, I figured out that I can transpile TypeScript to JavaScript ( albeit without type checking ), even in the browser using SWC. That means that we can allow you to use TypeScript for writing mods without having a separate compile step, even when targeting the browser. And you can use your IDE such as VSCode for auto-completion and type checking.

[64kramsystem]

and it is arguably one of the most well-known languages of all time, for better or worse.

I'm in with Typescript :smile:

I'm personally neutral to it as a language, but after the success with Visual Studio Code, I recognize popularity as a big advantage.

[lenscas]

I'm in with Typescript 😄

Just a FYI, there is a pretty healthy Typescript to lua project https://github.com/TypeScriptToLua/TypeScriptToLua.

I personally also wouldn't assume that using the browser's JS engine will be good for performance due to the need to copy everything you want to share with JS (Or what JS wants to share with Rust). While normally you might have been able to get away with exposing a reference instead.

[zicklag]

I personally also wouldn't assume that using the browser's JS engine will be good for performance due to the need to copy everything you want to share with JS (Or what JS wants to share with Rust).

That's a good point. :+1:

In the strategy I'm working on, copies are only made when getting a specific value on a component, or when getting a specific value on a component, so at least it isn't trying to copy the whole world to JavaScript or anything like that, but if JavaScript, for instance, iterated through every component of a certain kind and read every single value of that component, it would end up copying all that data.

To avoid copying at the expense of a less-compliant JavaScript implementation we could try boa to run JavaScript in WASM.

Yet another option is to use c2rust to compile QuickJS to Rust, replace libc calls, then compile that to WASM as described here, but that would be a lot of work.

Just a FYI, there is a pretty healthy Typescript to lua project https://github.com/TypeScriptToLua/TypeScriptToLua.

Lua still isn't anything that we can run in the browser without a lot of extra binding/linking work of some sort, for all the strategies I've seen so far to do it, but it might not be harder than the QuickJS route so if we get that far it could be worth a look.

---

Anyway, I think the strongest argument in favor of JavaScrpt is the sandboxing support on native. If we have a reliable sandbox, then it makes a huge difference for our ability to provide a safe mod store in the future.

Either way, with the route I'm taking so far, the language we choose won't effect most of the design very much ( not yet anyway ), and we can change it later.

---

I made a little bit of progress, and I think that it should be simple enough to get scripts modifying components that implement Reflect, if the component already exists on an entity. Adding/removing entities and components won't be as simple, but we should be able to get stuff working if we get the basic concept down, and even if it isn't perfect, the mods should be able to do useful stuff earlier rather than later. :crossed_fingers:

[erlend-sh]

Sounds promising!

Regarding language choice and implementation for scripting, we’re fine with exploring multiple different paths here. There might be more than one right way to do scripting, with different constraints to solve for in web vs local, or modding vs prototyping.

On the topic of JavaScript and TypeScript, I’d also count Dart as an attractive alternative, as it can also compile to JS. It has a healthy gamedev community at this point: https://github.com/flame-engine/flame

For Lua in the browser, there’s this: https://github.com/ceifa/wasmoon

[lenscas]

For Lua in the browser, there’s this: https://github.com/ceifa/wasmoon

There are quite a few projects to get a lua VM in the browser. Some that are an entire lua implementation in JS and others that compile lua to wasm. However, none of them are working with mlua/rlua at this point. https://github.com/khvzak/mlua/issues/23 for the issue about it.

Funnily enough, it seems that someone managed to get mlua + luau working (or at least semi working?) not too long ago but it requires wasm32-unknown-emscripten as the target rather than the wasm32-unknown-unknown that the Rust ecosystem (for lack of better words) prefers.

A big pain point that this disconnect creates is:

a lot of crates usually only support wasm32-unknown-unknown and seems like winit does the same crying_cat_face which makes this useless for embedding in apps / games.

[zicklag]

A big pain point that this disconnect creates is:

Yeah, that's the issue. We really need WASM linking so that we can link a WASM build of lua such as wasmoon with a separately built Rust WASM binary. There are some experiments for that that we should look into if we want to pursue that.

---

There might be more than one right way to do scripting

:+1: :+1: Totally agree. For development bandwidth we'll want to focus on a middle-ground of what we can get working and what solves the most problems, but at the same time we can probably extend it later and support plenty of different scripting languages if we we really want to.

Just got to try stuff out and find out what we like.

---

I've been on vacation so I haven't given this a lot of focus time yet, but I'm still making progress here and there, and hopefully I'll have a script modifying system components soon.

[zicklag]

Got the first Punchy script working!

let counter = 0;

let systems = {
    update() {
        Punchy.log(`Script update #${counter}`, "trace");
        counter++;
    },
};

systems;

And it has auto-completion and type checking with the typescript definitions working. You'll be able to specify different functions such as update(), preUpdate() whatever other hooks we want you to be able to implement. The log function actually logs to the Bevy log at the specified log level.

Next I'll pass in the scriptable components and entities as an argument to the update() function, so that we should be able to actually modify the Bevy world.

Even once we've gotten that working there's still a lot to consider and figure out.

For instance, I can allow you to modify the translation, rotation, and scale of objects by binding to the transform component, but that doesn't mean you can call any functions on Transform necessarily. We may have to manually create bindings for many common functions vectors or stuff like that.

But we'll figure it out as we go, we're making progress!

[zicklag]

Just got a big task done for scripting: I reworked the script runner to be able to asynchronously load modules and to use JavaScript modules instead of standalone scripts.

This means that scripts can now use the import and export syntax ( though I still have to implement import ) and will allow us to enable dependencies on other scripts using URLs, like in the Deno or the browser, with no need for a package manager!

This should make it easier to make more advanced scripts with dependencies on other code, and dependencies will be asynchronously downloaded over the network if necessary or can be packaged locally with the rest of the game assets.

Also, I got the scripting engine implementation moved behind a scripting API trait that should make it easier to implement different scripting engines later.

Here's what the latest script looks like:

class Test1 implements ScriptSystems {
  counter: number = 0;

  update() {
    if (this.counter == 0) {
      Punchy.log("First update for my script!", "info");
    }

    Punchy.log(`Script update #${this.counter}`, "trace");
    this.counter++;
  }
}

export default new Test1();

[zicklag]

Just realized that the browser won't be able to load TypeScript modules through URLs as normal modules, so I'm postponing dependencies. In the future we have two options.

1. We bundle scripts and all their dependencies right in the browser! This could increase game loading times because it might be slow, but I think it's possible.
2. We build a subcommand into the punchy CLI that will automatically pre-bundle the dependencies for typescript files and prepare it for the browser. This would probably be a good option to pursue and would end up with rather nice trade-offs:
   • You can use TypeScript or JavaScript on native and in the browser without a build step, as long as you don't have any import statements.
   • If you want to import dependencies in scripts, you can use the punchy CLI to bundle the dependencies into a script bundle that can be efficiently loaded in the browser.

That probably wouldn't be very difficult to implement, and you could probably use the deno bundle command of the Deno CLI to get similar functionality today without any work on our part, but for now, we probably aren't going to even need dependencies, we just want simple scripts, so we'll not worry about it.

---

Now I'm working on getting the script runner to work in the browser.

[lenscas]

If you do plan to have scripts be executed b6 the js vm of the browser then I advise you to make it work first and to then measure the cost of ffi before brainstorming about dependencies.

Like I said before ffi is expensive if running this way so I wouldn't worry about the dependency thing yet too much.

Also, rather than allowing every url as a dependency why not restrict it to just a server owned by spicy lobster studio? That way basic code scanning is possible to implement if it proves to be desired and any transformations needed to the code to make it work everywhere can be done by that server rather than needing to be bundled with said target.

[lenscas]

Also, keep in mind that if the scripts run in the browser that they will have access to cookies, local storage, etc. So it is probably best to also see how to properly sandbox it from that.

[zicklag]

Also, rather than allowing every url as a dependency why not restrict it to just a server owned by spicy lobster studio?

The idea is that anybody can host their own punchy asset dir that can be played by the punchy web player, so that you can essentially push some assets to a Git repo, enable GitHub pages, and then instantly share a link to your game with anybody. This should actually work today with a link similar to this ( though I haven't tested yet, the code is there to make that work ):

https://fishfight.github.io/punchy/player/v0.0.3/?asset_url=https://zicklag.github.io/my_punchy_game

So if somebody wanted to host their own asset server that did automatic transformations of source files ( including spicy lobster ) that would be totally possible.

Also, keep in mind that if the scripts run in the browser that they will have access to cookies, local storage, etc. So it is probably best to also see how to properly sandbox it from that.

That's a good point. I think the only way to really sandbox that, though is to host it on it's own domain, which we should actually do, now that you mention it. It's similar to the concept of githubusercontent.com. Anything your users could influence should be put on a totally separate domain from anything else so that if it manages to set/get cookies etc. it can only impact other sites on that domain. i.e. That way we limit any possible damage malicious mods or scripts could cause to the game itself, and no other websites.

Another alternative that I think might be almost as secure ( or just as secure, I'd have to do some research ) is to allocate a sub-domain for the web player, as long as we are aware that anything on a sub-domain of that sub-domain could be potentially accessed by the game and it's potentially malicious mods.

Anyway, as long as mods can only maliciously impact the game itself, that's totally fine. The browser is repsonsible for protecting the user's system.

---

But yeah, I'm actually migrating back to using just plain scripts instead of modules now because of browser loading difficulties and the fact that we won't really need it. I think I'm getting close to a working MVP.

[zicklag]

Bevy 0.8.0 was released! And it has extra stuff needed for complete scripting support. And @jakobhellermann has a work-in-progress TypeScript plugin that looks like it has much more access to the Bevy API in the scripting interface than I thought anybody had yet.

It uses Deno core just like I was using too. We might be able to borrow from it, contribute to it, or otherwise collaborate. I'll have to have a look at what it's doing exactly.

https://github.com/jakobhellermann/bevy_mod_js_scripting

[odecay]

For anyone that wants a bit more context on the change in 0.8 that enables this https://bevyengine.org/news/bevy-0-8/#scripting-modding-progress-untyped-ecs-apis

It would be really cool if we could collaborate with bevy_mod_js_scripting if it has enough similarity to what youve been working on setting up.

[zicklag]

Just got the same script running in the browser and on native without changes! Now I've just got to get basic component access implemented and we should be able to get our first script working. :crossed_fingers:

---

It would be really cool if we could collaborate with bevy_mod_js_scripting if it has enough similarity to what youve been working on setting up.

It's very similar from what I can tell so far and we should be able to borrow the most important pieces. The differences center around how we are trying to get JavaScript to work in the browser without changes, but that's also the part that's "least important" and nearly independent of Bevy itself.

[erlend-sh]

New tech for Rust scripting: https://robert.kra.hn/posts/hot-reloading-rust/

[zicklag]

After 5 years of trying to get modding into a game engine, since the original time I tried to do it in Godot, I've finally moved the camera in a Bevy game, using a sandboxed TypeScript file, by directly accessing the ECS!

That's really exciting. 🥳

Also, it sounds like we're pretty much on the same page on browser support for scripting as bevy_mod_js_scripting: https://github.com/jakobhellermann/bevy_mod_js_scripting/issues/6.

So I might be able to contribute to that in order to get browser support working, and we can just add it as a dependency. I've read through most of it's code and I'm really happy with the way it's implemented, so it'd be great to contribute instead of copying a chunk of it into Punchy.

[zicklag]

Almost finished with the scripting implementation for the browser!

Here's a script incrementing the score in the browser every second.

Now I've just got to implement one last browser operation, clean up the code, and get https://github.com/jakobhellermann/bevy_mod_js_scripting/pull/11 ready to merge!

[erlend-sh]

Where’s this at? Did the browser present some additional challenges?

Keep in mind that we don’t have to support the browser use case for the v1 of this feature. Seems to me like we’ve already got what we need to roll out local scripting, right?

[zicklag]

I'm actually just waiting on review for https://github.com/jakobhellermann/bevy_mod_js_scripting/pull/11.

Everything other than module loading, which is non-essential for MVP, is working on both native and browser.

I can technically start working on finishing off the Punchy scripting MVP using my branch of bevy-mod-js-scripting, but I had some non-Punchy stuff that I've been busy with since last week.

[zicklag]

I'll push the code tomorrow, or later tonight if I get the chance, but I just got a working scripted, health item!! It replaces our current health item with the hard coded item type.

I can't believe it's working! The advantage of a script for a health item can be immediately seen in simple ways like the fact that instead of telling the item to refill 100000 health so that it goes to full, we can actually query the max health of the player and set the health to that.

Or we could easily modify it to add 25% of the player's max health, which would previously be impossible without adding more YAML attributes that are hard-coded into the game. This doesn't mean we can just "script all the things" yet, because there are a lot of holes in the functionality and things that we have to work around, but it's a huge step, and it looks like the future is full of possibilities for how to improve.

health.item.yaml:

name: Health

image:
  image: health.png
  image_size: [22, 20]

kind: !Script
  script: ./health.ts

health.ts:

type Health = {
  0: number;
};
const Health: BevyType<Health> = { typeName: "punchy::damage::Health" };
type Stats = {
  max_health: i32,
  movement_speed: f32,
};
const Stats: BevyType<Stats> = { typeName: "punchy::fighter::Stats" };

export default {
  post_update() {
    const grabEvents = punchy.getItemGrabEvents();

    for (const event of grabEvents) {
      const fighter = event.fighter;

      const [health, stats] = world.get(fighter, Health, Stats);

      health[0] = stats.max_health;
    }
  },
};