Phew, this was a bit of a mind bender, but it's finally done :)
There's a lot of dark magic in this PR, so I'm going to go into a deeper explanation than I usually do.
What does this do?
This PR allows for us to define traits with any kind of method signatures, provided all inputs and outputs implement Serialize and Deserialize, and that there's a Default supertrait (for singleton initialization). Shims and accesors will be automagically generated to interface with a wasm module through that trait. I reworked the player plugins to showcase this with a more complicated interface involving vectors and strings.
How do we use this?
We use this in three places:
In the trait definition (wasm_wrap decorates the trait):
In the trait implementation (wasm_export decorates the impl block):
#[wasm_export]
impl Player for Wanderer {
//...
}
When calling the methods. For each method in the trait, a wasm_ prefixed global function is generated, that takes the same number of arguments plus two for a wasm store and instance:
let action = wasm_act(store, instance, tiles, last_result)
let name = wasm_name(&mut store, &instance)?;
How does it work?
The wasm_export procedural macro expands to shims like this:
#[no_mangle]
pub fn __wasm_shim_act(
argument_pointer_0: i32,
argument_pointer_1: i32,
argument_length_0: u32,
argument_length_1: u32,
) -> i32 {
let argument_slice_0 =
unsafe { ::std::slice::from_raw_parts(argument_pointer_0 as _, argument_length_0 as _) };
let reconstructed_argument_0 =
bomber_lib::bincode::deserialize(argument_slice_0).expect("Failed to deserialize argument");
let argument_slice_1 =
unsafe { ::std::slice::from_raw_parts(argument_pointer_1 as _, argument_length_1 as _) };
let reconstructed_argument_1 =
bomber_lib::bincode::deserialize(argument_slice_1).expect("Failed to deserialize argument");
let output = __WASM_SINGLETON
.lock()
.unwrap()
.act(reconstructed_argument_0, reconstructed_argument_1);
let serialized_output =
bomber_lib::bincode::serialize(&output).expect("Failed to serialize output");
unsafe {
__WASM_BUFFER
.iter_mut()
.zip(serialized_output.iter())
.for_each(|(o, i)| *o = *i);
}
serialized_output.len() as i32
}
They take pairs of indices to a static mutable buffer, from which they reconstruct the parameters to the inner wrapped method. They then delegate to the singleton (when appropriate), serialize the result back to the same buffer, and return the length of the buffer. This respects the wasm calling conventions regardless of the types, using the static buffer as a memory scratchpad to communicate with the caller. There is a lot of unsafe here, but this is the secure, sandboxed wasm environment so it's not a big risk.
Meanwhile, the wasm_wrap macro expands to accessors like this:
#[cfg(not(target_family = "wasm"))]
pub fn wasm_act(
store: &mut ::wasmtime::Store<()>,
instance: &::wasmtime::Instance,
surroundings: Vec<(Tile, Distance)>,
last_result: LastTurnResult,
) -> ::anyhow::Result<Action> {
let memory = instance
.get_memory(store.as_context_mut(), "memory")
.ok_or(::anyhow::private::new_adhoc("Wasm memory block not found"))?;
let get_wasm_buffer_address = instance
.get_typed_func::<(), i32, _>(store.as_context_mut(), "__wasm_get_buffer_address")?;
let wasm_buffer_base_address = get_wasm_buffer_address.call(store.as_context_mut(), ())?;
let mut wasm_buffer_address = wasm_buffer_base_address;
let surroundings = bincode::serialize(&surroundings)?;
let surroundings_address = wasm_buffer_address as usize;
let surroundings_length = surroundings.as_slice().len();
memory.write(
store.as_context_mut(),
surroundings_address,
surroundings.as_slice(),
)?;
wasm_buffer_address += surroundings_length as i32;
let last_result = bincode::serialize(&last_result)?;
let last_result_address = wasm_buffer_address as usize;
let last_result_length = last_result.as_slice().len();
memory.write(
store.as_context_mut(),
last_result_address,
last_result.as_slice(),
)?;
wasm_buffer_address += last_result_length as i32;
let method = instance.get_typed_func::<(i32, i32, i32, i32), i32, _>(
store.as_context_mut(),
"__wasm_shim_act",
)?;
let return_length = method.call(
store.as_context_mut(),
(
surroundings_address as _,
surroundings_length as _,
last_result_address as _,
last_result_length as _,
),
)?;
let mut dynamic_buffer = ::alloc::vec::from_elem(0u8, return_length as usize);
memory.read(
store.as_context_mut(),
wasm_buffer_base_address as usize,
dynamic_buffer.as_mut_slice(),
)?;
let result = bincode::deserialize(dynamic_buffer.as_slice())?;
Ok(result)
}
There's a lot going on, but it's fairly repetitive: It mainly sets up the wasm memory buffer with the appropriate serialized data, passes the indices and lengths, and deserializes the shim output using the returned buffer position. There's no unsafe at all on the calling layer, so there's no risk for undefined behaviour in the game.
All in all, it should hopefully make our future work on the player interface completely decoupled from wasm, so we can focus only on the game mechanics. Also, as far as I'm aware there's nothing the attendees could do to expose the plumbing, since the interface types are fixed, so as long as they use the right attribute macros they should be good to go.
Unrelated changes
Some of the game behaviour code has changed as I modified the examples a little to showcase how this works. Hopefully they feel a bit closer to the design we have in mind, but nothing is set on stone; after all the point of this PR is to make the player interface more flexible so we can quickly iterate on it.
Limitations
This could do with some more documentation and comments. My brain was a bit fried after wrangling procedural macros for most of the evening so I wanted to put it up for preliminary review :)
The memory area used to communicate the caller and wasm contexts is fixed size (10000 bytes for this iteration). This means the combined size of inputs to any function can't exceed 10000 bytes when serialized, and neither can the output. There are ways to improve this using wasm growable memory, but it would require a deep dive into wasm internals for what seems like little gain at this stage.
This can only work with trait blocks, and at the moment it can't help exposing shims for functions the players come up with. This is likely not a problem as we wouldn't know what to do with those anyway.
This also doesn't directly help giving the wasm modules callbacks into the calling context. I think this is mostly for the best, as callbacks will complicate the synchronization requirements on the caller side as multiple wasm modules could be accessing the same data concurrently. We can always manually implement certain imports lately for things like RNG or logging if we find them necessary.
Phew, this was a bit of a mind bender, but it's finally done :)
There's a lot of dark magic in this PR, so I'm going to go into a deeper explanation than I usually do.
What does this do?
This PR allows for us to define traits with any kind of method signatures, provided all inputs and outputs implement
Serialize
andDeserialize
, and that there's aDefault
supertrait (for singleton initialization). Shims and accesors will be automagically generated to interface with awasm
module through that trait. I reworked the player plugins to showcase this with a more complicated interface involving vectors and strings.How do we use this?
We use this in three places:
wasm_wrap
decorates the trait):wasm_export
decorates theimpl
block):wasm_
prefixed global function is generated, that takes the same number of arguments plus two for awasm
store and instance:How does it work?
The
wasm_export
procedural macro expands to shims like this:They take pairs of indices to a static mutable buffer, from which they reconstruct the parameters to the inner wrapped method. They then delegate to the singleton (when appropriate), serialize the result back to the same buffer, and return the length of the buffer. This respects the
wasm
calling conventions regardless of the types, using the static buffer as a memory scratchpad to communicate with the caller. There is a lot ofunsafe
here, but this is the secure, sandboxedwasm
environment so it's not a big risk.Meanwhile, the
wasm_wrap
macro expands to accessors like this:There's a lot going on, but it's fairly repetitive: It mainly sets up the
wasm
memory buffer with the appropriate serialized data, passes the indices and lengths, and deserializes the shim output using the returned buffer position. There's nounsafe
at all on the calling layer, so there's no risk for undefined behaviour in the game.All in all, it should hopefully make our future work on the player interface completely decoupled from wasm, so we can focus only on the game mechanics. Also, as far as I'm aware there's nothing the attendees could do to expose the plumbing, since the interface types are fixed, so as long as they use the right attribute macros they should be good to go.
Unrelated changes
Some of the game behaviour code has changed as I modified the examples a little to showcase how this works. Hopefully they feel a bit closer to the design we have in mind, but nothing is set on stone; after all the point of this PR is to make the player interface more flexible so we can quickly iterate on it.
Limitations
wasm
modules callbacks into the calling context. I think this is mostly for the best, as callbacks will complicate the synchronization requirements on the caller side as multiplewasm
modules could be accessing the same data concurrently. We can always manually implement certain imports lately for things like RNG or logging if we find them necessary.