Open ithinkicancode opened 1 year ago
Hi @ithinkicancode. You are correct that there currently isn't a test
subcommand.
The main reason for that is that it would be nice to test the component itself in a wasm runtime rather than running it as a native executable via the default cargo test harness.
Some design needs to go into a proper implementation of that, though.
i've been thinking about this too. one idea is that each function under test takes a component instance as input (ie an object that exports can be accessed through) and some custom #[test]
macro that takes in custom testing parameters (eg. wasi or no wasi, runtime features, etc).
syntax is wrong but something like
#[component::test(
wasi_enable = true,
)]
fn test_export_caller(component: Component) { // or whatever the proper type is
assert!(component.exports.call_generate_rand_nums(32).is_ok()); // calls `generate_rand_nums()`, required us to enable wasi or error otherwise
}
Currently, I've been having to do this by include_bytes!
the wasm blob that's generated by cargo component build
, instnatiate the blob, and then write tests around it.
@jakehemmerle, do you have an example of instantiating a wasm blob (from the build) and writing tests against it? I'd be interested in trying out this route. :) Would appreciate it!
A temporary solution is to use the CARGO_TARGET_<triple>_RUNNER
environment variable for that purpose. Assuming the latest wasmtime has been installed, you can run:
CARGO_TARGET_WASM32_WASI_RUNNER="wasmtime --preview2 --wasm-features=component-model" cargo test
Here is a more complete example.
@jakehemmerle, do you have an example of instantiating a wasm blob (from the build) and writing tests against it? I'd be interested in trying out this route. :) Would appreciate it!
@ithinkicancode
In this example, I instantiate a Wasm component with a Wasm runtime (in a wrapper) and then assert return values against its exported functions.
ec_runtime/tests/runtime.rs
/// Points to the `template-barebones` program binary.
const BAREBONES_COMPONENT_WASM: &[u8] = include_bytes!("../../target/wasm32-unknown-unknown/release/template_barebones.wasm");
use ec_runtime::{ Runtime, InitialState };
#[test]
fn test_barebones_component() {
let mut runtime = Runtime::new();
// The barebones example simply validates that the length of the data to be signed is greater than 10.
let longer_than_10 = "asdfasdfasdfasdf".to_string();
let initial_state = InitialState {
data: longer_than_10.into_bytes(),
};
let res = runtime.evaluate(BAREBONES_COMPONENT_WASM, &initial_state);
assert!(res.is_ok());
}
#[test]
fn test_barebones_component_fails_with_data_length_less_than_10() {
let mut runtime = Runtime::new();
// Since the barebones example verifies that the length of the data to be signed is greater than 10, this should fail.
let shorter_than_10 = "asdf".to_string();
let initial_state = InitialState {
data: shorter_than_10.into_bytes(),
};
let res = runtime.evaluate(BAREBONES_COMPONENT_WASM, &initial_state);
assert!(res.is_err());
}
The Wasm runtime looks like this:
ec_runtime/src/lib.rs
use wasmtime::{
component::{bindgen, Component, Linker},
Config, Engine, Store, Result,
};
use thiserror::Error;
/// Note, this is wasmtime's bindgen, not wit-bindgen (modules)
mod bindgen {
use super::bindgen;
bindgen!({
world: "program",
path: "../wit/application.wit"
});
}
pub use bindgen::{ Program, Error as ProgramError, InitialState };
/// Runtime `Error` type
#[derive(Debug, Error)]
pub enum RuntimeError {
/// Program bytecode is invalid.
#[error("Invalid bytecode")]
InvalidBytecode,
/// Runtime error during execution.
#[error("Runtime error: {0}")]
Runtime(ProgramError)
}
/// Runtime allows for the execution of programs. Instantiate with `Runtime::new()`.
pub struct Runtime {
engine: Engine,
linker: Linker<()>,
store: Store<()>,
}
impl Default for Runtime {
fn default() -> Self {
let mut config = Config::new();
config.wasm_component_model(true);
let engine = Engine::new(&config).unwrap();
let linker = Linker::new(&engine);
let store = Store::new(&engine, ());
Self {
engine,
linker,
store,
}
}
}
impl Runtime {
pub fn new() -> Self {
Self::default()
}
}
impl Runtime {
/// Evaluate a program with a given initial state.
pub fn evaluate(&mut self, program: &[u8], initial_state: &InitialState) -> Result<(), RuntimeError> {
let component = Component::from_binary(&self.engine, program).map_err(|_| RuntimeError::InvalidBytecode)?;
let (bindings, _) = Program::instantiate(&mut self.store, &component, &self.linker).map_err(|_| RuntimeError::InvalidBytecode)?;
// TODO fix this unwrap
bindings.call_evaluate(&mut self.store, initial_state).unwrap().map_err(|e| RuntimeError::Runtime(e))
}
}
The associated wit is:
package entropy:core
world program {
// similar to `variant`, but no type payloads
variant error {
invalid-transaction-request(string),
evaluation(string)
}
/// Contains signature request data that is used by the runtime. Passed into `wasmtime::Store` for state (or maybe `wasmtime::Linker`).
export evaluate: func(config: initial-state) -> result<_, error>
record initial-state {
/// The preimage of the user's data under program evaulation (eg. RLP-encoded ETH transaction request).
data: list<u8>
}
}
Currently there is no
test
command incargo-component
. The standardcargo test
doesn't work because the standard tooling has no visibility tobindings
,exports
or any generated structs and enums. I'd think it's a good devX that devs can follow the same workflow just like when writing regular Rust apps. Please point me to the right direction if I miss any docs wrt testing!