Create a dry run feature

[JesseAbram]

Allowing the runtime to be run in the browser would create a dry run feature that can be shipped and useful for checking that configs and aux data work for programs to be used.

This would mitigate users inputting incorrect fields and having to re set the configs (also would help in the programs side but that doesn't need to be done in the browser so is definitely easier and arguably already exists)

If we aren't able to run the runtime in the browser a third party api solution is possible but less ideal

[ameba23]

I had a go at 'transpiling' the template barebones program using the jco command line tool and it works great:

npm install -g @bytecodealliance/jco
jco transpile template_barebones.wasm --out-dir .

This gives us these files:

• template_barebones.core.wasm - the transpiled wasm blob
• template_barebones.d.ts - typescript interface
• template_barebones.js - js interface

To use it with node js i also had to make a package.json containing: { "type": "module" }

Then i added an index.js that looks like this: (sorry for my non-idiomatic javascript)

import('./template_barebones.js').then(({ evaluate }) => {
  function tryProgram(message, auxilaryData, config, oracledata) {
    let error
    try {
      evaluate({ message, auxilaryData }, config, oracledata)
    } catch (err) {
      error = err
    }
    console.log(error? error : 'Successful evaluation')
  }

  // Succeeds (this example will sign any message of at least 10 bytes)
  tryProgram(new Uint8Array(10))
  // Fails
  tryProgram(new Uint8Array(9))
})

I ran it like this:

$ node index.js
Successful evaluation
ComponentError: [object Object] (see error.payload)
    at evaluate (file:///home/turnip/radish/src/entropy/barebones-js/template_barebones.js:147:11)
    at tryProgram (file:///home/turnip/radish/src/entropy/barebones-js/index.js:15:7)
    at file:///home/turnip/radish/src/entropy/barebones-js/index.js:25:3 {
  payload: { tag: 'evaluation', val: 'Length of message is too short.' }
}

Which is the desired output.

You can see more clearly how to use it by looking at the typescript interface:

export interface SignatureRequest {
  message: Uint8Array,
  auxilaryData?: Uint8Array,
}
export type Error = ErrorInvalidSignatureRequest | ErrorEvaluation;
export interface ErrorInvalidSignatureRequest {
  tag: 'invalid-signature-request',
  val: string,
}
export interface ErrorEvaluation {
  tag: 'evaluation',
  val: string,
}
export function evaluate(signatureRequest: SignatureRequest, config: Uint8Array | undefined, oracleData: Uint8Array | undefined): void;
export function customHash(data: Uint8Array): Uint8Array | undefined;

What i can't figure out how to do is actually do the transpiling in JS without using the command line tool. I tried doing this:

import { transpile } from '@bytecodealliance/jco/component';
import { readFileSync } from 'fs'
let wasmBytes = readFileSync('./template_barebones.wasm');
transpile(new Uint8Array(wasmBytes)).then(files => {
  // this should give us the files from the bullet points above
})

I added jco as a dependency in the package.json, did npm install and then:

$ node index.js
file:///home/turnip/radish/src/entropy/barebones-js/node_modules/@bytecodealliance/jco/obj/js-component-bindgen-component.js:3478
  var {name: v2_0, noTypescript: v2_1, instantiation: v2_2, importBindings: v2_3, map: v2_4, compat: v2_5, noNodejsCompat: v2_6, base64Cutoff: v2_7, tlaCompat: v2_8, validLiftingOptimization: v2_9, tracing: v2_10, noNamespacedExports: v2_11, multiMemory: v2_12 } = arg1;
             ^

TypeError: Cannot destructure property 'name' of 'arg1' as it is undefined.
    at generate (file:///home/turnip/radish/src/entropy/barebones-js/node_modules/@bytecodealliance/jco/obj/js-component-bindgen-component.js:3478:14)
    at generate (file:///home/turnip/radish/src/entropy/barebones-js/node_modules/@bytecodealliance/jco/src/browser.js:5:20)

Node.js v18.15.0

I dont really understand whats wrong.

If someone who knows JS well (@mixmix ) wants to take a look, the api doc for the transpile function is in the readme here: https://github.com/bytecodealliance/jco/tree/main?tab=readme-ov-file#transpilecomponent-uint8array-opts-promise-files-recordstring-uint8array-

[ameba23]

Update i got transpiling to work in JS by bumping node to latest version (23.2.0) and adding the option wasiShim: false. I had to read the source to figure this out as the api docs are not very good.

import { transpile } from '@bytecodealliance/jco';
import { readFileSync } from 'fs'
let wasmBytes = readFileSync('./template_barebones.wasm');
transpile(new Uint8Array(wasmBytes), { wasiShim: false }).then((wit) => console.log(wit))

This will just dump out the files as Uint8Arrays:

$ node checkwit.mjs
{
  files: {
    'component.core.wasm': Uint8Array(21492) [
        0,  97, 115, 109,   1,   0,   0,   0,   1, 102,  14,  96,
        2, 127, 127,   0,  96,   3, 127, 127, 127,   1, 127,  96,
        2, 127, 127,   1, 127,  96,   0,   0,  96,   4, 127, 127,
      127, 127,   0,  96,  11, 127, 127, 127, 127, 127, 127, 127,
      127, 127, 127, 127,   1, 127,  96,   1, 127,   0,  96,   3,
      127, 127, 127,   0,  96,   4, 127, 127, 127, 127,   1, 127,
       96,   1, 127,   1, 127,  96,   6, 127, 127, 127, 127, 127,
      127,   0,  96,   6, 127, 127, 127, 127, 127, 127,   1, 127,
       96,   5, 127, 127,
      ... 21392 more items
    ],
    'component.d.ts': Uint8Array(538) [
      101, 120, 112, 111, 114, 116,  32, 105, 110, 116, 101, 114,
      102,  97,  99, 101,  32,  83, 105, 103, 110,  97, 116, 117,
      114, 101,  82, 101, 113, 117, 101, 115, 116,  32, 123,  10,
       32,  32, 109, 101, 115, 115,  97, 103, 101,  58,  32,  85,
      105, 110, 116,  56,  65, 114, 114,  97, 121,  44,  10,  32,
       32,  97, 117, 120, 105, 108,  97, 114, 121,  68,  97, 116,
       97,  63,  58,  32,  85, 105, 110, 116,  56,  65, 114, 114,
       97, 121,  44,  10, 125,  10, 101, 120, 112, 111, 114, 116,
       32, 116, 121, 112,
      ... 438 more items
    ],
    'component.js': Uint8Array(6300) [
       99, 108,  97, 115, 115,  32,  67, 111, 109, 112, 111, 110,
      101, 110, 116,  69, 114, 114, 111, 114,  32, 101, 120, 116,
      101, 110, 100, 115,  32,  69, 114, 114, 111, 114,  32, 123,
       10,  32,  32,  99, 111, 110, 115, 116, 114, 117,  99, 116,
      111, 114,  32,  40, 118,  97, 108, 117, 101,  41,  32, 123,
       10,  32,  32,  32,  32,  99, 111, 110, 115, 116,  32, 101,
      110, 117, 109, 101, 114,  97,  98, 108, 101,  32,  61,  32,
      116, 121, 112, 101, 111, 102,  32, 118,  97, 108, 117, 101,
       32,  33,  61,  61,
      ... 6200 more items
    ]
  },
  imports: [],
  exports: [ [ 'customHash', 'function' ], [ 'evaluate', 'function' ] ]
}

We still need to figure out how to dynamically import them in order to evaluate the program.

[FarafonovBogdan]

import { transpile } from '@bytecodealliance/jco';
import { readFileSync, writeFileSync, mkdirSync } from 'fs';
import { dirname } from 'path';

function ensureDirectoryExistence(filePath) {
  const dir = dirname(filePath);
  mkdirSync(dir, { recursive: true });
}

const wasmBytes = readFileSync('./add.wasm');

transpile(new Uint8Array(wasmBytes), {
  name: 'add', 
}).then(files => {

  for (const [filename, content] of Object.entries(files.files)) {
    const outputPath = `./dist/${filename}`;

    ensureDirectoryExistence(outputPath);

    writeFileSync(outputPath, content);
    console.log(`Generated file: ${outputPath}`);
  }
}).catch(err => {
  console.error('Error during transpile:', err);
});

[ameba23]

@FarafonovBogdan thank you, i tried this and it works great.

I am wondering if we can import the code without needing to write it to the filesystem first.

Eg: i can import and run javascript like this:

const moduleData = "export function hello() { console.log('hello') }"
const url = "data:text/javascript;base64," + btoa(moduleData)
import(url).then(mod => mod.hello())

But the problem is that the generated javascript from transpiling our program internally imports the wasm blob in the same way:

const module0 = fetchCompile(new URL('./add.core.wasm', import.meta.url));

const isNode = typeof process !== 'undefined' && process.versions && process.versions.node;
let _fs;
async function fetchCompile (url) {
  if (isNode) {
    _fs = _fs || await import('node:fs/promises');
    return WebAssembly.compile(await _fs.readFile(url));
  }
  return fetch(url).then(WebAssembly.compileStreaming);
}

So i guess in a nodejs context the only way to run the transpiled code without writing it to the filesystem would involve modifying the generated code.