Worker bundle

[bmaranville]

Add a new build target: a bundled interface that accesses h5wasm through a Web Worker proxy.

Motivation

• access to WORKERFS, allows random access to files without reading entire file contents into memory
• HDF5 reads are effectively running in different thread than the page

Differences from standard library

• all access is async

Usage

import { save_to_workerfs, save_to_memfs, save_bytes_to_memfs, get_file_proxy } from 'h5wasm/worker/worker_proxy_bundle.js';

const loaded_files = [];
const workerfs_input = document.getElementById("save_to_workerfs");
const load_plugin_input = document.getElementById("load_plugin");
workerfs_input.addEventListener("change", async (event) => {
  const file = event.target.files[0];
  const filepath = await save_to_workerfs(file);
  loaded_files.push(filepath);
});
load_plugin_input.addEventListener("change", async (event) => {
  const file = event.target.files[0];
  const ab = await file.arrayBuffer();
  const bytes = new Uint8Array(ab);
  const filepath = await save_bytes_to_memfs(`/usr/local/hdf5/lib/plugin/${file.name}`, bytes);
  // console.log({filepath});
});

// ... load a local file called "water_224.h5" in file input
// ... load plugin libH5Zbshuf.so

const h5wasm_file_proxy = await get_file_proxy(loaded_files[0]); // loaded_files[0] === '/workerfs/water_224.h5'
root_keys = await h5wasm_file_proxy.keys();
// ['entry_0000']
const entry = await h5wasm_file_proxy.get('entry_0000');
// GroupProxy {proxy: Proxy(Function), file_id: 72057594037927938n}
await entry.keys()
// ['0_measurement', '1_integration', '2_cormap', '3_time_average', '4_azimuthal_integration', 'BM29', 'program_name', 'start_time', 'title', 'water']
dset = await entry.get('0_measurement/images')
await dset.metadata;
// {signed: true, type: 0, cset: -1, vlen: false, littleEndian: true, …}
await dset.shape;
// [10, 1043, 981]
s = await dset.slice([[0,1]]);
// Int32Array(1023183) [2, 0, 2, 0, 2, 1, 2, 2, 0, 0, 3, 0, 2, 4, 4, 1, 2, 3, 0, 1, 3, 0, 0, 3, 2, 4, 2, 7, 1, 1, 3, 3, 3, 2, 2, 2, 2, 0, 1, 6, 1, 1, 1, 1, 1, 2, 3, 1, 1, 2, 1, 3, 2, 1, 1, 0, 4, 1, 1, 2, 4, 6, 1, 0, 1, 7, 0, 2, 3, 1, 3, 1, 4, 2, 3, 0, 4, 0, 2, 3, 4, 2, 2, 1, 3, 2, 2, 1, 3, 4, 1, 1, 3, 1, 2, 2, 3, 2, 1, 2, …]
console.time('slice'); s = await dset.slice([[1,2]]); console.timeEnd('slice');
// slice: 37.31884765625 ms

[bmaranville]

This is a derivative product from h5wasm - @axelboc what do you think about making it a separate (but dependent) package (e.g. https://github.com/h5wasm/h5wasm-worker-proxy or something like that?)

It would make the packaging easier, and also simplify importing it into another project.

[axelboc]

Good call!

[bmaranville]

moved to https://github.com/h5wasm/h5wasm-worker ... maybe this will be the last move :)