wish: makeMarshal packaged for use in es2017 / Google Apps Script

[dckc]

I have re-implemented makeMarshal badly several times now due to friction around npm packages and modules and stuff vs just copy-and-paste of functions.

as long as using makeMarshal is harder than copy-and-paste-from-stack-overflow, we should expect folks to reimplement it. badly

could be as simple as running a bundler that supports tree shaking on an entrypoint consisting of makeMarshal() . A UMD file would probably suffice.

[dckc]

I tried the bundler approach and ran into lack of support for 0n literals and such.

In discussion with @erights , I'm learning that perhaps what I want is a specification of marshal consisting of something other than the existing code... something that facilitates high quality re-implementation.

I might expand the scope of this to "marshal specification for interop" or the like. A new issue might be better, but as long as I'm the only one using this issue, I might economize.

[kriskowal]

I tried the bundler approach and ran into lack of support for 0n literals and such.

Which bundler? If it’s one of ours, that’s a bug.

[dckc]

esbuild is the bundler I used. It worked well enough.

but the lack of support for 0n was in my current deployment target: google sheets.

[dckc]

inter-protocol/test/liq-bid-sheet is some code I kludged together to read the state of Inter Protocol contracts in a Google sheet.

boardSlottingMarshaller is the 3rd or 4th "poor man's marshal" I have written because of friction around import '@endo/marshal' vs. copy-and-paste-from-stack-overflow.

This issue was originally a wish for a single script to avoid module issues (Apps Script is not alone in having friction around modules; it lacks support completely). But after experimenting with esbuild to bundle makeMarshal I discovered that the V8 Runtime in  Apps Script doesn't support bigint literals... among the esbuild JS targets it seems to correspond to something around es2017.

I'm narrowing this issue to be about that specific use case and making a new issue for marshal specification for interop.

I think it's likely that this one will get closed as "out of scope" / "wontfix".

[dckc]

webpack and google-apps-script-starter: no joy

I tried using

• barretodavid/google-apps-script-starter version 00ec00a from Apr 5

My work is in inter-fun. The last commit I pushed was:

• WIP: flailing

I don't have very good notes on why it didn't work, but I have good news on an alternative approach...

[dckc]

independent implementation qualified with property-testing against reference impl

Prompted by another customer interested in using our stuff in Google Sheets, and inspired by arbPassable, I put together unmarshal.js, another quick-n-dirty independent implementation of the parts of makeMarshal that I need (only keys, and only unmarshalling), but this time it's qualified against the reference implementation using property-based testing:

1. start with an implementation that basically throws on everything
2. use the property test below to check compatibility with the reference implementation on arbitrary keys
3. when the test throws, fill in the necessary code
4. repeat until it stops failing

This isn't guaranteed to cover all cases, but it's a good way to get incrementally closer.

testProp(
  'marshal.unserialize re-implementation handles all keys',
  [arbKey],
  (t, k) => {
    // reference implementation
    const m = makeMarshal(makeProvideSlot(), undefined, smallcapsOpts);
    const { body, slots } = m.toCapData(k);

    // code under test
    const ctx = customMakeMarshal(undefined, makeProvideRemotable());

    const k2 = ctx.unserialize({ body, slots });
    const k3 = ctx.unserialize({ body, slots });
    t.deepEqual(k, k2, 'arbKey deepEqual 1st custom unmarshal');
    t.deepEqual(k, k3, 'arbKey deepEqual 2nd custom unserialize');
    t.true(keyEQ(k2, k3), '1st, 2nd custom unmarshal are keyEQ');
  }
);

https://github.com/dckc/inter-fun/blob/356da78498653f3190e9925ca574d0fe5d9c927e/test/test-unmarshal.js#L32-L49

[dckc]

@gibson042 notes related discussion in #1686 and points out that unmarshalling can work in a faked SES environment.

$ node --input-type=module -e '
  import "data:text/javascript;charset=utf-8,globalThis.harden||=Object.freeze;globalThis.assert||=()=>{};globalThis.assert.typeof||=()=>{};globalThis.assert.details||=String.raw;globalThis.assert.quote||=JSON.stringify;";
  import "@endo/eventual-send/shim.js";
  import { Far } from "@endo/far";
  import { makeMarshal } from "@endo/marshal";
  const m = makeMarshal(undefined, (id, iface = "") => Far(`${iface}<${id}>`));
  console.log(m.fromCapData({ body: `#{"bigint":"+42","remotable":"$0.Foo"}`, slots: ["id:a"] }));
'
{ bigint: 42n, remotable: Object [Alleged: Foo<id:a>] {} }