endojs / endo

Endo is a distributed secure JavaScript sandbox, based on SES
Apache License 2.0
761 stars 68 forks source link

feat(bundle-source): Zip original sources with --no-transforms #2294

Closed kriskowal closed 4 weeks ago

kriskowal commented 1 month ago

Closes: #2295 Refs: #400, #2252

Description

This change adds a mode to the bundle-source command with the initial flag --no-transforms that generates “endo zip base64” style bundles without applying the module-to-program transform and SES shim censorship evasion transforms, such that the original files on disk appear in the zip file. This is a preparatory step, necessary for building test artifacts, in advance of full support for this bundle style.

Security Considerations

bundle-source is part of the Endo and Agoric toolkit and it, or its surrogate, participate in the toolchain for generating content that can be confined by Hardened JavaScript, but is not trusted by Hardened JavaScript at runtime. It does however currently run with all the authority of the developer in their development environment and its integrity must be carefully guarded.

Scaling Considerations

No improvements expected at this time, but in pursuit of #400, it may be possible to move the heavy and performance sensitive JavaScript transform components from bundle-source to import-bundle and only suffer the performance cost of these transforms on Node.js, where those costs are more readily born by some runtimes. Precompiled bundles may continue to be the preferred medium for deployment to the web, for example.

Documentation Considerations

We will need to advertise the --no-transforms flag eventually, since there will be a period where it is advisable if not necessary to generate contracts and caplets targeting the XS runtime.

Testing Considerations

I have included a test that verifies the API behavior and manually run the following to verify behavior for the CLI:

rm -rf bundles
yarn bundle-source --no-transforms --cache-json bundles demo/circular/a.js circular-a
rm -rf circular-a
mkdir -p circular-a
jq -r .endoZipBase64 bundles/bundle-circular-a.json | base64 -d > circular-a/circular-a.zip
(cd circular-a; unzip circular-a.zip)
jq . circular-a/compartment-map.json
# verifying the final module entires have parser: 'mjs'

Compatibility Considerations

This flag is opt-in and breaks no prior behaviors. This introduces a new entry to the build cache meta-data and may cause some bundles to be regenerated one extra time after upgrading.

Upgrade Considerations

This should not impact upgrade, though it participates in the greater #400 story which will require xsnap upgrades to come to bear.

kriskowal commented 1 month ago

Looking for review buddies for work pursuant to XS native compartments, #400. This change touches bundle-source code I’ve shared maintainership with @dckc and compartment-mapper which has benefited tremendously from contributions from @naugtur and @boneskull, built on substantial work by @michaelfig. I would like to page @gibson042 into Compartment Mapper since it touches upon work we propose to TC39.