Importing dep containing .wasm files requires --allow-read for inner wasm dependencies

[leighmcculloch]

Version: Deno 2.0.0

👋🏻 When importing dependencies built by wasm-pack and that contain wasm dependencies, the --allow-read permission must be given to allow the package to read the wasm modules.

import init, { decode } from "npm:@stellar/stellar-xdr-json@0.0.4";
await init();
console.log(
    decode(
        "TransactionResult",
        "AAAAAAAAAGQAAAAAAAAAAQAAAAAAAAABAAAAAAAAAAA=",
    ),
);

$ deno main.ts
┏ ⚠️  Deno requests read access to "/Users/leighmcculloch/Library/Caches/deno/npm/registry.npmjs.org/@stellar/stellar-xdr-json/0.0.4/stellar_xdr_json_bg.wasm".
┠─ Requested by `fetch()` API.
┠─ Learn more at: https://docs.deno.com/go/--allow-read
┠─ Run again with --allow-read to bypass this prompt.
┗ Allow? [y/n/A] (y = yes, allow; n = no, deny; A = allow all read permissions) > 

🤔 On one hand the need to provide the permission seems reasonable, obvious even, the npm module is reading a file.

🧐 However, it also seems unreasonable, unnecessary, and harms the developer experience.

💭 The file being read is part of the dependency. When importing a dependency there is some expectation that the dependency will be read. Whilst the way the wasm is accessed is different to the js/ts file, that seems like an implementation detail. The act of importing an npm module should allow the dependency to read it's own files. There is little, ?nothing?, to gain by requiring permissions to control whether the dependency can read it's own artifacts.

❗ The status quo is a problem because it's not possible to write scripts that work across multiple systems to allow this permission in a way that's portable.

👀 On my own system I can change the command to the following, but for another user, a user on Windows or Linux, that command is no longer portable.

❯ deno --allow-read=$HOME/Library/Caches/deno/npm/registry.npmjs.org/@stellar/stellar-xdr-json/0.0.4/stellar_xdr_json_bg.wasm main.ts
{"fee_charged":100,"result":{"tx_success":[{"op_inner":{"payment":"success"}}]},"ext":"v0"}

[bartlomieju]

This has been a problem for the longest time. What's even more interesting is that requiring permission checks is not even the worst part. It's streaming or reading the WASM file.

The solution to this problem is https://github.com/denoland/deno/issues/2552 which we are working on and would like to ship in Deno 2.1. This would allow WASM files to be statically analyzable in the module graphs and thus not require to pass permission checks (just like statically analyzable dynamic imports). That said, existing libraries/build tools will need to adapt to it, before permission checks will not be necessary.

Though with proposals like https://github.com/tc39/proposal-source-phase-imports advancing to Stage 3 it's inevitable anyway.

[leighmcculloch]

What's even more interesting is that requiring permission checks is not even the worst part. It's streaming or reading the WASM file.

+1 There's a lot to improve here.

However, I think addressing the permission issue on its own would make a significant difference to the developer experience of using embedded wasm in deno projects. Anyone using wasm-pack has a work around for the other issues of streaming and fetching, as it generates code for variety of environments that works just fine across node and deno.

The issue I experience is largely when bringing those npm packages to deno. They work rather seamlessly with node, and function on deno, but much of the time require abandoning meaningful use of deno's permission system and resorting to a blanket --allow-all or --allow-read.

Would it be reasonable to automatically inject an --allow-read for all dependency files?

For example, in the case that a script imports npm:@stellar/stellar-xdr-json@0.0.4, then implicitly apply the following permission:

--allow-read=$HOME/Library/Caches/deno/npm/registry.npmjs.org/@stellar/stellar-xdr-json/0.0.4/

[bartlomieju]

For example, in the case that a script imports npm:@stellar/stellar-xdr-json@0.0.4, then implicitly apply the following permission:

--allow-read=$HOME/Library/Caches/deno/npm/registry.npmjs.org/@stellar/stellar-xdr-json/0.0.4/

Seems somewhat reasonable, but it will be a tough sell to the team. Anyway, I'll bring it up during the next design meeting and see if there's something we can do.