Agoric / agoric-sdk

monorepo for the Agoric Javascript smart contract platform
Apache License 2.0
327 stars 206 forks source link

build that tried to bundle fs failed while giving success code #5823

Closed erights closed 1 year ago

erights commented 2 years ago

I did a yarn && yarn build at the top of agoric-sdk. This actual failure of my yarn build is only on my local development copy. There is no reason think it is interesting. What is interesting is the masking of this error, which caused me to miss it for a long time. The end of my yarn build looks like

> @agoric/smart-wallet
$ exit 0

> @agoric/wallet-ui
$ yarn build:ses && yarn build:react
$ cp ../../../node_modules/ses/dist/lockdown.umd.js public/
$ react-app-rewired build
Creating an optimized production build...

Compiled successfully.

File sizes after gzip:

  618.13 kB (-4 B)  build/static/js/index.9aef649a.a7c4fdde.js
  618.11 kB (-1 B)  build/static/js/main.7ad6bd68.js
  43.82 kB          build/static/js/bridge.1298522f.d42c76ae.js
  7.47 kB           build/static/js/685.e989e6e8.chunk.js
  5.23 kB           build/static/js/200.1383e1d3.chunk.js
  4.15 kB           build/static/js/477.c0d80a0a.chunk.js
  2.94 kB           build/static/js/770.c2760126.chunk.js
  2.6 kB            build/static/js/624.0c94941c.chunk.js
  1.34 kB           build/static/css/index.9aef649a.8f42cb0d.css
  1.33 kB           build/static/css/main.8f42cb0d.css

The bundle size is significantly larger than recommended.
Consider reducing it with code splitting: https://goo.gl/9VhYWB
You can also analyze the project dependencies: https://goo.gl/LeUzfb

The project was built assuming it is hosted at /wallet/.
You can control this with the homepage field in your package.json.

The build folder is ready to be deployed.

Find out more about deployment here:

  https://cra.link/deployment

✨  Done in 94.39s.

However, on the way, in packages/inter-protocol, I see errors like

Expected "2022-07-25T23:11:21.371Z" is same as "2022-07-25T22:31:50.501Z"
/Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/inter-protocol/bundles add: runStake from /Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/inter-protocol/src/runStake/runStake.js
(Error#1)
Error#1: ENOENT: no such file or directory, stat '/Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/microtime/index'
Error: ENOENT: no such file or directory, stat '/Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/microtime/index'

(Error#2)
Error#2: ENOENT: no such file or directory, stat '/Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@agoric/babel-generator/lib/source-map'
Error: ENOENT: no such file or directory, stat '/Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@agoric/babel-generator/lib/source-map'

(Error#3)
Error#3: ENOENT: no such file or directory, stat '/Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@agoric/babel-generator/lib/printer'
Error: ENOENT: no such file or directory, stat '/Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@agoric/babel-generator/lib/printer'

after 250 of these, I finally get

TypeError#251: Failed to load module "./src/runStake/runStake.js" in package "file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/inter-protocol/" (100 underlying failures: Cannot find external module "child_process" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "crypto" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "os" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "perf_hooks" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "process" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "worker_threads" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "v8" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "vm" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "crypto" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "child_process" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "crypto" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/, Cannot find external module "url" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/, Cannot find external module "stream" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/swing-store/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/swing-store/, Cannot find external module "stream" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/SwingSet/, Cannot find external module "child_process" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/xsnap/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/xsnap/, Cannot find external module "os" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/xsnap/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/node_modules/@rollup/plugin-node-resolve/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/node_modules/@rollup/plugin-node-resolve/, Cannot find external module "url" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/node_modules/@rollup/plugin-node-resolve/, Cannot find external module "util" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/node_modules/@rollup/plugin-node-resolve/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "url" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/node_modules/@rollup/plugin-commonjs/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/node_modules/@rollup/plugin-commonjs/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/node-gyp-build/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/node-gyp-build/, Cannot find external module "os" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/node-gyp-build/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/swing-store/, Cannot find external module "crypto" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/swing-store/, Cannot find external module "stream" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/swing-store/, Cannot find external module "util" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/swing-store/, Cannot find external module "zlib" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/packages/swing-store/, Cannot find external module "events" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/lmdb/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/lmdb/, Cannot find external module "os" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/lmdb/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/tmp/, Cannot find external module "os" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/tmp/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/tmp/, Cannot find external module "crypto" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/tmp/, Cannot find external module "module" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/compartment-mapper/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@rollup/pluginutils/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "url" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "events" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/glob/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/glob/, Cannot find external module "util" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/glob/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/commondir/, Cannot find external module "url" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "assert" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "util" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/import-meta-resolve/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/lmdb/, Cannot find external module "url" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/lmdb/, Cannot find external module "assert" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/rimraf/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/rimraf/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/rimraf/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/fs.realpath/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/minimatch/, Cannot find external module "util" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/inherits/, Cannot find external module "util" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/glob/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/glob/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/glob/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/glob/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/better-sqlite3/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/better-sqlite3/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/resolve/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/resolve/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/resolve/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/resolve/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/node_modules/source-map/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@endo/bundle-source/node_modules/source-map/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/node-gyp-build-optional-packages/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/node-gyp-build-optional-packages/, Cannot find external module "module" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/node-gyp-build-optional-packages/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/fs.realpath/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/fs.realpath/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/bindings/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/bindings/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/picomatch/, Cannot find external module "fs" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/better-sqlite3/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/better-sqlite3/, Cannot find external module "util" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/better-sqlite3/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/resolve/, Cannot find external module "os" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/resolve/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/file-uri-to-path/, Cannot find external module "util" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/debug/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/picomatch/, Cannot find external module "path" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/picomatch/, Cannot find external module "os" in package file:///Users/markmiller/src/ongithub/agoric/agoric-sdk/node_modules/@babel/highlight/node_modules/supports-color/
erights commented 2 years ago

With everything else built, when I go into the inter-protocol directory and do a local yarn build I get exactly the same symptoms, including the successful error code at the end!

warner commented 2 years ago

We isolated this to an import graph that strayed into the Node.js builtins. The yarn build actually succeeded and was not the cause of any problems.

One new file in the contract's module graph did something like import { timerUtility } from '@agoric/swingset-vat', thinking to take advantage of packages/SwingSet/src/index.js re-exporting timerUtility from src/vats/timer/utility.js. A deep import of @agoric/swingset-vat/src/vats/timer/utility.js would have worked fine. But by (implicitly) importing index.js, the bundling process was obligated to follow every import from swingset's index.js, which included swingset's controller.js, which imports fs and other Node.js builtins (none of which have any source, of course). We don't have any sort of tree-shaking here: every file named by an import statement (even if only imported for side-effects) must be included in the bundle, transitively.

I filed endojs/endo#1238 about improving the error message to help us track down such problems faster (showing the path of modules between the entry point and the missing one, so we can spot the point at which it meanders into unexpected territory).

erights commented 1 year ago

@warner so it sounds like this can be closed?

dckc commented 1 year ago

yes; builds seem to be working now.