search

Agoric / agoric-sdk

monorepo for the Agoric Javascript smart contract platform
Apache License 2.0
327 stars 208 forks source link

`yarn create @agoric/dapp demo` fails with yarn 4 (nodeLinker pnp) #9852

Open 0xpatrickdev opened 3 months ago

0xpatrickdev commented 3 months ago

Describe the bug

yarn create @agoric/dapp demo fails with yarn 4. The error message suggests it might be due to Plug'n'Play (the default nodeLinker setting).

To Reproduce

Steps to reproduce the behavior:

  1. Ensure using node 18.18.x and yarn 4.x 
    
node -v
v18.18.2

yarn -v 4.3.1


2. Run
```sh
yarn create @agoric/dapp demo
  1. See error 
    
yarn create @agoric/dapp demo
➤ YN0000: · Yarn 4.3.1
➤ YN0000: ┌ Resolution step
➤ YN0085: │ + @agoric/create-dapp@npm:0.1.1-u16.2, @agoric/access-token@npm:0.4.22-upgrade-16a-dev-fb592e4.0, @agoric/assert@npm:0.6.1-upgrade-16a-dev-fb592e4.0, and 423 more.
➤ YN0000: └ Completed in 1s 361ms
➤ YN0000: ┌ Post-resolution validation
➤ YN0086: │ Some peer dependencies are incorrectly met by dependencies; run yarn explain peer-requirements for details.
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0013: │ 426 packages were added to the project (+ 121.3 MiB).
➤ YN0000: └ Completed in 0s 279ms
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0007: │ protobufjs@npm:6.11.4 must be built because it never has been before or the last one failed
➤ YN0007: │ better-sqlite3@npm:9.6.0 must be built because it never has been before or the last one failed
➤ YN0007: │ @agoric/xsnap@npm:0.14.3-upgrade-16a-dev-fb592e4.0 must be built because it never has been before or the last one failed
➤ YN0007: │ microtime@npm:3.1.1 must be built because it never has been before or the last one failed
➤ YN0000: └ Completed in 18s 786ms
➤ YN0000: · Done with warnings in 20s 477ms

/private/var/folders/tz/jycbd2h54t39vpjnypzwktc80000gn/T/xfs-8f80eeba/dlx-25296/.pnp.cjs:10434 Error.captureStackTrace(firstError); ^

Error: @agoric/create-dapp tried to access .bin, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

Required package: .bin (via ".bin/agoric") Required by: @agoric/create-dapp@npm:0.1.1-u16.2 (via /private/var/folders/tz/jycbd2h54t39vpjnypzwktc80000gn/T/xfs-8f80eeba/dlx-25296/.yarn/cache/@agoric-create-dapp-npm-0.1.1-u16.2-4cdbbfdecb-a5578b9b12.zip/node_modules/@agoric/create-dapp/src/)

Require stack:

Node.js v18.18.2



## Expected behavior
Users should be able to use `yarn create @agoric/dapp` with yarn 4

## Platform Environment
 - what OS are you using? what version of Node.js?
 OS X, Node 18.18. Also fails on 18.20

 - is there anything special/unusual about your platform?
 No
 - what version of the Agoric-SDK are you using? (run `git describe --tags --always`)
create-dapp depends on `"agoric": "^0.21.1"`
0xpatrickdev commented 3 months ago

I suspect the solve here is adding a .yarnrc.yml with nodeLinker: node-modules, but am unsure how to test this locally. (I am familiar with yarn link but not for something like yarn create, where a project/package.json doesn't yet exist).

I also suspect we will need to tell CI (which currently uses yarn 1.22.x) to run this with yarn 4.x, and also add a test like:

diff --git a/packages/create-dapp/test/sanity.test.js b/packages/create-dapp/test/sanity.test.js
index f4424d587..4e6395a68 100644
--- a/packages/create-dapp/test/sanity.test.js
+++ b/packages/create-dapp/test/sanity.test.js
@@ -15,4 +15,5 @@ test('sanity', async t => {
   t.is(await myMain(['--help']), 0, '--help exits zero');
   t.is(await myMain(['--version']), 0, '--version exits zero');
   t.is(await myMain(['--zorgar']), 1, 'unknown flag fails');
+  t.is(await myMain(['demo']), 0, 'create-dapp demo exits 0');
 });
0xpatrickdev commented 2 months ago

Note: https://github.com/Agoric/agoric-sdk/issues/451 is a potential solution, but probably the one with the longest lead time. It seems like we have two interim solutions, the second of which might close this issue:

  1. Document that developers should use yarn 1.22.x in Getting Started. See https://github.com/Agoric/documentation/issues/1174

    • Note: dapp-agoric-basics uses yarn 1.22.x at the time of writing, while dapp-agoric-basics seems to be using yarn 4.x. I'm not sure how this will affect things, but something to keep in mind

  2. Add a .yarnrc.yml with nodeLinker: node-modules to /packages/create-dapp, as mentioned above

mujahidkay commented 2 months ago

but am unsure how to test this locally.

I am currently using verdaccio to proxy a private locally-hosted registry and overriding the npmRegistryServer with it to actually pull packages from it.

mujahidkay commented 2 months ago

So just adding a .yarnrc.yml with nodeLinker: node-modules didn't work. I believe yarn4 has pnp globally enabled - so I tried to switch it to use node-modules explicitly (can use either yarn config set nodeLinker node-modules or put it in a .yarnrc.yml in the directory you are gonna run yarn create @agoric/dapp demo.

It got rid of the pnp error but it threw the following:

node:internal/errors:496
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/private/var/folders/5n/lpny95rx2mz3jwpr6vbbsvg80000gn/T/xfs-93f97866/dlx-84923/node_modules/@agoric/vats/tools/board-utils.js' imported from /private/var/folders/5n/lpny95rx2mz3jwpr6vbbsvg80000gn/T/xfs-93f97866/dlx-84923/node_modules/agoric/src/lib/format.js
    at new NodeError (node:internal/errors:405:5)
    at finalizeResolution (node:internal/modules/esm/resolve:327:11)
    at moduleResolve (node:internal/modules/esm/resolve:980:10)
    at defaultResolve (node:internal/modules/esm/resolve:1193:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:404:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:373:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:250:38)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:39)
    at link (node:internal/modules/esm/module_job:75:36) {
  url: 'file:///private/var/folders/5n/lpny95rx2mz3jwpr6vbbsvg80000gn/T/xfs-93f97866/dlx-84923/node_modules/@agoric/vats/tools/board-utils.js',
  code: 'ERR_MODULE_NOT_FOUND'
}

This^ was resolved by bumping agoric dependency to ^0.22.0-u16.2 in packages/create-dapp/package.json.

I believe the user triggering yarn create has to set their nodeLinker to node-modules using either of the two ways i mentioned above, and then run the yarn create command to be able to install the package and run the dapp successfully.

What do you think/suggest @0xpatrickdev ? With this^, we would need to update the docs accordingly as well to mention those steps.

0xpatrickdev commented 2 months ago

@mujahidkay thanks for the great analysis here. I responded in the PR

mhofman commented 2 months ago

It got rid of the pnp error but it threw the following:

I cannot reproduce this error, for me after setting the linker, it inits the dapp correctly

➤ YN0085: │ + @agoric/create-dapp@npm:0.1.1-u16.2, @agoric/access-token@npm:0.4.22-upgrade-16a-dev-fb592e4.0, @agoric/assert@npm:0.6.1-upgrade-16a-dev-fb592e4.0, and 423 more.

The fact that we ended up resolving to upgrade-16a-dev packages is not good, we need to revisit how we generate pre-release tags for dev branches.

mujahidkay commented 2 months ago

It got rid of the pnp error but it threw the following:

I cannot reproduce this error, for me after setting the linker, it inits the dapp correctly

➤ YN0085: │ + @agoric/create-dapp@npm:0.1.1-u16.2, @agoric/access-token@npm:0.4.22-upgrade-16a-dev-fb592e4.0, @agoric/assert@npm:0.6.1-upgrade-16a-dev-fb592e4.0, and 423 more.

The fact that we ended up resolving to upgrade-16a-dev packages is not good, we need to revisit how we generate pre-release tags for dev branches.

Thanks @mhofman for the insights! I think I understand what happened here. For me, it was pulling @agoric/create-dapp@npm:0.1.0 which resulted in the above error. Now, I'm assuming this shouldn't ever happen with npm registry and could have been due to a misconfigured localhost environment.

Do you know why the agoric-sdk/packages/create-dapp/package.json still points to 0.1.0 (different from what's on npm)?

mhofman commented 2 months ago

Do you know why the agoric-sdk/packages/create-dapp/package.json still points to 0.1.0 (different from what's on npm)?

Our release process currently does not have a well defined way to merge back the changes of the release branch into master, which means that we've not been able to incorporate the version and generated changelog changes of releases back into trunk.

It is partially blocked technically on https://github.com/Agoric/agoric-sdk/issues/9490, but we do need a process in place anyway.