upgrade to Yarn 4

[turadg]

https://github.com/Agoric/agoric-sdk/pull/8461 is failing on integration with this repo

This PR upgrades Yarn from v1 to v4 and fixes issues that emerged with the stricter package resolution.

It also drops Node 16 which is EOL.

### Blocked on
- [ ] https://github.com/Agoric/agoric-sdk/issues/9209

[cloudflare-pages[bot]]

Deploying documentation with    Cloudflare Pages

Latest commit:	27b7c08
Status:	✅  Deploy successful!
Preview URL:	https://1dcdba8e.documentation-7tp.pages.dev
Branch Preview URL:	https://ta-yarn-4.documentation-7tp.pages.dev

View logs

[turadg]

This is failing agoric install because that command uses the --link-folder option and thus assumes Yarn version 1 for downstream projects.

$ yarn workspaces focus [--json] [--production] [-A,--all] ...
$ yarn workspaces foreach [--from #0] [-A,--all] [-R,--recursive] [-W,--worktree] [-v,--verbose] [-p,--parallel] [-i,--interlaced] [-j,--jobs #0] [-t,--topological] [--topological-dev] [--include #0] [--exclude #0] [--no-private] [--since] [-n,--dry-run] <commandName> ...
agoric: (Error#1)
Error#1: yarn install updates failed in .
  at yarnInstallEachWorktree (file:///home/runner/work/documentation/documentation/agoric-sdk/packages/agoric-cli/src/install.js:73:15)
  at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
  at async installMain (file:///home/runner/work/documentation/documentation/agoric-sdk/packages/agoric-cli/src/install.js:279:3)
  at async Command.parseAsync (/home/runner/work/documentation/documentation/agoric-sdk/node_modules/commander/lib/command.js:935:5)
  at async main (file:///home/runner/work/documentation/documentation/agoric-sdk/packages/agoric-cli/src/main.js:400:5)

This seems like an unnecessary coupling to me. I could make it check for the Yarn version but I wonder if @michaelfig has any better ideas.

[michaelfig]

seems to be failing

It would be great to port agoric install to whatever corepack would use, but I don't have much experience outside of Yarn 1, so it would take some time to gather the requisite knowledge to do it. I'd expect to see that put in priority on a project board somewhere.

[turadg]

Cloudflare build failure looks due to 9a487fc, not yarn. If other jobs pass then I'll rerun omitting that commit. cc @Jovonni @amessbee the agoric-sdk submodule is at https://github.com/Agoric/agoric-sdk/commit/03b6420649be77d010bc783efc5aaee5c7ed45e7 which is 4.5 months old.

[github-actions[bot]]

Cloudflare deployment logs are available here