Builds fail with web3

[odoe]

Bug / Enhancement

I was trying to use web3 in a Dojo app, but I get some odd errors.

With a vanilla dojo create app --name dojo-web3 --tsx, it will build but throws a runtime error. If I delete the package-lock.json and node_modules, I get a build time error.

Package Version: 7.0 & 7.0.5

Code

import Web3 from 'web3';

import routes from './routes';
import App from './App';

const registry = new Registry();
registerRouterInjector(routes, registry);

// this could be window.ethereum if a wallet browser installed
const web3 = new Web3('ws://localhost:8546');

sample repo

Expected behavior:

I would expect a build without errors and a workflow to build a Dapp with Dojo.

Actual behavior:

// dojo vanilla install
// builds no errors, fails in the browser
util.js:103 Uncaught (in promise) ReferenceError: Buffer is not defined
    at Object../node_modules/core-util-is/lib/util.js (util.js:103)
    at __webpack_require__ (bootstrap:68)
    at Object.<anonymous> (_stream_readable.js:67)
    at Object../node_modules/readable-stream/lib/_stream_readable.js (main.js:49474)
    at __webpack_require__ (bootstrap:68)
    at Object../node_modules/readable-stream/readable-browser.js (readable-browser.js:1)
    at __webpack_require__ (bootstrap:68)
    at Object.<anonymous> (response.js:3)
    at Object../node_modules/stream-http/lib/response.js (response.js:224)
    at __webpack_require__ (bootstrap:68)

// dojo after delete package-lock.json and node_modules and reinstall
./node_modules/stream-browserify/index.js
Module not found: Error: Can't resolve 'readable-stream/duplex.js' in '/home/odoe/dev/block
chain-dev/dojo-web3/node_modules/stream-browserify'
resolve 'readable-stream/duplex.js' in '/home/odoe/dev/blockchain-dev/dojo-web3/node_module
s/stream-browserify'
  Parsed request is a module
  using description file: /home/odoe/dev/blockchain-dev/dojo-web3/node_modules/stream-brows
erify/package.json (relative path: .)
    Field 'browser' doesn't contain a valid alias configuration
    resolve as module
      looking for modules in /home/odoe/dev/blockchain-dev/dojo-web3

Node Version: 14.16.0 NPM Version: 6.17.11

[denov]

I dug into this a bit to find that's it related to the web3 package requiring these

1. xhr2-cookies.XMLHttpRequest
2. http
3. https

you can see this by adding the following to the top of a new app

import * as http from 'http';
console.info(http) 

[denov]

this looks like it's a result of - https://github.com/webpack/changelog-v5/blob/master/README.md#automatic-nodejs-polyfills-removed

by hacking @dojo/cli-build-app/config/dev.config.js and adding at line 107. the line above return config

config.node: { Buffer: true }

i was able to get my app to load.

[denov]

I'd like to suggestion this change to src/dev.config.ts and src/dist.config.ts

if (Array.isArray(args.polyfills)) {
    config.node = args.polyfills.reduce(
        (polyfills: webpack.Node, module: string) => {
            polyfills[module] = true;
            return polyfills;
        },
        {} as webpack.Node
    );
}

so one could then add this to their .dojorc file so resolve this issue.

{
   'build-app':  { node: [ 'Buffer' ] }
}

my first stab (i couldn't get the linter to not complain about using reduce) https://github.com/dojo/cli-build-app/commit/a62b866a0001eb68c6012cf4583b1e2010ef1822

why?

1. without Dojo is somewhat validating the OCP principal of good software design. webpack has been abstracted in away one can not configure all parts without ejecting.
2. without Dojo has cut itself off from using any libraries that require the node.js polyfills. eg, http and web3
3. unless somebody passed values via .dojorc nothing happens, so unless one really needs it nothing happens. this seem in line with externals.