documenting usage of worldview with webpack

[soumith]

worldview has a dependency on draco3d, which has a wasm module for it's decoder.

webpack is one of the most common asset bundler in the javascript / frontend-development world. Webpack out of the box doesn't support emscripten-compiled wasm files.

The most popular React development tool Create React App also uses Webpack.

Using worldview inside any app that uses webpack fails most commonly with the following error while loading draco3d:

Module parse failed: magic header not detected
File was processed with these loaders:
 * ./node_modules/file-loader/dist/cjs.js
You may need an additional loader to handle the result of these loaders.
Error: magic header not detected

After several hours of understanding how create-react-app's build works, and then how webpack works and interacts with wasm files, and finding the relevant webpack issue: https://github.com/webpack/webpack/issues/7352

I ended up adding this to my webpack.config.js in the rules section

    rules: [
       /*...............*/
       // needed for draco3d loading.
       // see https://github.com/webpack/webpack/issues/7352 for more info
       {
         test: /draco_decoder\.wasm$/,
         type: "javascript/auto", // ← !!
         loader: "file-loader",
         options: {
           publicPath: "dist/"
         }
       },
       /*...............*/

Hope this helps the next person who is going to run into this.

[soumith]

closing because no further action is expected or requested from the devs

[pkulijing]

Hi @soumith, I just ran into the same problem as you mentioned in the issue. Adding the rule in webpack.config.js solved one of the errors, but I still fail to build a demo for worldview due to a few errors like these:

ERROR in ./node_modules/draco3d/draco_encoder_nodejs.js 31:275-288
Module not found: Error: Can't resolve 'fs' in '/home/lijing17/Developer/ddsdemo/demo-worldview/node_modules/draco3d'
 @ ./node_modules/draco3d/draco3d.js 5:26-59
 @ ./node_modules/regl-worldview/dist/index.esm.js 13:0-30 2775:9-36
 @ ./src/Example.js 2:0-62 4:42-51 4:92-97 31:40-44 31:86-90
 @ ./src/index.js 3:0-32 11:38-45

ERROR in ./node_modules/draco3d/draco_encoder_nodejs.js 31:203-226
Module not found: Error: Can't resolve 'path' in '/home/lijing17/Developer/ddsdemo/demo-worldview/node_modules/draco3d'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
        - add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
        - install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
        resolve.fallback: { "path": false }
 @ ./node_modules/draco3d/draco3d.js 5:26-59
 @ ./node_modules/regl-worldview/dist/index.esm.js 13:0-30 2775:9-36
 @ ./src/Example.js 2:0-62 4:42-51 4:92-97 31:40-44 31:86-90
 @ ./src/index.js 3:0-32 11:38-45

They seem to be related to using node modules like 'fs' and 'path' that are not applicable in browser environment. I modified my package.json according to this answer, and the build succeeded, but nothing is rendered in the browser and the following error is reported in the console:

Uncaught ReferenceError: process is not defined
    at getNodeEnv (webpack://demo-worldview/./node_modules/regl-worldview/dist/index.esm.js?:1422:3)
    at WorldviewContext.initialize (webpack://demo-worldview/./node_modules/regl-worldview/dist/index.esm.js?:2088:16)
    at WorldviewBase.componentDidMount (webpack://demo-worldview/./node_modules/regl-worldview/dist/index.esm.js?:2469:22)
    at commitLifeCycles (webpack://demo-worldview/./node_modules/react-dom/cjs/react-dom.development.js?:19814:22)
    at commitLayoutEffects (webpack://demo-worldview/./node_modules/react-dom/cjs/react-dom.development.js?:22803:7)
    at HTMLUnknownElement.callCallback (webpack://demo-worldview/./node_modules/react-dom/cjs/react-dom.development.js?:188:14)
    at Object.invokeGuardedCallbackDev (webpack://demo-worldview/./node_modules/react-dom/cjs/react-dom.development.js?:237:16)
    at invokeGuardedCallback (webpack://demo-worldview/./node_modules/react-dom/cjs/react-dom.development.js?:292:31)
    at commitRootImpl (webpack://demo-worldview/./node_modules/react-dom/cjs/react-dom.development.js?:22541:9)
    at unstable_runWithPriority (webpack://demo-worldview/./node_modules/scheduler/cjs/scheduler.development.js?:653:12)

I have no clue how to continue. Do you have any suggestions?

[pkulijing]

Ok I'm able to solve all the problems. Thanks for providing the hardest part of the solution. I totally agree with you on the necessity to document the usage of webpack! I shared mine in https://github.com/cruise-automation/webviz/issues/699