Refactor Mind-AR to ES Modules format. Decouple ThreeJS

[jmschrack]

In reference to: https://github.com/hiukim/mind-ar-js/issues/104

I set ThreeJS as a peerDependency:{ "three": ">=0.136.0"} in package.json and marked 'three' as external. This should allow people to use their chosen version of ThreeJS. Since ThreeJS does not follow SemVer conventions, it's always possible that a future update of ThreeJS could break stuff. We'll just worry about that when it becomes a problem.

I've refactored the code base to be ES Module compliant as this gives us a lot of benefits:

• we can use the new Webpack5 method for using workers since 'worker-loader' was deprecated.
• Webpack can use ESBuild's "tree-shaking" for optimizing dependencies.
• better compatibility with ThreeJS as they have gone the ESM route as well

Breaking Changes:

• For people using script tags to import Mind-AR, they will need to do the same for ThreeJS
• The reference to window.MINDAR.IMAGE/FACE.THREE has been removed.

[jmschrack]

If you're wondering why some of the commits mention Vite, I originally forked this to use Vite instead. But decided to commit the ESM work back to the main branch for Webpack5.

[hiukim]

@jmschrack

Thanks for the PR! Looking forward to migrate it to ES6 module.

Did a quick check. We might have an issue with the face tracking examples. (using <script> tag)

I tried to change it to import cv from '../libs/opencv.js', but that doesn't work because the opencv.js is built externally, and there is no export.

I'm not sure what's the proper way to handle this.

[jmschrack]

I recall running into this elsewhere on another project. I'll check it out and see if I can overcome it.

[jmschrack]

@hiukim I fixed opencv.js and pushed the fix up.

However, I discovered that webpack externals don't like using source maps in the browser which is what ThreeJS has moved to. I'll look into it some more.

[jmschrack]

@hiukim Alright, I finally solved the issue with webpack5 and external ThreeJS in the browser. Turns out that ES Module support is considered "experimental" in Webpack5 and it took a fair amount of trial and error to find the right combination of config settings to get it to work.

Might I suggest switching away from webpack to Vite instead? It uses Rollup.js which has better support for building libraries. https://github.com/jmschrack/mind-ar-js/tree/vite

[hiukim]

@jmschrack thx for the update. Did you have a chance to try any of the examples? none of them is working for me now.

image tracking examples:

face tracking examples: It doesn't load, the mindar-face aframe component doesn't init I guess, maybe because of type=module. not entirely sure yet.

Just tried your vite branch. Same issue. I'm open to switching to that though if it's better.

[jmschrack]

Strange... All of the face-tracking examples work for me except Example1. (Albeit looks like i missed a type="module" tag on one of my commits. ) The Image-Tracking examples break randomly for me. Sometimes working sometimes not.

After messing around with it for a while, it seems there is a race condition of some sort between modules and AFrame.

When they do work, I get a __webpack_modules__[moduleId] is not a function error which I can't seem to dig up any info on other than this: https://github.com/webpack/webpack/issues/13951 Which is funny to me because both of them stated they solved the issue by switching to Rollup.js which is what Vite uses.

[jmschrack]

@hiukim So after digging more into the issue more, it's a catch-22.

AFrame needs to the code to be non-module script in the header of the HTML file. This way all of the custom MindAR systems/components get registered with AFrame before the body section of the HTML file gets processed by the DOM. That's perfectly doable with Webpack, but not with Vite/Rollup.

However, ThreeJS has gone the modules only route via SourceMaps which Webpack does not support with externals. (And even experiments.outputModule=true fails on various parts when trying to use WebWorkers)

I made an experimental solution that uses Vite to generate ES and CJS libraries for the the core framework and ThreeJS. The Aframe implementation references the built CJS library (instead of the window.MINDAR variables) and then Webpack is used to package up the aframe stuff. I tested and all examples are working for me.

https://github.com/jmschrack/mind-ar-js/tree/esm-webpack-vite-hybrid (make sure you run npm install first!)

[hiukim]

@jmschrack I did a quick trial on your esm branch: https://github.com/jmschrack/mind-ar-js/tree/esm and seems like all examples work, except the image-tracking three.html example

All aframe examples seems good to me with your multi-webpack configs. Maybe we don't need the vite-webpack hybrid?

[jmschrack]

@hiukim Haha, you make it sound like it's 80% of the way there! 👍 But 4 out of 5 of your examples use A-Frame though, so to me it's only 50% there. 👎 I use ThreeJS every day professionally, so I'm biased to ensuring that it works as well. ;)

The problems with A-Frame and ThreeJS are this:

• ThreeJS requires module support. Which as of Webpack 5.75.0 is not fully supported. (In our case, it breaks when it tries to load the WebWorkers).
  • This requires us to use Rollup.js (optionally via Vite)
• A-Frame was not made for modules and has a non-standard setup which does not work well with modules. (Which is one of the reasons why they don't use the official ThreeJS under the hood.)
  • This optimally prefers Webpack.
  • It can work with modules (Rollup.js)

I did push an update on my vite branch that, as of now, has all of the A-Frame & ThreeJS examples working properly.
Before MindAR registers the custom systems/components with AFrame, it checks if the document DOM has already been parsed. If so, after registering, it reloads the DOM via document.body.innerHTML=document.body.innerHTML. It's not a bad solution, but my instincts don't like it.

As for now, we can go forward with Vite instead because we can get both working with Vite as opposed to only A-Frame with Webpack. (I think the best solution is the hybrid Webpack+Vite approach, personally. But I'll defer to you on that.)

[hiukim]

@jmschrack

haha, personally I'm always using threeJS version too since it's more flexible. But AFRAME is more user friendly for beginners, especially people without much web development experiences, so it's good to keep it.

Not really 50%, because the face tracking ThreeJS example also works for me! It looks like the only issue left is image tracking ThreeJS, and I would guess the error is either casued by worker or opencv.

[jmschrack]

@hiukim That's because FaceTarget doesn't use a WebWorker. The problem comes down to the ImageTracking's Controller Worker. If I use experimental.outputModule=true, I get Uncaught TypeError: Failed to execute 'importScripts' on 'WorkerGlobalScope': Module scripts don't support importScripts(). If instead I use target="web" I get Uncaught TypeError: __webpack_modules__[moduleId] is not a function at ./src/image-target/matching/matching.js (matcher.js:51:2) at ./src/image-target/matching/matcher.js (hough.js:174:2) at ./src/image-target/controller.worker.js (src_image-target_controller_worker_js.js:10:78) (I also can't get webpack to accept the external ThreeJS either in this mode)

I think this is a dead end for now. (Maybe a future webpack update will fix this?)

The two phase build process of Vite to generate the core&three libraries and then webpack to generate the aframe compatible libraries works fine, and requires no major code changes. Maybe in the future Vite or Webpack will update to solve this, but for now this is the simplest solution that doesn't require writing custom code to band-aid missing feature of webpack.

[hiukim]

@jmschrack I'll try to dig a bit deeper during the weekend