Decouple ThreeJS from MindAR

[Blitzy]

The Three.js variants of the MindAR library are including Three.js in the bundle. Three.js should instead be left as an external dependency of MindAR. As of writing, MindAR is inextricably tied to a bundle version of Three.js r132.

[hiukim]

@Blitzy That's a very good point!

but I couldn't figure out a way to decouple it without losing the high level API. The raw tracking part is independent of THREE.js, but people will need to write a lot more code to get a working application.

Welcome any suggestions.

[Blitzy]

In the simplest form you could just make Three.js an external dependency in the weback configs.

https://webpack.js.org/configuration/externals/

This would allow users of MindAR to provide their own copy of Three.js and allow them to use updated versions of Three.js as they release quite frequently. It also allows you to keep doing the Three.js specific things that are being done.

However, I would not recommend relying on Three.js at all and simply have some sort of output pipe for the anchor transforms/matrices from MindAR that can be applied directly to whatever objects the devs want to apply it too. This is cleaner and keeps MindAR closer to a dependency-free utility API instead of its own rendering framework. It also lets author's setup Three.js the way they want/need to for their project.

Obviously just making Three an external dependency in webpack is the shortest path 😋

[Makio64]

I run into the same problem, my solution is basic : copy the class in your repertory : MindARThree.

in your main.js

import MindARThree from './MindARThree'

replace the top part of MindARThree by :

import * as MindAR from 'mind-ar/dist/mindar-image.prod'
import { PerspectiveCamera, Quaternion, Group, sRGBEncoding, WebGLRenderer, Scene, Matrix4, Vector3 } from 'three';
const {CSS3DRenderer} = require('three/examples/jsm/renderers/CSS3DRenderer.js');

 // for some reason MindAR return an empty object but this window is not empty..
const Controller = window.MINDAR.IMAGE.Controller
const UI = window.MINDAR.IMAGE.UI 

const cssScaleDownMatrix = new Matrix4();
cssScaleDownMatrix.compose(new Vector3(), new Quaternion(), new Vector3(0.001, 0.001, 0.001));

class MindARThree {
  constructor({container, imageTargetSrc, maxTrack, captureRegion=false, uiLoading="yes", uiScanning="yes", uiError="yes"}) {

Done.

Notice : we load mindar-image.prod and not the mindar-image-three.prod ;)

[hiukim]

@Blitzy

In the latest release, I've decoupled the threejs code as much as possible. Now only two files require threejs https://github.com/hiukim/mind-ar-js/blob/master/src/image-target/three.js https://github.com/hiukim/mind-ar-js/blob/master/src/face-target/three.js

I also tried to make the dependency external as you suggested, but without success. Do you want to give it a shot?

[Makio64]

I also gave a try on externals but it was not working if three wasnt imported at the top script.

[Makio64]

I think the best architecture woud be like threejs "examples" where peoples can go in and take what they want :)

[hiukim]

As a related issue, there is a breaking change in three.js v137 that apply importmap (I personally have never heard of importmap before). In short, all the threejs examples got updated, the require XXXX from '../../../build/three.module.js' is changed to require("three"), and there is an importmap thing in every examples. Checkout this commit for details: https://github.com/mrdoob/three.js/pull/23255/files

Now I couldn't even get v137 to work :(

So we probably need to proper solution to handle three.js in the long term. See if anyone has some good advice.

[Makio64]

My solution here work with the latest version of threejs : https://github.com/hiukim/mind-ar-js/issues/104#issuecomment-1019562231 The only things, the developer have to embed Threejs by themself but with a good "how to" it will not be a problem for the the peoples who want this level of control.

Also I think its a bit similar to the solution offer by Jeeliz library and his JeelizThreeHelper.

[hiukim]

@Makio64 Yes, I think your solution could be a good workaround for now for people who wants high level control over threejs, until a better solution come up.

[FiltrosInsta]

Hello @hiukim, thanks for your great contribution to the WebAR world!!

Would you know how to rotate the camera to see it upside down just like Instagram?

[uyendo-tmrw-vn]

I have a problem same you

[eaallen]

Hello @hiukim, thanks for your great contribution to the WebAR world!!

Would you know how to rotate the camera to see it upside down just like Instagram?

This question belongs in it own thread. But here is a CSS solution

video {
  -webkit-transform: scaleX(-1);
  transform: scaleX(-1);
}

Apparently, this solution will not work on firefox. I got this from a classic stackoverflow search: https://stackoverflow.com/a/50176947/15034002

[Eisenh]

I would love a way to get the canvas coordinates and pose of the matched image before invoking Three.js. in fact, I have use cases that do not need three.js or any AR. Simply being able to create and track an arbitrary image is important for more than AR.

[forerunrun]

I run into the same problem, my solution is basic : copy the class in your repertory : MindARThree.

in your main.js

import MindARThree from './MindARThree'

replace the top part of MindARThree by :

import * as MindAR from 'mind-ar/dist/mindar-image.prod'
import { PerspectiveCamera, Quaternion, Group, sRGBEncoding, WebGLRenderer, Scene, Matrix4, Vector3 } from 'three';
const {CSS3DRenderer} = require('three/examples/jsm/renderers/CSS3DRenderer.js');

 // for some reason MindAR return an empty object but this window is not empty..
const Controller = window.MINDAR.IMAGE.Controller
const UI = window.MINDAR.IMAGE.UI 

const cssScaleDownMatrix = new Matrix4();
cssScaleDownMatrix.compose(new Vector3(), new Quaternion(), new Vector3(0.001, 0.001, 0.001));

class MindARThree {
  constructor({container, imageTargetSrc, maxTrack, captureRegion=false, uiLoading="yes", uiScanning="yes", uiError="yes"}) {

Done.

Notice : we load mindar-image.prod and not the mindar-image-three.prod ;)

Hi @Makio64 do you have a more detailed insight into how you're doing this? The instructions here are a little confusing for me, do you happen to have a live example showing how you've implemented the code in full?