Babylon.js Support

[andreyrd]

I've been working on a Babylon.js renderer for ARnft. I found online that there used to be a branch where you were working on adding it. I have things half-working, right now I can't get the model to track with any stability.

[kalwalt]

Yes it is right, there was a also a PR https://github.com/webarkit/ARnft/pull/150 but as said in this https://github.com/webarkit/ARnft/pull/150#issuecomment-844048984 ARnft now is independent from rendering, it send only data matrix and nft infos(width, height and dpi).

[kalwalt]

I was working on this repository https://github.com/kalwalt/ARnft-babylonjs but its is still a Wip, i'm not a Babylonjs expert so i found some difficulties to set up the project. Maybe i will come back. Anyway i looked on @eddiecarbin repository https://github.com/kalwalt/InsectMobileDemo/ (my fork, look at testing-branch).

[andreyrd]

So my separate repo takes a similar approach to the threejs renderer. Except it 1) does not rely on Typescript and can be much more easily imported with webpack 2) simplifies the API a little bit to make it easier to use without losing functionality.

I think what I was missing was decomposing the matrix, like you are in your repo. I was trying to assign the matrix directly using freeze(World|Projection)Matrix.

If you don't mind, I'd like to show you the repo I have, so you can choose which to move forward with.

[kalwalt]

So my separate repo takes a similar approach to the threejs renderer. Except it 1) does not rely on Typescript and can be much more easily imported with webpack 2) simplifies the API a little bit to make it easier to use without losing functionality.

Typescript is not a must, there are advantages and disadvantages choosing Typescript, and from a webpack point of view yes it's easier to setup. If you have made some simplifications that make the code better i would like to see them.

I think what I was missing was decomposing the matrix, like you are in your repo. I was trying to assign the matrix directly using freeze(World|Projection)Matrix.

Are you referring to Babylonjs code?

If you don't mind, I'd like to show you the repo I have, so you can choose which to move forward with. Absolutely i will happy to see and test the code. After we can think also to create a repository inside the webarkit org to work on.

[andreyrd]

I think what I was missing was decomposing the matrix, like you are in your repo. I was trying to assign the matrix directly using freeze(World|Projection)Matrix.

Are you referring to Babylonjs code?

Yeah, in the code that updates the root mesh:

    // Shared by all add* methods. Sets up the root mesh to listen to AR controller events with new matrix
    _addRoot (root, name, visibility) {
        this.target.addEventListener(`getMatrixGL_RH-${this.uuid}-${name}`, e => {
            root.setEnabled(true);

            const matrix = this._objectToMatrix(e.detail.matrixGL_RH);
            const rotationMatrix = matrix.getRotationMatrix();
            const rotation = new Quaternion().fromRotationMatrix(rotationMatrix);
            root.rotation = rotation.toEulerAngles();
            const position = Vector3.TransformCoordinates(Vector3.Zero(), matrix);
            root.setAbsolutePosition(position);
            // root.freezeWorldMatrix(matrix);
        });

        this.target.addEventListener(`nftTrackingLost-${this.uuid}-${name}`, e => {
            root.setEnabled(!!visibility);
        });
    }

I thought it'd be enough to set the matrix directly, like in Three.js, but using freezeWorldMatrix in Babylon.js. But no, we need to decompose the matrix into rotation & position, like you guys did in your repos.

Typescript is not a must, there are advantages and disadvantages choosing Typescript, and from a webpack point of view yes it's easier to setup. If you have made some simplifications that make the code better i would like to see them.

Absolutely i will happy to see and test the code. After we can think also to create a repository inside the webarkit org to work on.

Here is the repo: https://github.com/i8studios/ARnft-Babylon.js

The README should have a simple enough usage example, but let me know if something doesn't work, feel free to message me on Slack as well for a faster reply. I've implemented addModel and addVideo so far.

[kalwalt]

Thank you Andrew, i will look at your code! :slightly_smiling_face: