Three.js + Unity WebGL integration

[MrRamiBenDhia]

I hope you're all doing well. I'm currently working on a project where I'm aiming to integrate Unity with Salesforce. My approach involves building the Unity project into WebGL and then uploading it to a Salesforce Lightning Web Component (LWC) project.

However, I've hit a roadblock with this integration, and I'm encountering a couple of issues that I'm struggling to resolve. Firstly, even the basic step of uploading the WebGL files to the LWC project is causing errors, and it seems that the files are not being accepted by the platform.

Additionally, I've also attempted to use the Three.js library to showcase simple 3D models within the Salesforce environment. Unfortunately, I'm facing a challenge here as well but nothing seems to render or display as expected.

I'm curious if anyone in the community has successfully achieved a similar integration before, especially when incorporating Three.js for 3D rendering. Specifically, I'd like to know if there are any best practices, tips, or guidelines for integrating Unity (WebGL) projects and utilizing the Three.js library with Salesforce LWC. Are there any potential pitfalls to watch out for?

I would greatly appreciate any insights, experiences, or advice that anyone can provide. If you've encountered similar challenges or have successfully integrated Unity with Salesforce while incorporating Three.js, your input could be extremely valuable to me.

Thank you in advance for taking the time to read my extended question and share your expertise.

Best regards,

[msrivastav13]

I am a developer advocate here at Salesforce and curious to see what errors you are getting!

Did you upload the file to the static resource and try importing in lwc?

[MrRamiBenDhia]

Hi @msrivastav13 ,

Thank you so much for taking the time to assist me! I appreciate your guidance.

Regarding the Three.js library, I've successfully uploaded it as a static resource. Currently, I'm in the initial stages of creating a proof of concept by rendering a simple cube. Once I have that working, I'll be moving on to uploading more complex .glb or .fbx files.

I've also posted on Stack Overflow where I included some code snippets and additional context. If you're interested, you can find my post here: https://stackoverflow.com/questions/76919630/using-three-js-with-lightning-web-component-lwc

Lastly, I'm intrigued by the possibility of integrating Unity WebGL builds with Lightning Web Components and Salesforce. If you have any insights or resources on how to achieve this, I'd greatly appreciate it.

Thanks again for your help and support!

[msrivastav13]

Are you working with lightning locker or with LWS enabled in your org?

Also if your lib involves manual DOM manipulation I recommend try using lwc:dom="manual"

[MrRamiBenDhia]

I've created a Lightning Web Component (LWC) with functional HTML elements. However, adding a Three.js element doesn't render properly, even with lwc:dom="manual". I tried this:

<div lwc:dom="manual">

</div>

When I try to add any element inside the div it gives this error Invalid contents for element with "lwc:dom". Element must be empty

[nolanlawson]

@MrRamiBendhia Correct, lwc:dom="manual" does not allow any elements to be defined inside of it in the HTML. The element must be empty, and then you render elements into it using JavaScript.

Docs: https://lwc.dev/guide/reference#lwc%3Adom%3D%22manual%22

[MrRamiBenDhia]

@nolanlawson, Thanks for taking the time. Really appreciate your feedback. I've made some updates based on your suggestions,

<template>
  <h1>Three JS test</h1>
  <div class="container" lwc:dom="manual"></div>
</template>

import { LightningElement } from 'lwc';
import { loadScript } from 'lightning/platformResourceLoader';
import THREE_JS from '@salesforce/resourceUrl/ThreeJS'; 

export default class ThreeJSComp extends LightningElement {
    renderedCallback() {
        // Load Three.js library dynamically
        loadScript(this, THREE_JS)
            .then(() => {
                this.initializeThreeJS();
            })
            .catch(error => {
                console.log('Error loading Three.js:',error);
            });
    }

    initializeThreeJS() {
        const container = this.template.querySelector(".container");

        // Create a scene, camera, and renderer
        const scene = new THREE_JS.Scene();
        const camera = new THREE_JS.PerspectiveCamera(75, container.clientWidth / container.clientHeight, 0.1, 1000);
        const renderer = new THREE_JS.WebGLRenderer();
        renderer.setSize(container.clientWidth, container.clientHeight);
        container.appendChild(renderer.domElement);

        // Create a cube geometry for demonstration
        const geometry = new THREE_JS.BoxGeometry();
        const material = new THREE_JS.MeshBasicMaterial({ color: 0x00ff00 });
        const cube = new THREE_JS.Mesh(geometry, material);
        scene.add(cube);

        // Set camera position
        camera.position.z = 5;
    }
}

but I'm still having trouble getting the desired output.

Would you be able to take a look and see if there's anything else I can improve?

[kevinv11n]

@MrRamiBenDhia renderedCallback() gets invoked many times so your code needs to guard for that. What's the observed vs expected behavior?

I recommend looking at https://github.com/trailheadapps/lwc-recipes/tree/main/force-app/main/default/lwc/libsD3/. It's a functioning example of using d3 as a static resource and targeting a DOM element decorated with lwc:dom="manual". I expect you can use a similar approach, which is already very similar to the code you shared.

[pmdartus]

@MrRamiBenDhia I took your example and made it work on the LWC playground: live example

Here is the come I the component code I came up with:

<template>
    <canvas lwc:ref="canvas" style="width: 500px; height: 500px"></canvas>
</template>

import { LightningElement } from 'lwc';

const THREE_JS_SRC =
  'https://cdnjs.cloudflare.com/ajax/libs/three.js/0.155.0/three.js';

/** Mock implementation of the platform resource loader. */
function loadScript(src) {
  const scriptElm = document.createElement('script');

  return new Promise((resolve, reject) => {
    scriptElm.addEventListener('load', resolve);
    scriptElm.addEventListener('error', reject);

    scriptElm.src = src;

    document.head.appendChild(scriptElm);
  });
}

export default class App extends LightningElement {
  _isRendered = false;

  async renderedCallback() {
    if (this._isRendered) return;
    this._isRendered = true;

    await loadScript(THREE_JS_SRC);
    this._initScene();
  }

  _initScene() {
    const canvas = this.refs.canvas;
    const { width, height } = canvas.getBoundingClientRect();

    const scene = new THREE.Scene();
    const camera = new THREE.PerspectiveCamera(75, width / height, 0.1, 1000);

    const renderer = new THREE.WebGLRenderer({ canvas });
    renderer.setSize(width, height);

    const geometry = new THREE.BoxGeometry(1, 1, 1);
    const material = new THREE.MeshBasicMaterial({ color: 0x00ff00 });
    const cube = new THREE.Mesh(geometry, material);
    scene.add(cube);

    camera.position.z = 5;

    renderer.render(scene, camera);
  }
}

I have made the following adjustment to your code:

• Added renderer.render(scene, camera), without this Threejs doesn't render the scene at all.
• Protected _initScene from getting invoked multiple times by adding a _isRendered flag.
• Replaced the <div> from the template with a <canvas> element, and passed it directly to the THREE.WebGLRenderer constructor. This avoids directly manipulating the DOM in the renderedCallback
• Look up the canvas element using a ref instead of a query selector. Looking up elements using refs is more performant than via querySelector. It doesn't make much of a difference in this case, but I find it a lot cleaner to read.
• Added a mock implementation of loadScript to be able in the playground. You should get rid of it when pushing this code to the Salesforce platform.

[MrRamiBenDhia]

I really appreciate your contribution to this issue. I've incorporated your code into my project and made the necessary changes.

I've made the changes @kevinv11n suggested to avoid being invoked multiple times

I even tried the D3 lib to determine whether the problem was related to security restrictions or platform-specific limitations but I got it to work and it renders perfectly

also Thank you @pmdartus for your time, I've replaced initializeThreeJS() with your _initScene()

my function did call the renderer.render(scene, camera) in the animation loop that I left out earlier

here is my final .js file

import { LightningElement } from "lwc";
import { loadScript } from "lightning/platformResourceLoader";
import THREE_JS from "@salesforce/resourceUrl/ThreeJS";
import { ShowToastEvent } from "lightning/platformShowToastEvent";

export default class ThreeJSComp extends LightningElement {
  threeJSInitialized = false;

  async renderedCallback() {
    if (this.threeJSInitialized) return;

    this.threeJSInitialized = true;

    try {
      await Promise.all([loadScript(this, THREE_JS)]);

      this._initScene();
    //   this.initializeThreeJS();
    } catch (error) {
      this.dispatchEvent(
        new ShowToastEvent({
          title: "Error loading threeJS",
          message: error.message,
          variant: "error"
        })
      );
    }
  }

  initializeThreeJS() {
    const container = this.template.querySelector(".container");

    // Create a scene, camera, and renderer
    const scene = new THREE_JS.Scene();
    const camera = new THREE_JS.PerspectiveCamera(
      75,
      container.clientWidth / container.clientHeight,
      0.1,
      1000
    );
    const renderer = new THREE_JS.WebGLRenderer();
    renderer.setSize(container.clientWidth, container.clientHeight);
    container.appendChild(renderer.domElement);

    // Create a cube geometry for demonstration
    const geometry = new THREE_JS.BoxGeometry();
    const material = new THREE_JS.MeshBasicMaterial({ color: 0x00ff00 });
    const cube = new THREE_JS.Mesh(geometry, material);
    scene.add(cube);

    // Set camera position
    camera.position.z = 5;

    // Animation loop
    const animate = () => {
      // requestAnimationFrame(animate);
      cube.rotation.x += 0.01;
      cube.rotation.y += 0.01;
      renderer.render(scene, camera);
    };
    animate();
  }

  _initScene() {
    const canvas = this.refs.canvas;
    const { width, height } = canvas.getBoundingClientRect();

    const scene = new THREE_JS.Scene();
    const camera = new THREE_JS.PerspectiveCamera(75, width / height, 0.1, 1000);

    const renderer = new THREE_JS.WebGLRenderer({ canvas });
    renderer.setSize(width, height);

    const geometry = new THREE_JS.BoxGeometry(1, 1, 1);
    const material = new THREE_JS.MeshBasicMaterial({ color: 0x00ff00 });
    const cube = new THREE_JS.Mesh(geometry, material);
    scene.add(cube);

    camera.position.z = 5;

    renderer.render(scene, camera);
  }
}

Despite the changes, I'm still not seeing any results. (blank white card) I'll continue to troubleshoot and try to resolve the issue. Once again, thank you both for your assistance. Your input has been invaluable, and I appreciate your time and effort.

[pmdartus]

Is there any error in the console? If no, I would suggest, adding a breakpoint and stepping through the code.