Access to RGB camera using Spin app

[Peiffert]

Hi, I am also trying to perform tracking based on holojs and Spin capabilities but I can't find a way to access to the Hololens camera. I notice someone mentioned it before in a closed issue #169:

@Almost-Done @xabuloes Yes I tried too and it works fine with the fix. Just a question, I tested successfully on SampleApps and ThreeJSApps but I can't make it work on the HoloJSWebViewer, it can't access the source "camera://local/default" through a Get. Any idea how to connect it to the webcam? (webcam and microphone capabilities added right ) Thanks!

Based on the code example of #169 issue, running on Spin, frame.onload is never called. I guess frame.src = "camera://local/default"; isn't right or enough. Can you please tell me if there is another way. Thank you for your help.

[Almost-Done]

Access to the PV camera is not implemented in HoloJs V2 and Spin. It was also omitted from the breaking changes list - sorry about that.

It should be straightforward to add back.

[Peiffert]

Alright, thank you for this quick answer. 👍

[Almost-Done]

Let's keep this issue open; I'll try to merge a fix in the next day or so.

[Almost-Done]

I merged the changes with https://github.com/microsoft/HoloJS/commit/3f2f2064f3f3e43e8a39849dd04ea5dede611a39

The fix is in the 1.5.0 NuGet package The fix is in Spin v3.1

[Almost-Done]

Here's a JS snippet:

    var imageElement = document.createElement('img');
    imageElement.onload = function(e) {
        let texture = new THREE.Texture( this );
        texture.needsUpdate = true;

        let material = new THREE.MeshBasicMaterial( {  map: texture } );
        let geometry = new THREE.BoxBufferGeometry(0.3, 0.3, 0.3);
        let cube = new THREE.Mesh(geometry, material);
        cube.position.z = -1.5;
        cube.position.y = 1.6;
        scene.add(cube);
    };

    imageElement.src = 'camera://local/default';

After onload fires, you can set the source again to 'camera://local/default' and onload will fire again with a new photo. However, this is inefficient; a better way would be to have camera as a source for a Video element and grab frames as needed.