Support WebRTC Remote MediaStream

[soundTricker]

Is your feature request related to a problem? Please describe. Currently, AR.js only supports a web camera of own browser. But I need to use remote MediaStream on AR.js.

Describe the solution you'd like Add sourceType: "stream" and sourceStream property to THREEx.ArToolkitSource. Then AR.js process that stream like sourceType: webcam like below code.

const arToolkitSource = new THREEx.ArToolkitSource({sourceType: 'stream', sourceStream: webRTCRemoteStream});

Note: Sorry I don't know a-frame.

Describe alternatives you've considered

Currently if we want to use a remote stream, we should write like below.

const arToolkitSource = new THREEx.ArToolkitSource({sourceType: 'video'});
arToolkitSource.init(() => {
      rootDiv.appendChild(arToolkitSource.domElement);
      onResize();
});
arToolkitSource.domElement.srcObject = webRTCRemoteStream;

Additional context I've already use remote video stream with AR.js on our hobby project. (Raspberry Pi Tank AR shooting game) https://www.youtube.com/watch?v=Dpf0kKpWWfQ https://github.com/kabuku/makerfaire-tokyo-2019/blob/master/web/src/app/game-main/components/scene.component.ts

There is solution for this issue, but i think this feature should include into AR.js

[nicolocarpignoli]

Hi! yes, it would be a nice addition. A PR is welcome for this!

[JaeDukSeo]

@soundTricker First of all thank you so much for this question. However, can I ask more questions about this method?

I am planning to have a flask server running along a node server, and I want to stream a video from flask to ar js in node server. May I ask the details on how to accomplish this?

More specifically how to use ar js without aframe and with arToolkitSource

[nicolocarpignoli]

@JaeDukSeo you can use AR.js with threejs instead of aframe. There is documentation on the threejs folder, on the Readme. Also check the great examples from stemkoski, you can found them on the principal AR.js readme.

Anyway, this goes 'off' this specific issue. Please open another one if you find difficulties on working with ar.js using threejs. There is also the http://gitter.im/AR-js/Lobby for quick help

[JaeDukSeo]

@nicolocarpignoli for sure thank you for the answer much appreciated

[nicolocarpignoli]

Hi there, there is a new repository of AR.js: https://github.com/AR-js-org/AR.js/ If this issue is still relevant, please re-open in the new one. thank you.