Black screen when extending A-Frame hit-test-example with AR.js marker detection

MisterNox commented 2 years ago

Do you want to request a feature or report a bug? Report a bug.

What is the current behavior? I would like to extend a project I found on glitch, which includes an example of a-frame and hit-testing

Link to the original Example: https://glitch.com/~ar-starter-kit

The example itself works great but when I add a script tag to import aframe-ar.js and press the AR-Button I get a black screen and nothing happens. In the end I would like to add marker detection to the existing project.

I tried different version combinations but I dont seem to find a solution (master and dev versions).

If the current behavior is a bug, please provide the steps to reproduce.

<html>

<head>
  <script>
    // WebXR requires https: to work so ensure redirected if needed.
    if (location.hostname !== 'localhost' && window.location.protocol === 'http:') window.location.protocol = 'https:';
  </script>
  <!-- the AFrame library -->
    <script src="https://aframe.io/releases/1.3.0/aframe.min.js"></script>

  <!-- ##### HERE I HAVE ADDED THE SCRIPT TAG  ##### -->
        <script src="https://raw.githack.com/AR-js-org/AR.js/master/aframe/build/aframe-ar.js"></script>

  <!-- include a default environemnt which makes VR  look nice, https://github.com/feiss/aframe-environment-component -->
    <script src="https://unpkg.com/aframe-environment-component@1.3.1/dist/aframe-environment-component.min.js"></script>
  <!-- this shadow helper ensures that the model has a shadow beneath it -->
  <script src="ar-shadow-helper.js"></script>
  <script src="ar-cursor.js"></script>
    <script src="main.js"></script>

Thank you in advance for your help.

Please mention other relevant information such as the browser version, Operating System and Device Name I have tried it on my Samsung S10+ in the latest Chrome App

What is the expected behavior? Expected would be, that the camera starts and one can place the objects somewhere.

If this is a feature request, what is motivation or use case for changing the behavior?

MisterNox commented 2 years ago

Okay first of all I have forgotten the arjs tag in the a-scene. But besides that ar.js seems to inject the video element right at the beginning no matter what I do. With this code snippet I was able to stop the stream and remove it so when I click on the AR button it seems to work.

    window.addEventListener("arjs-video-loaded", (e) => {
      var videoEl = document.getElementById('arjs-video');
      // now get the stream 
      stream = videoEl.srcObject;
      // now get all tracks
      tracks = stream.getTracks();
      // now close each track by having forEach loop
      tracks.forEach(function(track) {
        // stopping every track
        track.stop();
      });
      // assign null to srcObject of video
      videoEl.srcObject = null;
      videoEl.remove();
    });

The problem now is, that marker detection does no work.

Can you help me with that? Thanks in advance.

MisterNox commented 2 years ago

Nobody who has an idea about this?

kalwalt commented 2 years ago

Okay first of all I have forgotten the arjs tag in the a-scene. But besides that ar.js seems to inject the video element right at the beginning no matter what I do. With this code snippet I was able to stop the stream and remove it so when I click on the AR button it seems to work.
    window.addEventListener("arjs-video-loaded", (e) => {
      var videoEl = document.getElementById('arjs-video');
      // now get the stream 
      stream = videoEl.srcObject;
      // now get all tracks
      tracks = stream.getTracks();
      // now close each track by having forEach loop
      tracks.forEach(function(track) {
        // stopping every track
        track.stop();
      });
      // assign null to srcObject of video
      videoEl.srcObject = null;
      videoEl.remove();
    });
The problem now is, that marker detection does no work.

Can you help me with that? Thanks in advance.

You can't do this, at least i don't reccommend doing this; the arController and other classes are already inited and you can re-init.

The reason that the code doesn't works as you expected could be the aframe version you are using, there are a couple of bugs with newer version than 1.0.4 that are not solved.
Copy and pasting the code is not a good practice. Probably there are some conflictcs with the aframe components from the example and AR.js ( see for example a-scene). You need to identify the source of the problem.

I suggest to you instead to go by little steps adding some code and see in this manner where it fails.

MisterNox commented 2 years ago

First of all thank you for your reply. And yes you are right, just copy and paste is not a good way to handle this. That's why I was trying to dig into this during the last days. But let's start at the beginning:

When you use aframe in combination with ar.js and only build up a simple scene this works great. Ar.js injects a video tag with the id "arjs-video" which acts more or less as the container to show the mobile camera stream in the background.

If you go instead with the approach shown in the glitch example aframe itself seems to handle the appearance of the mobile camera stream differently.

The aframe approach of the glitch project includes the "hit-testing"-feature I need and the aframe+arjs approach the "marker detection" (I would assume the frames are running through a function where some kind of feature detection compared to the marker happens, but this is just wild guessing). So in my opinion the handling of the stream has to be combined somehow. Is it maybe possible to use the stream of the aframe approach as an input stream to the video tag? If yes, where in aframe is the stream attached? I was clicking myself through the source code of the camera rendere etc. but could not find it.

I am really willing to dig deeper into this but it would have more chance of success if an experienced person like you could hint me into a direction. Of course, if you have some time to look at this yourself this would be also great. In the end I think I am bound to the latest version of aframe because of the hit-test feature.

Thank you in advance for any hints, ideas or approaches.

AR-js-org / AR.js

Black screen when extending A-Frame hit-test-example with AR.js marker detection #428