Video is black on Android Chrome, iOS Chrome and iOS Safari

[thylacinelol]

Hi,

I tried this arNFT_video_example on multiple devices and the video seems to be black (or just stuck on 1st frame?). Here's a screenshot of what the video looks like:

The video work only on android chrome. On android firefox, iOS chrome, iOS safari - the video is black. I also tried using the library with other videos but same happens with other videos.

[kalwalt]

Thank you for testing @thylacinelol ! Unfortunately i have no other devices for testing, so some bug may arise. I think that i need to add an extra attribute to https://github.com/webarkit/ARnft/blob/be2944db4bf61896cdbdd1435942d60ce489b8c0/examples/arNFT_video_example.html#L17-L26

[kalwalt]

Do you receive some error message in the console log?

[kalwalt]

Maybe adding webkit-playsinline as in:

   <video src="mov_bbb.mp4"
     preload="auto" id="vid" response-type="arraybuffer" loop
     crossorigin webkit-playsinline autoplay muted playsinline>
  </video>

https://github.com/AR-js-org/AR.js/blob/9f4d3c2ebef717030ef9077af2dcc626980d283d/aframe/examples/image-tracking/nft-video/index.html#L56-L59

[kalwalt]

Sorry i already added webkit-playinline

[thylacinelol]

Hey, you're right, there is an error message in the console: Uncaught (in promise) DOMException: The play method is not allowed by the user agent or the platform in the current context, possibly because the user denied permission..

[kalwalt]

Hey, you're right, there is an error message in the console: Uncaught (in promise) DOMException: The play method is not allowed by the user agent or the platform in the current context, possibly because the user denied permission..

Ok, we need to apply the right crossorigin attribute for that kind of browsers.

[kalwalt]

i think that the crossOrigni attribute is correct but in some browsers it's needed the explicit consent by the user as described in this stackoverflow article https://stackoverflow.com/questions/57504122/browser-denying-javascript-play

[thylacinelol]

There's a "best practices" guide for safari here, looks interesting:

var promise = document.querySelector('video').play();

if (promise !== undefined) {
    promise.catch(error => {
        // Auto-play was prevented
        // Show a UI element to let the user manually start playback
    }).then(() => {
        // Auto-play started
    });
}

[thylacinelol]

Similar approach on mozilla:

let startPlayPromise = videoElem.play();

if (startPlayPromise !== undefined) {
  startPlayPromise.then(() => {
    // Start whatever you need to do only after playback
    // has begun.
  }).catch(error => {
    if (error.name === "NotAllowedError") {
      showPlayButton(videoElem);
    } else {
      // Handle a load or playback error
    }
  });
}

[kalwalt]

Can you try one of these approach please? :slightly_smiling_face:

[thylacinelol]

Ok I tried this approach for android firefox and it still doesn't work. I tried logging to console everything and I get to the part "// Start whatever you need to do only after playback has begun." which means that it works regardless of this script.

However, I checked the logs more closely and found some other warnings which do not appear in chrome. Here's a screenshot of chrome console log:

And here's a screenshot of firefox console log:

Maybe these warnings are related to this issue?

WebGL warning: tex(Sub)Image[23]D: Resource has no data (yet?). Uploading zeros.
WebGL warning: drawElementsInstanced: Tex image TEXTURE_2D level 0 is incurring lazy initialization.

[kalwalt]

Ok I tried this approach for android firefox and it still doesn't work. I tried logging to console everything and I get to the part "// Start whatever you need to do only after playback has begun." which means that it works regardless of this script.

However, I checked the logs more closely and found some other warnings which do not appear in chrome. Here's a screenshot of chrome console log:

And here's a screenshot of firefox console log:

Maybe these warnings are related to this issue?

WebGL warning: tex(Sub)Image[23]D: Resource has no data (yet?). Uploading zeros.
WebGL warning: drawElementsInstanced: Tex image TEXTURE_2D level 0 is incurring lazy initialization.

Then maybe is an issue with Three.js, maybe the addVIdeo function need some tweaks.

[kalwalt]

I will try to add this in the addVideo function: https://github.com/webarkit/ARnft/blob/be2944db4bf61896cdbdd1435942d60ce489b8c0/src/ARnft.js#L143-L158

ARVideo.preload = 'auto'
ARVideo.autoload = true

[kalwalt]

Can you test this code in webarkit/ARnft#173 @thylacinelol ? i can't do remote debugging with my device. :sob:

[thylacinelol]

@kalwalt I tried the branch fix-video-issue, but the video is still black.

The logs in android firefox remote debugger tool still have these lines:

WebGL warning: tex(Sub)Image[23]D: Resource has no data (yet?). Uploading zeros. three.min.js:6:305215
WebGL warning: drawElementsInstanced: Tex image TEXTURE_2D level 0 is incurring lazy initialization.

There's also this warning in the beginning of the log, but no idea if it's related:

No canvas available ARToolkitNFT.js:1:747991

I also tried a bunch of other things (upgrade three.js version; make the code in addVideo as close to the threejs video example as possible), but it also doesn't work :octopus:

[kalwalt]

@thylacinelol this is a bad new, can tour create a gist of your testing example 🙏? I will work 👷‍♂️ a little this evening, if i have time 🙂.

[thylacinelol]

Here's a git patch with the changes I made. It's not a lot and are just changes to addVideo to try to match what's in the three.js video example.

I'm sure this issue is something simple that was missed, because the three.js video example works fine on all devices.

[kalwalt]

Here's a git patch with the changes I made. It's not a lot and are just changes to addVideo to try to match what's in the three.js video example.

I'm sure this issue is something simple that was missed, because the three.js video example works fine on all devices.

Yes i have also this idea. I hope to fix soon this. Thank you for the gist!

[kalwalt]

this answer in the stackoverflow article seems intetresting. I will try it. :slightly_smiling_face:

[kalwalt]

I created a live example https://kalwalt.github.io/kalwalt-interactivity-AR/ARnft/arNFT_video_example.html with latest from https://github.com/webarkit/ARnft/pull/173# commit https://github.com/webarkit/ARnft/pull/173/commits/c7401aa7c8d551226066963f0175c7a41c393102 tested on Firefox under Android, now the Threejs warning messages are gone, but i can't see the video (nor a black screen).

[kalwalt]

@ThorstenBux this your code https://github.com/ThorstenBux/ar-magazine/blob/d2fd0ce0f084ba88ab5857fde95528b73e54f9f7/src/threejs_wasm_worker.js#L32-L48 work on Android Firefox?

[ThorstenBux]

This is so old. I don't know if that still works

[kalwalt]

Because the new changes (new ARnft version 0.9.0) the addVideo is now part of ARnft-theejs i'm testing the example https://kalwalt.github.io/kalwalt-interactivity-AR/ARnft/arNFT_video_example.html again but it track but no video at all. It worked on localhost on Desktop. I need to check the threejs version in the example, could be that.

[kalwalt]

i will also move the issue to the new repository.