search

aframevr / aframe

:a: Web framework for building virtual reality experiences.
https://aframe.io/
MIT License
16.69k stars 3.98k forks source link

iOS 12.2 video HLS Wrong Color / Occasional black screen #4134

Closed fyuvb closed 9 months ago

fyuvb commented 5 years ago

Recently iOS version 12.2 was pushed to everyone and it seemed to break video playback of aframe.

Problem 1: Color not mapped correctly iOS seemed fixed the color BRGA problem since iOS 11 for native HLS, link . However, aframe still apply the ios10hls shader for playback on iOS native hls. This is messing up the colors to a wrong one. Using standard "flat" shader should be good enough for color.

Problem 2: Native HLS occasional black screen upon video Playback. I have provided a sample code snippet. There is a certain chance such that canvas would display black screen. One finding is that when you change device orientation or enter / exit VR mode, the playback would become normal again. Could anyone advice how to fix this? Thanks!

Another thing that I found is that, some videos cannot play on iOS 12.2, like this one: https://github.com/mrdoob/three.js/raw/dev/examples/textures/sintel.mp4. While the example of aframe still works: https://aframe.io/examples/showcase/videosphere/. This is similar to Problem 2 that orientation change or enter / exit VR mode could put the playback back to normal. Could anyone advice? 

<html>
  <head>
    <script src="https://aframe.io/releases/0.9.1/aframe.min.js"></script>
    <style>
      #overlay {
        position: absolute;
        z-index: 1;
        top: 0;
        left: 0;
        width: 100%;
        height:100%;
        display: flex;
        align-items: center;
        justify-content: center;
        opacity: 1;
        background-color: #000000;
        color: #ffffff;
      }
      #overlay > div {
        text-align: center;
      }
      #overlay > div > button {
        height: 20px;
        width: 100px;
        background: transparent;
        color: #ffffff;
        outline: 1px solid #ffffff;
        border: 0px;
        cursor: pointer;
      }
      #overlay > div > p {
        color: #777777;
        font-size: 12px;
      }
    </style>
  </head>
  <body>
    <a-scene>
      <assets>
        <video id="video" loop crossorigin="anonymous" webkit-playsinline src="https://bitmovin.com/player-content/playhouse-vr/m3u8s/105560.m3u8">
        </video>
      </assets>
      <a-videosphere src="#video" rotation="0 -90 0"></a-videosphere>
    </a-scene>

    <div id="overlay">
      <div>
        <button id="startButton">Click to Play</button>
        <p>Automatic video playback with audio requires a user interaction.</p>
      </div>
    </div>

    <script>
      var overlay = document.getElementById('overlay');
      var startButton = document.getElementById( 'startButton' );
      var video = document.getElementById('video');
      startButton.addEventListener( 'click', function () {
        video.play();
        overlay.style = 'display:none';
      }, false );
    </script>

  </body>
</html>
esterlus commented 5 years ago

can confirm that the ios10hls shader is no longer necessary.

agosman commented 5 years ago

I'm also experiencing problem #2: a-videospehere showing only black. (iPhone SE, iOS 12.3.1) After rotating my device, the video always shows up immediately. Any advice on how to fix this woud be much appreciated.

vkolova commented 5 years ago

I suspect this is what 'fixes it' when rotating the device:

https://github.com/aframevr/aframe/blob/9785b6cd23eefec6113601ff28e91204c57c0c48/src/core/scene/a-scene.js#L141-L152

Calling .resize() on the scene didn't fix anything, though.

Any ideas if this is the right direction?

konstantin55000 commented 5 years ago

Hello, have very same problem on ios 12.4.1 any latest major browser including Safari Seams like red color have dropped away, so video on videosphere is only with blue yellow.. Does have any ideas how to fix that problem, or any demo's with videosphere where it work?

KulwinderJSingh commented 5 years ago

Can confirm that HLS video does not play on IOS 12.4.1. Video color is different or just show a black screen. Same video works fine on OS under A-frame scene.

andreyrd commented 4 years ago

Seems like the call to renderer.setSize() is what fixes Problem #2 on rotate: https://github.com/aframevr/aframe/blob/9785b6cd23eefec6113601ff28e91204c57c0c48/src/core/scene/a-scene.js#L557

For some reason, the canvas is too small until you start playing the video? Not sure what's going on there, but my quick hack for now is this:

      // Hack to fix bug with aframe & iOS video
      // https://github.com/aframevr/aframe/issues/4134
      function videoHackListener () {
        setTimeout(function () {
          $scene.renderer.setSize($scene.canvas.width, $scene.canvas.height);
        }, 500); // ugly but aframe's 100ms was not enough in my testing
        $video.removeEventListener('play', videoHackListener);
      }
      $video.addEventListener('play', videoHackListener);

where $scene is a reference to your <a-scene> and $video is a reference to your <video.

andreyrd commented 4 years ago

Turns out you should NOT do that, because while it works, it keeps making the renderer bigger and bigger until it eats all performance.

antiantivirus commented 4 years ago

I'm also experiencing the same issue with red channel not visible. Seems to be linked to ios10hls shader

antiantivirus commented 4 years ago

Is a-frame still applying the ios10hls shader on ios devices? I've had to revert to B&W video on mobile for the time being. Is there a fix that might be implementable for the red channel issue?

fabien-lg commented 4 years ago

I fixed the wrong color issue by creating a small component for videosphere that listens to materialvideoloadeddata and puts back the flat shader. flipY needs to be re-applied on the texture too.

antiantivirus commented 4 years ago

Ah nice, would you be able to share the code for this? Maybe we should bring this to attention of a-frame devs as something that needs a fix?

fabien-lg commented 4 years ago

@antiantivirus I'm using typescript:

import * as THREE from 'three';

interface VideoSphereShaderSchema {
}

export class VideoSphereShader extends ComponentWrapper<VideoSphereShaderSchema> {
    constructor() {
        super('videosphere-shader', {});
    }

    public init() {
        this.el.addEventListener('materialvideoloadeddata', (e: any) => {
            this.el.setAttribute('material', 'shader', 'flat');
            e.detail.texture.flipY = true;
        });
    }
}

new VideoSphereShader().register();

And on the VideoSphere:

        <a-videosphere
            src="#video"
            videosphere-shader
        ></a-videosphere>
antiantivirus commented 4 years ago

cheers fabien. slightly adapted the code to make it regular old javascript. make sure you register the component in the header so that it's available to aframe when the a-scene loads.

      AFRAME.registerComponent('ios-video-fix', {
        init: function(){
          let el = this.el;
          el.addEventListener('materialvideoloadeddata', e => {
            el.setAttribute('material', 'shader', 'flat');
            e.detail.texture.flipY = true;
          })
        }
      });

and then on a component add the attribute ios-video-fix

b0ot commented 1 year ago

As of today, this still appears an active problem. I had the issue where the colors were off for HLS video. I applied the ios-video-fix component to the videosphere and it seems like it is working now. Thank you! Hopefully this fix gets incorporated into the AFrame codebase.