Safari/Firefox DOM Exception 18 for Streaming Video Texture

[jonobr1]

https://dl.dropboxusercontent.com/u/1595444/shaka-player/stream-texture.html (on Chrome clicking play on the video element shows an upside down render of the video on the canvas element. In Safari/Firefox texImage2D is real upset)

[mrdoob]

Is the upside down issue part of the problem?

[jonobr1]

Upside down is not part of the problem. That was me being lazy >_<

[mrdoob]

I can reproduce here. Have you tried having an in-between canvas?

[jonobr1]

:+1:, in-between like a canvas 2d?

[mrdoob]

Yep, use a canvas 2d as texture and do drawImage of the video to it.

[jonobr1]

I'll do that now. https://dl.dropboxusercontent.com/u/1595444/shaka-player/shaka-canvas.html

[jonobr1]

Looks like in Safari there is no DOM Exception 18 thrown, but the image is not transferred. Same with Firefox. Does that mean it's a browser implementation thing?

[tahasabih]

Facing the same issue. Safari throws Dom Exception 18, Firefox doesn't load the image but also shows no errors. @jonobr1 are you getting any errors in firefox?

[jonobr1]

No errors in Firefox, but also no image. I'm going to file bugs with WebKit and Mozilla respectively and see if any additional information comes up.

I don't know about you guys but I haven't seen any demos of streaming video uploaded to a WebGL texture. Maybe there is one with YouTube as the source? On Thu, Feb 11, 2016 at 11:11 PM Taha Sabih notifications@github.com wrote:

Facing the same issue. Safari throws Dom Exception 18, Firefox doesn't load the image but also shows no errors. @jonobr1 https://github.com/jonobr1 are you getting any errors in firefox?

— Reply to this email directly or view it on GitHub https://github.com/mrdoob/three.js/issues/8110#issuecomment-183208337.

[tahasabih]

Streaming video in itself is not an issue. If you create a video element in js and use video.src to set the source of the video element, the video streams fine. Adaptive streaming with shaka is a problem. I feel like the problem is with CORS implementations in webkit and gecko hence the DOM Exception 18 in Safari. The example below works perfectly with same domain video sources on all browsers.

            var video = document.createElement( 'video' );
            video.width = 640;
            video.height = 360;
            video.autoplay = true;
            video.loop = true;
            video.src = "<your mp4 source>";

           // adding the line below makes cross origin videos work, but just for chrome
           // video.crossOrigin = 'anonymous';

            var texture = new THREE.VideoTexture( video );
            texture.minFilter = THREE.LinearFilter;

            var material   = new THREE.MeshBasicMaterial( { map : texture } );

            mesh = new THREE.Mesh( geometry, material );

[jonobr1]

Just to clarify I don't think video.src = "<your mp4 source>"; is streaming. I believe it's progressive download. But, yes I agree with you.

[danrossi]

Guys are people still only working this out. It is only going to get worse now people are trying to do VR video.

Thankfully there is a ticket open or else if I tried to communicate this I'd get shut down.

For years Safari has had issues with CORS. It's a serious flaw and they have treated it with full contempt. 2D canvas drawing is also affecting.

I have created a ticket with Apple directly because an issue with webkit has been sitting there since July 2014 unassigned. WebGL video is basically useless on IOS and Safari until then. But then Iphone playing inline video is also required.

I believe filing duplicates with Apple might help them hurry it up.

Please see these. I have almost finished getting video textures and 360 video working fully on Android, Chrome Dev has only just fixed rendering issues I had to work tirelessly with chromium on that. There is now video playback performance issues with Chrome.

I'm now onto this mess with Safari !

This is a radar of the actual apple ticket with the issue word for word

https://openradar.appspot.com/24641824

https://forums.developer.apple.com/message/113161#113161

The actual webkit issue. They are a piece of work.

https://bugs.webkit.org/show_bug.cgi?id=135379

Basically this has nothing to do with three.js but @mrdoob please keep this open it's important.

You need to do reverse proxy hacks that will never scale just like with canvas 2D drawing for snapshots. It is documented here for now

https://flowplayer.electroteque.org/snapshot/fp6

location /video/ {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-NginX-Proxy true;
        proxy_pass //videos.electroteque.org/;
        proxy_redirect off;
    }

That will forward to cloudfront. It will work on any video including HLS.

[danrossi]

It seems this is more intentional sabotage than anything to block this on IOS. Firefox on IOS doesn't have CORS support either for no reason at all. It's showing up my CORS proxy hack fallback feature. I thought at least Firefox would be fully functional.

[tahasabih]

this still doesn't explain why firefox desktop won't draw images. It sends the origin headers, and if used without Shaka works fine. Only when the video source is being referred to a blob (as is the case with adaptive streaming) does it fail to draw the image. It also reports no errors which is frustrating.

[danrossi]

Dash works fine with Dash.js and video textures. But I will confirm across Windows/OSX once I'm done refactoring their changes.

[danrossi]

Firefox OSX and Windows is fully functional. With Dash or normal html5 streaming.

Apart from the following:

I get these logs appear on both OSX and Windows.

Error: WebGL: Disallowing antialiased backbuffers due to blacklisting.

I am running Firefox on Windows 10 in Vmware. Because of this I think Firefox is blocking Webgl. It was falling back to the CanvasRenderer. I had to specifically enable webgl with this config.

webgl.force-enabled=true

You need to check for WebGL support and fallback to the CanvasRenderer but I don't believe that is your issue.

Both WebGL and Canvas need CORS support to work so Firefox is ruled out. It's all those webkit bludgers and Apple now.

[danrossi]

FYI this is what my generated video tag looks like.

You need the crossorigin property that is the whole fuss with Safari. Whoever said it doesn't work on Firefox that is not correct.

Safari have failed to implement it for years and that is why WebGL does not work with it for cross domain videos which requires the reverse proxy hack.

IE support for CORS is also dodgy. IE11 has failed to implement it also. Edge browser is fine with CORS and Webgl.

<video crossorigin="anonymous" x-webkit-airplay="allow" preload="auto" webkit-playsinline="true" src="blob:http%3A//localhost%3A8000/8a0dd373-8cb1-4a1d-9ad2-60d0b8f001bf" width="1024" height="512"></video>

[jonobr1]

Hey @danrossi, love the enthusiasm! Do you have an online example of it working in dash.js?

Also, I filed bugs with both Safari and Firefox. You can see the progress on both of them here:

https://bugs.webkit.org/show_bug.cgi?id=154189

https://bugzilla.mozilla.org/show_bug.cgi?id=1248054

[danrossi]

Try dash.js with video tag like that. I will confirm with the proxy hack if safari is working with dash.

I suggest to get Apple to get involved with the CORS issues by firing off duplicate tickets on the CORS issue with them like I have. I mean the webkit ticket may be sitting there since July 2014 but something as crippling as this they should have taken action over.

[danrossi]

Confirming the proxy hack does nothing when using MediaSource on Safari it is fine with mp4 files. So even if they finally implement CORS this will still be broken.

I've left a comment there in the webkit issue, but considering the major CORS ticket has been left unassigned since 2014 , Safari and IOS is pretty dead for VR.

Firefox is perfectly fine for me on Windows 10 and OSX. Firefox does not have CORS on IOS so useless there. I might make a ticket for them to do so, so at least something is working on IOS.

[danrossi]

I've left another comment on your webkit ticket. They are related to the main CORS issue and probably should be merged and dealt with at once.

What really drives me crazy is your ticket is assigned but the ticket for the actual CORS problem has been left high and dry for 2 years rendering VR on Safari useless.

Again this one has nothing to do with three.js but should be left open for those only realising Safari and IOS is stuffed.

[danrossi]

OK after doing some research for inline video hacks for Iphone which I've already added to the Apple ticket it does seem drawing to the 2D canvas is ok without CORS. Its when trying to get a data uri from it it has the problem so my mistake. So it seems when using WebGL it is also a problem because Safari lack of CORS support and have to resort to the canvas renderer instead.

It won't or does not perform well on IOS as it drops frames and I believe it needs an audio tag running to play the audio through so that won't work out well for mpeg dash.

see http://stanko.github.io/html-canvas-video-player/

[danrossi]

Just to keep things simple I've updated this raw webgl example to use mp4 for safari. I've also updated that webkit ticket. It's because of the lack of CORS support.

https://jsfiddle.net/7t77rz6L/11/

"SecurityError: DOM Exception 18: An attempt was made to break through the security policy of the user agent."

[gsabran]

Thanks for taking care of reporting the issue. I've seing this outside three.js as well, and it's very annoying..!

[danrossi]

Apple ! It needs a mass wave of duplicate tickets if possible. I mean heck Youtube are sure of themselves with WebGL support but Safari will not work and does not work in their current 360 player so I'm not sure why they aren't doing anything about it. I have confirmed also even with the proxy hack Iphone will not play inline so VR and VR glasses is useless on an Iphone completely. It requires a very dodgy hack of playing audio separate and updating the time of the video to get a new frame so not actually playing the video. That won't work with dash.

[MasterJames]

And it's not a legal issue with format rights being the real non-technical issue? So you've tried the other formats I'm assuming? webm ogg apparently there's also the idea of Media Source Extensions MSE there are also different high profile settings to MP4 H.264 that don't run on lower end hardware (issues). I recall problems with video format compatibility being non-technical and more licensing and legal or just control freak issues. Anyway so shocking so +1. I guess we can always boycott, in the end no one will use the stuff that doesn't work. I use Android and Chrome exclusively for many other reasons including this one.

[danrossi]

It's a webkit issue with lacking of CORS functionality. They've let it go to waste for years thinking they could get away with it. Only now people trying to use Webgl have a problem and just realising. An accident waiting to happen. The same with forcing Iphone not to play inline. Same domain videos is not a real world usage.

Android Chrome has webgl issues also rendering to canvas. I've just spent the whole month dealing with that and finally confirmed in that ticket the latest Dev is now working again. At least Android will work with VR glasses and WebVR, Iphone will not.

I will try once more with an actual local loaded dash file and see what happens.

[danrossi]

There is a status update on the webkit ticket. I believe they had to fix something else to work on that ticket but still nobody assigned to the actual cors bug ticket. Absolutely everything is counting on them fixing it and hopefully get a release into both IOS and safari. This should be highly embarrassing for apple. They let this go for many years for no reason.

[mrdoob]

@danrossi thanks for keeping us up to date! 😊

[terkelg]

@danrossi amazing work! Thanks for all the updates

[danrossi]

??? Webkit dev working now ?

I have not seen a status update saying otherwise but noise of commits. I will confirm.

this is the last update. Who knows if this actually fixes the lack of CORS support also.

https://bugs.webkit.org/show_bug.cgi?id=125157#c29

Who knows if it will ever make it into IOS. Iphone inline video playback issues are of concern also. Should I make another bug ticket to Apple about that ?

Oh sorry I thought you said it was working hah. Very close it seems. I have a radar ticket for Apple and they have never responded. It requires duplicates made if they are ever to take action or care that VR works on their platform.

[danrossi]

So I believe the pressure is mounting on Apple. They closed my ticket claiming there was a duplicate which you can't see or comment on and it looks like a very early ticket. But that still doesn't explain Iphone inline video playback for WebGL. It requires heaps of complaining for them to provide that it could take years.

I've gone back to check up with Chrome people and it seems Chrome Dev Android is still broken rendering WebGL. There is some flag that can be turned on to get it displaying but it will cause severe dropped frames.

The stock Android browser has CORS issues like Safari so useless. Firefox IOS lacks CORS support also and going to add a ticket to get them to implement it. This CORS stuff crippling everything is just shocking.

So if you guys need a support vector for mobile with all features working. Android Firefox nightly as they fixed orientation bugs there and that is it. Very sad.

[danrossi]

If of interest I have formally made a ticket here so at least Firefox on IOS may have WebVR working and forget about Safari. I've yet to confirm if it can play inline video on Iphone though. They might be quicker to act on CORS support but shocking it does not.

https://bugzilla.mozilla.org/show_bug.cgi?id=1256083

[danrossi]

Confirming Apple's disabling mechanisms force video into a native player for Firefox also. Intentional sabotage from the looks of it.

[makc]

this does not show anything on canvas in firefox on mac either (does work in chrome), not an ios issue.

[makc]

oh, ff starts to show the picture after about 2:50

[danrossi]

Don't get mistaken here. This is a CORS issue it has everything to do with IOS / Safari. I don't know anything about that dash example but I have tested Dash with video textures on Firefox on OSX and its fine.

[tahasabih]

With Dash on Firefox it seems to be a problem with the segments you make when you are processing the video files to make them dash compatible. Not segmenting the files seems to work on Firefox (might also explain why the last few seconds of a segmented video plays).

On 13 Mar 2016, at 9:51 pm, danrossi notifications@github.com wrote:

Don't get mistaken here. This is a CORS issue it has everything to do with IOS / Safari. I don't know anything about that dash example but I have tested Dash with video textures on Firefox on OSX and its fine.

— Reply to this email directly or view it on GitHub.

[danrossi]

This is my packaged files I have tested fine with three.js. I used bento4. the h264 keyframe interval is still (2,3,4) * frame rate like HLS. Not a problem with WebGL. The CORS issue is very much an issue for Webkit and Apple and Firefox on IOS. You have to specify the "crossorigin" attribute.

https://videos.electroteque.org/dash/nodrm/bbb/bbb.mpd

[danrossi]

Sorry to bring this here but people need to know.

Hi guys I just worked out something really wrong and backwards and the same old for mobile which brings me a world of pain.

Samsung Gear tries to launch it's own app with a browser on the S7. I had someone test WebGL / WebVR and when putting on the Gear glasses it launches its own application instead of Chrome. The whole point of WebVR is to launch the WebGL canvas when going fullscreen into the Glasses.

So what is happening is it seems their silly app uses stock Android browser code instead of Chrome and therefore has no support for WebGL video textures and cross domain video and even WebVR api.

Apps are so old news, expecting entire content downloaded to play it. So is progressive downloading mp4 files, it's all segmented streaming now so that model is out of date.

I just worked out that production Chrome seems to render WebGL video textures on the S7 but my S3 is completely broken and been trying to figure it out with chrome devs for months.

No idea what is even going on with IOS still. IOS is completely useless for VR right now too. Webkit still stuffing around with cross domain security support they failed to put in for years rendering VR useless. Hence all this above.

[jonobr1]

FYI: this commit https://github.com/mrdoob/three.js/commit/854ebf5bd6179a3046d4b901b12a9cbc99008c61 fixes the issue for Firefox 👍

[danrossi]

Hi according to this by stealth as Apple ignored my bug tickets and closed them, it seems Apple have fixed it but forked the code it seems in Safari but in macOS. An OS and browser nobody are even using yet.

If they don't merge it back into webkit and provide patched updates to current browsers then people will have to wait but still provide legacy proxy fallback.

This will still be broken for Dash though the proxy hack does not work on does but will confirm again.

https://twitter.com/zenoc/status/742770789880111104

[saar62097]

Hey guys thanks for all the info,

but actually, if someone could explain in detail how to implement the 'reverse proxy hacks' listed here: https://github.com/mrdoob/three.js/issues/8110#issuecomment-183570240

that would be great

thnx Sa'ar

[danrossi]

For apache

 ProxyPass /video/ //videos.electroteque.org/
    ProxyPassReverse /video/ //videos.electroteque.org/

For nginx

location /video/ {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-NginX-Proxy true;
    proxy_pass //videos.electroteque.org/;
    proxy_redirect off;
    }

[danrossi]

From what I gather current webkit browsers will never see the change. They will have it ready for a mac and IOS OS nobody even has. They are still mucking around with CORS for IOS because of "framework issues" holding it back.

Might have to resort to a Flash solution if Stage3D can render hardware accelerated but it doesn't like like it can. It's constantly drawing bitmapdata.

[saar62097]

thanks for the speedy reply

ok, so no workaround for me yet, until Apple fixes this.

What about catching "DOM Exception 18" for fallback options? How can i catch it?

I tried to put it on the render() call of a THREE.WebGLRenderer object but it does not get caught.

[danrossi]

You do the detection on a temp video. ie

testVideo.hasAttribute("crossOrigin")

You can also just do a browser detection. Its Safari and IE 11.

[danrossi]

Just a heads up about this crazy issue. I can't test VR mode on IOS yet unless I deploy to my Ipad.

Cordova apps use WebView. It has options to play inline so works around that pesky problem. It works around user interaction. This one is the craziest of them all and has been driving me crazy.

WebView seems to work around CORS completely even if its a local html file running on file://. WebGL is working in Cordova apps without the need for the crossorigin attribute. This is both shocking and good news. It won't need the proxy hack but Safari IOS/OSX will.

I can't check pseudo VR orientation controls but touch controls is working so is the old cardboard stereo effect. I will be updating that Webkit ticket.

Testing on Android is a problem because of the Chrome rendering bug that I spent 3 months trying to get the Chromium people to fix and gave up. It's completely black sadly and hopefully not a CORS issue that isn't showing errors.

[florianmaxim]

so did I get this right? it is STILL the only way to use webgl video textures with webgl by implementing that proxy hack on the server?

[danrossi]

Safari is completely and utterly bolloxed and so is IOS. If you use Cordova WebView based apps, you can play inline, auto play and don't ask why but CORS is not an issue. That would be for mp4 / HLS on IOS.

But as I've reported and found out . They throttle older devices and slow them down to force people to upgrade. The Ipad 3 is useless for WebGL and renders back at 5fps. It needs to be a newer device.

In Safari on OSX, mediasource is completely and utterly bolloxed. the CORS proxy doesn't work there. It needs to be an mp4 file or native HLS also. With native HLS you provide a CORS proxy url.

These duds at Apple are planning to release fixes in macOS , so in something nobody even has yet.

They are trying to fix underlying framework issues in IOS. So I doubt CORS will even be fixed in the next release of IOS.

Neither have WebVR either, both are a joke. They treat WebVR as some kind of joke and are years behind.

Well WebVR on desktop needs the Windows SDK's so OSX is not an option yet anyway but at least the apis could be there for client testing ? If Android has a default cardboard display no idea why IOS can't and have WebVR enabled ?

They left this CORS issue lingering for years and this is the product of that.