Android Chrome Dev still a problem with WebGL video textures

[danrossi]

Hi I've reported it to chromium bug tracker

https://code.google.com/p/chromium/issues/detail?id=578065

however there is a clear issue with some shader program within three.js as a raw webgl example doesn't seem to experience the same issue ?

Android Chrome did have issues with rendering video textures but it seems they fixed it in Dev. Both production and beta suffer exactly the same issue.

I'm really unsure who's problem it is.

I put the three.js video demo up on jsfiddler after loading the library over https S3 and add the CORS flag to even get it working. It produces a tonne of shader program console logs so perhaps it's an issue with the shader program used ?

https://jsfiddle.net/0agb03Ld/6/

That does not work and produces this

https://chromium.googlecode.com/issues/attachment?aid=5780650000000&name=Screen+Shot+2016-01-15+at+10.22.46+pm.png&token=ABZ6GAfcYv_HTynnU3fBsawJt4gMcn7Urg%3A1452858267536&id=578065&mod_ts_token=ABZ6GAemrbWl7j3UgpZc5nptfd7paSbkcQ%3A1452858267536&inline=1

This raw webgl function is ok in Dev but produces the same bug in the other browser versions.

https://jsfiddle.net/7t77rz6L/2/

https://chromium.googlecode.com/issues/attachment?aid=5780650002000&name=Screen+Shot+2016-01-15+at+10.51.32+pm.png&token=ABZ6GAdaPCEzbLEEf478cRVlas2ilE5VXw%3A1452858267537&id=578065&mod_ts_token=ABZ6GAfLFBaD9NA3lw4Hzskb4hOaKtW8eQ%3A1452858267537&inline=1

Let me know what to do as only Firefox is working currently.

[makc]

https://jsfiddle.net/7t77rz6L/2/ is working fine in desktop chrome so I guess it's android chrome problem?

ah wait, I have canary.

[danrossi]

Yes this is an Android Chrome problem. There was tonnes of tickets in regards to video textures there that they quickly closed it seems. the actual issue is evident on the production and beta versions and they "fixed" a regression issue it seems in dev which took me a while to work out. I'm unsure how it was ever working for anyone ?

however comparing both versions there is an obvious difference which I'm unsure what I can only guess a shader program. I don't know who is to deal with it sorry so I tracked it in both places.

[danrossi]

That jsfiddler link you posted is also the working version in Dev Chrome. I repurposed the example that krpano added to one of their issues. The three.js is not working I'm afraid. There is no issues on desktop although I'm yet to even get to windoze I'm stuck trying really hard to get something barely functional on android.

The standard webgl examples with different textures are fine so webgl isn't broken as such. its video textures, although 2D drawing video to a canvas (drawImage) has exactly the same problem as the screenshot so a snapshot feature I have built has the same issue.

[makc]

7(index):1 [.CommandBufferContext]GL ERROR :GL_INVALID_OPERATION : glFramebufferTexture2D: <- error from previous GL command

if only we knew what was that previous GL command, right?

[makc]

out of interest tried to open this fiddle on android via browserstack. got this:

[Rinkana]

Mobile devices require user input to start a video. You get the same issue if you embed a plain video tag on a website with autoplay. The autoplay does not work and the user needs to click the video in order to start it.

Mobile browsers do this in order to save bandwith.

What you can try is to bind a touchdown handler to the body and start playing the video from there.

This same issue should also happen on a iOS device.

And as a last thing, you say that the raw WebGL does not have the same issue. Maybe they load the video in another way so it does not have the same issue. You could check into that.

[makc]

@Rinkana indeed, but I think @danrossi never had this issue for some reason?

[danrossi]

Sorry that is the exact code from the demo in the sources. I will add in the user interaction code. All my test chrome browsers have that disabled. It's in chrome://flags.

My question is the other way, I'm confused how these video demos were ever working for people on Chrome Android :O

[Rinkana]

Have you checked the texture size? It seems that that has something to do with it: https://github.com/mrdoob/three.js/issues/6606

[danrossi]

I've tried the POW function and it makes no difference. Ive made its dimensions now the same as the other one.

https://jsfiddle.net/0agb03Ld/8/

[danrossi]

If you download Xamarin Android Player it's exactly the same.

I've added the POW function back into this version but just on the video element. I'm assuming the video texture needs to be upscaled and then the webgl canvas can be resized. That might distort the picture though ?

I now see a black picture or the same broken picture with text from other parts of the screen, in Chrome Beta its the same funny broken images as above. I get a tonne of logs like this

[.CommandBufferContext]GL ERROR :GL_INVALID_OPERATION : glFramebufferTexture2D: <- error from previous GL command
(index):1 [.CommandBufferContext]GL ERROR :GL_INVALID_OPERATION : glTexImage2D: <- error from previous GL command

https://jsfiddle.net/0agb03Ld/12/

I've tried to make the raw webgl example exactly the same with the POW function. It still displays regardless although as before it will have the same bug in the other Chrome browser versions.

https://jsfiddle.net/7t77rz6L/5/

[danrossi]

Just to show how crazy this whole Android business is. The same raw webgl version that is working in Dev does not in Production chrome version 47 because that version has the regression issue with video textures. It suffers the same bug as the three.js example.

easily to replicate in Xamarin Android Player.

https://chromium.googlecode.com/issues/attachment?aid=5780650006000&name=Screen+Shot+2016-01-18+at+6.03.25+pm.png&token=ABZ6GAcRUO87XdMkZu2_O07wyG-CUCXHzw%3A1453099976127&id=578065&mod_ts_token=ABZ6GAcWviybxKN-JeyFK5SWsjeZNTce3A%3A1453099976127&inline=1

[danrossi]

I'm finding other examples. This is from MozVR. Completely broken in Chrome Dev. It's using three.js also.

https://aframe.io/examples/showcase/videosphere/

And on that note the demo doesn't even play in Firefox either but its a problem with the video. Situation on Android no good right now and I have absolutely no idea how people have it working or have even tested it. This is linked on the MozVR page http://mozvr.com/

[danrossi]

There has been no progress on either side. I'm still trying to hunt down more examples of what may / may not be working with three.js variants. I say completely broken still on Android Chrome but not sure how to debug why I'm afraid.

[makc]

modified your last fiddle to handle the click https://jsfiddle.net/7t77rz6L/6/ browserstack says it works in android 5 / google nexus 6, 4.4 / nexus 5 does not connect ( 4.4 / nexus 7 works fine too, 6 / nexus 6 works. are you sure the problem is not with your device?

[danrossi]

@makc that is the raw WebGL example that is working but in Chrome Dev as they "fixed" a bug in that version. The production version still has the video texture bug so that example you posted won't work either and suffers the same problem.

I have had the same problem trying to draw a video frame to a 2D canvas, displays the same so it's related.

Any three.js variant does not. I've mentioned it a few times above but it has blown out of course.

This is straight from the demos.

https://jsfiddle.net/0agb03Ld/12/

It is not a POW texture issue either but I am now starting to resize the video tag with POW functions.

the code for that is

function nextPowerOf2(v)
{
    v--;
    v|=v>>1;
    v|=v>>2;
    v|=v>>4;
    v|=v>>8;
    v|=v>>16;
    return ++v;
}

If you download Xamarin Android Player, Chrome Dev, load the three.js example it will show the video texture issue.

I've asked for an update on their end however considering one demo is working and the other is not they might take priority on it and think they fixed the bug already.

[danrossi]

Hi there, it seems a flag in dev Android chrome "unified media pipeline" fixes this chrome bug. I am not sure why the raw webgl works but the three.js does not still but it now renders. It seems the issue is related to 2D canvas drawing of video I've had also and that fixes the same problem. The 2D drawing and webgl canvas bugs are related.

An update is here

https://code.google.com/p/chromium/issues/detail?id=578065#c18

[danrossi]

Enabling the "unified media pipeline" flag in Chrome Dev fixes rendering of three.js video textures but then it produces a problem of cpu performance problems.

It is not there yet but this might be a Chromium issue still even though the raw webgl demo works.

[mrdoob]

Have you checked that the filtering in the WebGL and three.js demos are the same (nearest, linear, etc)

[danrossi]

I've updated with those said filters and still broken picture.

They claim they can't see a problem with their nexus devices but I see a problem with Samsung S3. It begins to work with that suggested flag enabled but produces performance problems.

There is a heap of messages about unsupported shaders if that is of any help ? ie

THREE.WebGLRenderer: OES_texture_float extension not supported.
three.js:28892 THREE.WebGLRenderer: OES_texture_float_linear extension not supported.
three.js:28892 THREE.WebGLRenderer: OES_texture_half_float extension not supported.
three.js:28892 THREE.WebGLRenderer: OES_texture_half_float_linear extension not supported.
three.js:28892 THREE.WebGLRenderer: ANGLE_instanced_arrays extension not supported.
three.js:28892 THREE.WebGLRenderer: OES_element_index_uint extension not supported.
three.js:28961 THREE.WebGLRenderer: highp not supported, using mediump instead.
three.js:28892 THREE.WebGLRenderer: EXT_texture_filter_anisotropic extension not supported.

https://jsfiddle.net/0agb03Ld/13/

[danrossi]

It can be replicated easily also with Chrome Dev on Xamarin Android Player using the Nexus Android 4.4. There has been tickets in the past in regards to video textures.

They fixed some regressionin Chrome Dev but in this case any three.js variant out there does not work without the pipeline flag enabled but that raw webgl example does. The raw webgl example It is not my code , it was derived from krpano for a chromium ticket about the same issue.

So I am not across of any differences apart from these shaders after adding the extra texture filters.

[danrossi]

I've upgraded to version Android Chrome Dev 50.0.2645.0 and without doing anything it is now working. I uninstalled Chrome dev then reinstalled and got a fright because the same rendering bug appeared but then it went away again !

It does not require the unified media pipeline which was causing performance issues anyway.

https://jsfiddle.net/0agb03Ld/17/

[danrossi]

There is now obvious frame dropping and performance issues not happening with Firefox. It is the same for the raw webgl example. It's hitting high cpu when it should be on the gpu ? Really unsure how any of this was ever working for people on Android Chrome.

I suppose I will have to work with them and is not a three.js issue now.

[danrossi]

Hi I don't know what to do with this one. I don't believe it's related to three.js so I guess it can be closed and archived unless something three.js is doing is causing it to not render properly ?

Is Android Chrome even working with video textures for people ?

It was a known issue and still is. It doesn't render on my S3 without this pipeline flag enabled in Chrome dev but that causes severe frame dropping and performance issues. They claim they see nothing although one of their team managed to experience the rendering bug but have done nothing to fix it.

To reiterate it renders the raw webgl example linked here for some reason but not three.js I assume because of added shaders ?

see https://bugs.chromium.org/p/chromium/issues/detail?id=578065#c44

As far as I am aware now both IOS/Iphone and Android Chrome have various bugs preventing WebVR from even functioning sadly.

[danrossi]

I'm closing this. I have no idea if three.js can fix these problems even those the raw webGL is working and three.js is not.

I have now discovered after someone tested for me production Chrome is somehow working on the S7. My S3 is still broken so there is a graphics hardware issue.

I will close this for now but I believe S3 still needs to be supported.

[mrdoob]

For the record...

Both the three.js and webgl versions work on Chrome 49/Nexus 5x:

https://jsfiddle.net/0agb03Ld/20/ https://jsfiddle.net/7t77rz6L/12/

I just had to add the click event listener as videos don't autostart on mobile.

[danrossi]

It must be a problem with S3 graphics. I don't have access to any of these NExus devices which the chrome people are also using. The Chrome 49 is also working on S7 then too no worries. It was only functioning for me in Chrome Dev with a special flag enabled they are messing with but was causing performance problems !

[saar62097]

Hey guys thnx for the info.

I am having the same issue (video wont render on Android eith three.js) and my code is supposed to run on multiple apps that i have no control of. I see there is no js error with this issue. Can u reccomend a way for me to detect this programmatically in order to show users a fallback?

Thnx

[danrossi]

There is known issues in production Chrome pre the S7. To get it working you have to use Chrome Dev and turn on the media pipeline flag which causes severe performance and dropped frames issues.

For pre S7 you want pseudo VR and cardboard support at the very least. Ironically however WebVR is completely useless with Gear on the S7. It launches an app replacing the browser. Which uses the stock WebView which WebGL doesn't function in at all, a bit of a joke. I have to debug and confirm once I get a hold of it.

I have confirmed though WebGL on the production Chrome on the S7 works with video textures.

Firefox is the only usable browser for Android right now really, Chrome is inconsistent. Orientation API's are fixed in Firefox so it's up to them to release it.

The chromium people only test on Nexus and are very hard to get through to. I ran out of steam spending months dealing with them. My ticket is so massive, constantly repeating myself it's unmanageable now.

I will try and get back to it soon but I've been waiting for the "others" to resolve things to make web VR useful on mobile.

This is one of them, Webkit and IOS is the other.

[danrossi]

Hi guys I have come back to this pesky problem. I have not got access to a Nexus device and it sounds like this only works in Nexus devices. Does anyone have access to a Samsung device ?

I updated the Chromium ticket. The easiest way to see the issue is within a Xamarin Android emulator, doesn't matter which version. I have never managed to get this working properly on Chrome Android at all. It's just a black screen or it has ghosting artefacts. The same issue is with Canvas based drawing so not just WebGL. So when trying to draw video to a canvas it's also broken and produces the same result.

[danrossi]

I managed to muck around in the emulator. It works on Chrome Dev because of the said flag they enabled. But it causes frame dropping and video performance issues on the S3. On the emulator frames are ok but the audio cut out. It doesn't like Mp4 playback either it has to be webm.

On Chrome production is the said problem.

[danrossi]

OK this might be of interest in terms of Android and IOS support.

IOS/OSX Safari requires the CORS proxy hack. The webkit people are still hopelessly working on it and may become support for it in later IOS/OSX versions.

IOS WebView WebGL has no CORS restrictions it works out of the box perfectly. You can also set it to play inline video which is a requirement and auto playback if you wish.

IOS Iphone for Html5 video WebGL video textures. It's completely bolloxed because of no inline playback support. Inline playback is being supported in newer IOS versions not even out yet. So cordova based apps for IOS for Iphone specifically is the only way to go there.

Iphone is useless right now.

There is noticeable frame dropping issues I now need to work out on the Ipad3. I believe WebGL is possibly not working in WebView on IOS either.

Samsung S3 Chrome has the said problem rendering video to either WebGL or drawing to the 2D canvas. It produces weird ghosting or a black output. They have fixed something in Chrome Dev for this but I believe their fix is causing frame dropping issues on S3 so bad performance.

Samsung S3 Firefox I believe is the only possible support but it also has specific requirements. It can't play webm or mp4. It needs to be ogv, HLS, Dash or it produces a black frame. It seems hardware dependant.

The orientation bugs that were put into nightly I'm not sure are in the stable version of Firefox ?

Samsung S3 stock browser has no WebGL support.

Samsung S7 Chrome played back WebGL video textures perfectly without a hitch. No frame dropping and the touch and orientation controls were smooth and no issues.

I even got it partially working in Samsung Gear which requires the Samsung VR browser, It's supposed to support WebVR but it was loading the cardboard stereo view so doesn't look like it yet. It has to be manually changed into 360 mode also. Upon doing that I discovered a field of view setting problem, its too close.

There was black bars on both sides for some reason when turning around. I now need to get both an S7 and Gear to properly work out these issues.

So after all that headache it was hardware related for Chrome as Firefox works.

After all those issues trying to muck around with emulators which also had the same hardware rendering bug and trying to hopelessly get play store installed to even get Chrome on them. They are a waste of time and a headache too.

[danrossi]

Just a side note for anyone looking. Some things have changed recently in Chrome on Android and both stable and dev have started rendering on the S3 again. So no ghosting or black frame with WebGL errors other then unsupported shader functions.

But generally older models like the S3 is too laggy to use for video texture WebGL rendering. It still causes underlying video playback and performance issues with dropped frames.

Probably bare minimum would be S5. The S7 provides a constant frame rate, even better than a laptop.