Youtube videos don't render in the Off-Screen Rendering Example

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Run the OSR example
2. For example http://www.youtube.com/watch?v=wdFAqD2-XC0&feature=topvideos in 
the address bar of the offscreen plugin

What version of the product are you using? On what operating system?
r212, Windows 7 x64

Original issue reported on code.google.com by predator...@yahoo.com on 30 Mar 2011 at 11:03

[GoogleCodeExporter]

YouTube uses a windowed Flash plugin for playing video. It would seem that 
Flash doesn't support the WM_PRINTCLIENT message. To make this work it will 
probably be necessary to implement the alternative approach for windowed 
plugins which involves making them visible but a little bit off the screen, and 
then copying from the window's device context. It should be noted that HTML5 
video and windowless Flash plugins both currently work with the off-screen 
rendering.

Original comment by magreenb...@gmail.com on 30 Mar 2011 at 4:12

• Changed state: Accepted

[GoogleCodeExporter]

Odd thing is, when an youtube page is loading, the flash plugin is visible for 
a second, there are some ad banners on the right which are visible to me. Also, 
the adobe flash page renders fine with the banner (which is flash) rendering 
just right. 
Also:

http://www.gametrailers.com/video/fault-line-battlefield-3/712299

renders just fine. 

I believe it is an issue related only to youtube's flash player.

Original comment by predator...@yahoo.com on 30 Mar 2011 at 4:35

[GoogleCodeExporter]

I'm not sure why the Flash video window is visible for a second.

What you're seeing in the other places are probably windowless Flash plugins. 
This link provides more information about windowed vs windowless plugins: 
http://neugierig.org/software/chromium/notes/2009/07/windowed-windowless-plugins
.html

Original comment by magreenb...@gmail.com on 30 Mar 2011 at 4:54

[GoogleCodeExporter]

Is there any ETA on when this will be fixed? This is, as far as I can tell, the 
only reason for me to not adopt CEF right now.

Original comment by dreijer...@gmail.com on 9 Apr 2011 at 4:22

[GoogleCodeExporter]

I'm also using OffScreen rendering

This video is played correctly: 
http://www.youtube.com/watch?v=ovivii38IiA&feature=topvideos

This video isn't played (but audio is ok): 
http://www.youtube.com/watch?v=Y4h8uOUConE&feature=relmfu

Original comment by raffaele...@gmail.com on 13 Apr 2011 at 6:25

[GoogleCodeExporter]

I'm not able to build due to low disk space currently, I would debug it to see 
what's really going on when the page loads. Seems to me like something happens 
when HandleLoadEnd fires, as the flash window is visible before the page fully 
loads.

I also hear sound from the videos, just the flash window disappears after 
loading the page

Original comment by predator...@yahoo.com on 13 Apr 2011 at 9:40

[GoogleCodeExporter]

Off-screen rendering of windowed plugins currently works as follows:

1. Parent the plugin to a hidden window created in 
browser_webview_delegate_win.cc BrowserWebViewDelegate::CreatedPluginWindow().
2. Draw the plugin by sending it a WM_PRINT message in webwidget_host_win.cc 
WebWidgetHost::Paint().

To fix the off-screen rendering problem with acrobat and flash we need to 
instead do the following:

1. Parent the plugin to a visible window positioned immediately off the screen.
2. Use RedrawWindow() and GetDC() to force immediate redrawing of the plugin 
window and retrieve the result.

Original comment by magreenb...@gmail.com on 27 Apr 2011 at 2:57

[GoogleCodeExporter]

I've started working on it.

In browser_webview_delegate_win.cc, 
BrowserWebViewDelegate::CreatedPluginWindow() I changed the create window with 
this:

HWND parent = CreateWindow(kPluginWindowClassName, NULL, 
WS_OVERLAPPED|WS_CLIPCHILDREN|WS_CLIPSIBLINGS|WS_VISIBLE,
0, 0, 1024, 1024, NULL, NULL, GetModuleHandle(NULL), NULL);

added WS_VISIBLE and set initial size to a temporary value of 1024x1024. 
I made this to have the window visible, later I will set the right values.

In webwidget_host_win.cc, WebWidgetHost::Paint() i commented this

//SendMessageToPlugin(geom->window, WM_PRINT,
// reinterpret_cast<WPARAM>(drawDC),
// PRF_OWNED | PRF_ERASEBKGND | PRF_CLIENT | PRF_NONCLIENT);

and added after these lines:

RedrawWindow(geom->window, NULL, NULL, RDW_UPDATENOW);
HDC hdc = GetDC(geom->window);
BitBlt(drawDC, geom->clip_rect.x(), geom->clip_rect.y(),
      geom->clip_rect.right(), geom->clip_rect.bottom(), hdc, 0, 0,SRCCOPY);
ReleaseDC(geom->window, hdc);

Now, with this solution I have 2 problem: 
HandlePaint doesn't fire when flash is updated but this I don't think is a 
problem now.
The second is that when the plugin window goes offscreen or it's covered by 
another window, the bitblt don't gives me the correct image.

Have you any hint to go on?

Original comment by alberto....@gmail.com on 28 Apr 2011 at 2:55

[GoogleCodeExporter]

> HandlePaint doesn't fire when flash is updated but this I don't think is a 
problem
> now.

We'll probably need to do either of the following:

1. Use SetWindowsHookEx to intercept WM_PAINT messages sent to the plugin 
window, or
2. Use a timer to redraw the plugin window at a set interval.

> The second is that when the plugin window goes offscreen or it's covered by 
> another window, the bitblt don't gives me the correct image.

Try the following combination of flags to RedrawWindow:
RDW_INVALIDATE | RDW_UPDATENOW | RDW_NOERASE | RDW_ALLCHILDREN | RDW_FRAME

Original comment by magreenb...@gmail.com on 28 Apr 2011 at 3:13

[GoogleCodeExporter]

> Try the following combination of flags to RedrawWindow:
> RDW_INVALIDATE | RDW_UPDATENOW | RDW_NOERASE | RDW_ALLCHILDREN | RDW_FRAME

Still don't work, same behavior.

Original comment by alberto....@gmail.com on 28 Apr 2011 at 3:27

[GoogleCodeExporter]

>> Try the following combination of flags to RedrawWindow:
>> RDW_INVALIDATE | RDW_UPDATENOW | RDW_NOERASE | RDW_ALLCHILDREN | RDW_FRAME

> Still don't work, same behavior.

Can you post a patch with your changes so far? You can create a patch file 
using TortoiseSVN or the 'svn diff' command-line tool. Making it work will 
likely require some trial and error playing with various flags and screen 
placement.

Original comment by magreenb...@gmail.com on 28 Apr 2011 at 3:41

[GoogleCodeExporter]

Here is in attach the patch file

Original comment by alberto....@gmail.com on 28 Apr 2011 at 3:52

Attachments:

• webwidget_host_win.cc.diff

[GoogleCodeExporter]

It looks like Flash might support WM_PAINT with a target device context handle:
http://www.gamedev.net/topic/327506-forcing-a-hidden-window-to-paint-to-an-arbit
rary-dc/

Original comment by magreenb...@gmail.com on 28 Apr 2011 at 4:14

[GoogleCodeExporter]

i think the berkelium project has this working, they have a movie of them 
rendering youtube videos to an opengl surface here:

http://www.sirikata.com/blog/?p=115

source code here:

https://github.com/sirikata/berkelium

Original comment by anthony....@gmail.com on 28 Apr 2011 at 9:56

[GoogleCodeExporter]

Also in Chromium this working. 
But the problem is to find where is the piece of code that render it. 

Do anyone knows where is it? 
When we found it we can work on it together to integrate it on CEF.

Original comment by alberto....@gmail.com on 29 Apr 2011 at 10:18

[GoogleCodeExporter]

The attached patch gets windowed flash plugins mostly working with off-screen 
rendering. Flash buttons (like "pause" with a youtube video) show correct click 
state but don't actually perform the action. The problem may have something to 
do with activation of the host window on mouse click event and perhaps 
PLUGIN_QUIRK_HANDLE_MOUSE_CAPTURE. Also, we may need to patch SetCursor (see 
PLUGIN_QUIRK_PATCH_SETCURSOR for an example) to handle cursor changes 
correctly. This patch redraws windowed plugins at an arbitrary 30fps.

Original comment by magreenb...@gmail.com on 29 Apr 2011 at 6:52

Attachments:

• windowed_plugin_r223.patch

[GoogleCodeExporter]

Hello, I think the solution is pretty simple.
Just Force the flash plugin to work in opaque wmode(if trasparent or opaque 
mode is not found), so the plugin is windowless.
Look at berkelium patch: 
https://github.com/sirikata/berkelium/blob/chromium8/patches/chromium_wmode_opaq
ue.patch
We could do the same thing with silverlight adding "<param name=windowless 
value=true />" but in this case i haven't found files to patch yet.

I hope this help you.

Original comment by gioggi...@gmail.com on 30 Apr 2011 at 8:59

[GoogleCodeExporter]

I can confirm it's possible to force also silverlight adding code like:
surrogate_names.push_back("windowless");
surrogate_arg_values.push_back("true");

Original comment by gioggi...@gmail.com on 30 Apr 2011 at 2:24

[GoogleCodeExporter]

Sounds like a better solution to me, since it enable us to read and update only 
the dirty area. With a windowed plugin would be a bit hacky to do the same, 
plus it would depend on the platform

Original comment by wesdataq...@gmail.com on 30 Apr 2011 at 3:48

[GoogleCodeExporter]

About comment 17, do you think this could be applied to plugin in general or 
only to Flash?

Original comment by raffaele...@gmail.com on 2 May 2011 at 10:37

[GoogleCodeExporter]

About comment 16, i have tried this patch but it doesn't work

Original comment by raffaele...@gmail.com on 2 May 2011 at 3:26

[GoogleCodeExporter]

> About comment 16, i have tried this patch but it doesn't work

Did you test with the cefclient application built at CEF revision 223? What 
version of Windows are you using? What URL did you test with?

Original comment by magreenb...@gmail.com on 2 May 2011 at 3:36

[GoogleCodeExporter]

I'm sorry, i don't use cef. I use a modified version of berkelium and i solved 
the problem with youtube before their patch(some youtube videos work in 
wmode=direct). I just tried to help you reporting my experience with berkelium.
In my spare time i will try to install cef.

Original comment by gioggi...@gmail.com on 2 May 2011 at 7:30

[GoogleCodeExporter]

About URLs, i have directly created the test web page using different wmode(s).

Original comment by gioggi...@gmail.com on 2 May 2011 at 7:34

[GoogleCodeExporter]

It's me again. Take a look here:
http://code.google.com/p/chromiumoffscreenrenderer/source/browse/trunk/Awesomium
/webview/WindowlessPlugin.h?spec=svn25&r=25#172
It's an old version of awesomium (chromium based too), maybe it can help you.
I'm sorry but i don't know how cef works yet.

Original comment by gioggi...@gmail.com on 2 May 2011 at 8:25

[GoogleCodeExporter]

I've seen Awesomium code in some early release (when it was to be LGPL) and as 
I recall it does the same - forces flash to windowless. It's a good approach, 
works flawless. The thing is, it doesn't work for plugins like VLC player

Original comment by predator...@yahoo.com on 3 May 2011 at 12:10

[GoogleCodeExporter]

I agree that forcing to windowless makes the most sense if it works flawlessly 
for Flash and Silverlight. As comment #26 states, it probably won't work for a 
number of common plugins including Acrobat.

Original comment by magreenb...@gmail.com on 3 May 2011 at 12:35

[GoogleCodeExporter]

I've updated chromium to 80310, and cef to 223, applied the patch on comment 16 
but still don't work.
I have WinVista 32bit, VisualStudio 2008 Express Edition, i've used many link 
on youtube. I've also used link on comment 5 but none of them works.

Original comment by raffaele...@gmail.com on 3 May 2011 at 4:10

[GoogleCodeExporter]

@comment#28:

I think this is an issue where Vista doesn't render unless the off-screen 
window has at least 1 pixel on the screen. Try removing the "-5" from the 
SetWindowPos call in the patch on lines 106 and 107. You can play with this X,Y 
position value to identify the "most off-screen" window position that still 
renders correctly.

Original comment by magreenb...@gmail.com on 3 May 2011 at 4:40

[GoogleCodeExporter]

Sorry but still don't works. Tested on WinVista and WinXp.
I've tried to move the plugin window within the visible area and offscreen area 
but the copy was made only for the visible area.

I think that BitBlt function could be the problem, or maybe is just a matter of 
choosing the correct flag for the last parameter. 

I've also tried two different flag: SRCCOPY (that copy only the part of window 
visible) and CAPTUREBLT (paints all black).

Original comment by raffaele...@gmail.com on 4 May 2011 at 8:36

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Revisions 229 and 230 force Flash and Silverlight plugins to use windowless 
mode with off-screen rendering using the approach described in comments #17-18. 
This does not fix the display for other windowed plugins like Acrobat.

Original comment by magreenb...@gmail.com on 10 May 2011 at 4:02

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Issue #228 has been created to track off-screen rendering support for other 
windowed plugins like Acrobat.

Original comment by magreenb...@gmail.com on 10 May 2011 at 5:17

• Changed state: Fixed