CEF3: Add off-screen rendering support

[GoogleCodeExporter]

Add off-screen rendering support. See 
http://magpcss.org/ceforum/viewtopic.php?f=10&t=645 for CEF3 information.

Original issue reported on code.google.com by magreenb...@gmail.com on 13 Feb 2012 at 7:15

• Blocking: #887

[GoogleCodeExporter]

@comment#100: (minor comment) so it is global setting, and both modes can't be 
mixed together? What happens if renderer limit reached and renderer process 
will be shared for host OSR browser and windowed browser inside?

Original comment by fdd...@gmail.com on 4 Apr 2013 at 4:04

[GoogleCodeExporter]

@comment#100: A command-line flag won't work because a windowed and windowless 
browser may theoretically share the same render process if they have the same 
origin, or if running in single-process mode. Consider adding a new member to 
the CefBrowserInfo class instead. The information is then sent to the render 
process via CefProcessHostMsg_GetNewBrowserInfo 
(CefContentRendererClient::RenderViewCreated and 
CefBrowserMessageFilter::OnGetNewBrowserInfo).

Original comment by magreenb...@gmail.com on 4 Apr 2013 at 4:10

[GoogleCodeExporter]

Attaching the basic osr_mac patch, rebased on top of trunk r1189.
I'm working on the popups patch and will upload it when it is ready.

Original comment by horia.ol...@gmail.com on 9 Apr 2013 at 2:21

Attachments:

• osr_mac4.1.patch

[GoogleCodeExporter]

Attaching the updated popups patch. This applies on top of the previous patch.

1. Changed, so that it fails if any of the width/height dimensions is 0.

2, 4 - Done.

3. Fixed, as previously discussed. However, this involved a patch for the 
WebKit WebView. Because the choice of popup menu type was govenered by a file 
static variable, the WebView API needed patching to make it work on a per 
instance basis.

Original comment by horia.ol...@gmail.com on 10 Apr 2013 at 4:31

Attachments:

• osr_mac_popups4.patch

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

@comment#104: Including the WebView patch in CEF is fine as a temporary fix. 
Can you file a bug with Chromium for the WebView changes, submit it for review, 
etc? Thanks.

Original comment by magreenb...@gmail.com on 12 Apr 2013 at 6:02

[GoogleCodeExporter]

@comment#103-104: Building with your patches using the 10.6 SDK on OS-X I'm 
getting the following errors:

cefclient_osr_widget_mac.mm:65:17: Use of undeclared identifier 
'NSWindowDidChangeBackingPropertiesNotification'; did you mean 
'NSWindowDidChangeScreenProfileNotification'?
cefclient_osr_widget_mac.mm:72:22: Use of undeclared identifier 
'NSWindowDidChangeBackingPropertiesNotification'; did you mean 
'NSWindowDidChangeScreenProfileNotification'?
cefclient_osr_widget_mac.mm:277:5: Instance method 
'-setWantsBestResolutionOpenGLSurface:' not found (return type defaults to 'id')
cefclient_osr_widget_mac.mm:728:45: Use of undeclared identifier 
'NSBackingPropertyOldScaleFactorKey'
cefclient_osr_widget_mac.mm:762:25: Instance method '-backingScaleFactor' not 
found (return type defaults to 'id')
cefclient_osr_widget_mac.mm:762:23: Assigning to 'float' from incompatible type 
'id'
cefclient_osr_widget_mac.mm:818:20: Instance method '-convertSizeToBacking:' 
not found (return type defaults to 'id')
cefclient_osr_widget_mac.mm:818:18: No viable overloaded '='
cefclient_osr_widget_mac.mm:828:26: Instance method '-convertRectToBacking:' 
not found (return type defaults to 'id')
cefclient_osr_widget_mac.mm:828:12: No viable conversion from 'id' to 'NSRect' 
(aka '_NSRect')
cefclient_osr_widget_mac.mm:841:26: Instance method '-convertRectFromBacking:' 
not found (return type defaults to 'id')
cefclient_osr_widget_mac.mm:841:12: No viable conversion from 'id' to 'NSRect' 
(aka '_NSRect')

It seems that NSWindowDidChangeBackingPropertiesNotification and related 
functionality was added in 10.7.3. 10.6 is still a supported platform so please 
change this code to compile/run with the 10.6 SDK.

Also, your WebView changes did not compile on OS-X due to the use of the static 
WebView::setUseExternalPopupMenus() function in RenderThreadImpl::Init().

Attached is an updated patch at trunk revision 1220 that combines your 
osr_mac4.1 and osr_mac_popups4 patches with the following additional changes:

1. Refactor the WebView patch to compile, minimize the required changes, 
properly initialize variables, etc.
2. Various documentation and style fixes.

Please use this new patch as the basis for fixing the above 
cefclient_osr_widget_mac.mm errors or just attach a new copy of 
cefclient_osr_widget_mac.mm with the necessary changes. Thanks :-).

Original comment by magreenb...@gmail.com on 12 Apr 2013 at 9:26

Attachments:

• osr_mac5.patch

[GoogleCodeExporter]

Thanks for taking the time to review the patch, and fixing stuff here and there 
:)

All of the above API errors are directly related to supporting HiDPI. They 
indeed became available only in the 10.7 OS and SDK, but that is because HiDPI 
became available to developers at that time.

The Mac OSX SDKs have a mechanism in place to build a binary with a minimum 
'deployment target' OS, for backwards compatibility with previous OS versions. 
This way, with a 10.7 SDK one can set the deployment target to 10.6, and the 
resulting binary will be able to run on 10.6 and higher. The newer APIs are 
used only if they are available at runtime, but the developer is responsible 
for checking if they are (respondsToSelector).

cefclient is the only project that actually accesses the platform specific view 
/ screen APIs, to determine DPI settings. Hence the only one that requires a 
specific OS X SDK is cefclient. The rest of the code is platform agnostic.

The alternatives I see are:
1. Use ifdefs to only exclude the HiDPI code from 10.6 SDK builds of cefclient. 
However, this means that the resulting binary would not have HiDPI support even 
when run on 10.7 or 10.8. To enable that support, another build with a 10.7 SDK 
or newer would be needed.
2. Migrate to using Mac OSX SDK 10.7 to build cefclient, but set the deployment 
target to 10.6.

The 2nd option is what I had in mind initially, as I had set in place the 
runtime checks to make cefclient work on 10.6. I've added an extra runtime 
check on the notifications code, because for some reason it seemed to crash on 
10.6.

I am attaching cefclient_osr_widget_mac.mm with all the offending code between 
'ifdefs', but if you remove those, the code should work fine on 10.6 as 
described in 2.

Original comment by horia.ol...@gmail.com on 15 Apr 2013 at 6:58

[GoogleCodeExporter]

@108 - adding the attachment, which for some reason did not get attached.

Original comment by horia.ol...@gmail.com on 15 Apr 2013 at 7:00

Attachments:

• cefclient_osr_widget_mac.mm

[GoogleCodeExporter]

Mac support has been added in trunk revision 1226 and 1453 branch revision 
1227. Thanks to all the people who helped out on this!

Build with the 10.7 SDK (set GYP_DEFINES='mac_sdk=10.7') to include Retina 
support in the cefclient OSR example as described in comment #108.

Original comment by magreenb...@gmail.com on 16 Apr 2013 at 7:26

[GoogleCodeExporter]

I am rendering (Windows 7) off-screen using transparent painting. All of my 
transparent png's render just fine but fonts don't (see attached image). I'm 
not using any custom fonts, just the standard Serif Font. Is this a limitation 
of Cef?

In the Cef example, Cef3 is rendering off screen and storing the result as a 
texture on a quad. A white quad is behind the chrome texture.

Original comment by dpokrovs...@secondstory.com on 23 Apr 2013 at 9:10

Attachments:

• cef_offscreen_font.png

[GoogleCodeExporter]

@comment#111: You can try toggling the use of SKIA vs GDI for text rendering to 
see if it makes an improvement. Change the value of 'enable_skia_text' in 
cef.gypi and then re-build CEF/Chromium.

Original comment by magreenb...@gmail.com on 23 Apr 2013 at 9:15

[GoogleCodeExporter]

What would be a good start to implement wmodes of 'direct' or 'window' for 
flash in offscreen CEF (Windows)?
Since flash has its own HWND for these render-modes, my first idea was it to 
pretend the Visible flag for this HWND and Hook the BeginPaint function so that 
it returns my own DC. After that, you could send WM_PAINT Messages for the 
Update of the Flash's HWND.
Is there maybe a simpler way? 

Original comment by berkeley...@googlemail.com on 16 May 2013 at 3:33

[GoogleCodeExporter]

@comment#113: Using windowed plugins with off-screen rendering rather defeats 
the purpose. Either use windowed rendering or windowless plugins.

Original comment by magreenb...@gmail.com on 16 May 2013 at 3:38

[GoogleCodeExporter]

Well, I have got a flash website, which uses Stage3D. Forcing this flash site 
to use opaque or transparent modes only results in a black screen.
My purpose was it to create the windowed plugin, but leave it hidden and let it 
render to an own Device Context.
I'm asking if there is maybe another method to use Stage3D with hardware 
acceleration in offscreen mode? 
- or is it something you cannot change, as the flash plugin itself is 
responsible for it?

Original comment by berkeley...@googlemail.com on 16 May 2013 at 3:52

[GoogleCodeExporter]

@comment#115: Windowed plugins are responsible for their own rendering. So the 
window for windowed plugins needs to be visible and parented/positioned 
correctly.

Original comment by magreenb...@gmail.com on 16 May 2013 at 3:54

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Hi everyone,
I'm trying to get accelerated rendering work under osr mode.

in browser_host_impl.cc, CefBrowserSettings::accelerated_compositing was 
original coded to become STATE_DISABLED when osr is enabled.
I modify this part to allow CefBrowserSettings::accelerated_compositing as it 
is.

When testing with 
https://www.webkit.org/blog-files/3d-transforms/poster-circle.html

I observed a gpu-process was spawned, but the rendering path seems to be 
software rendering instead. Actually the gpu-process looks like no action at 
all, consuming no cpu resource. So I think I must have done something wrong to 
make the hardware rendering path no work.

My understand is if the hardware rendering path is on its way, takes windows 
platform for example, a DirectX 3d shared resource handle will be available for 
browser process to use. 

Please kindly let me know if it is possible to make chrome rendering the 
hardware path even if currently we cannot capture the rendering result under 
osr mode.

Thanks.

Original comment by vincent...@ubitusinc.com on 20 Jun 2013 at 9:28

[GoogleCodeExporter]

Hi everyone,
I traced the code between different process and finally found why hardware 
rendering path is not working. It is because in 
CefRenderWidgetHostViewOSR::GetCompositingSurface, a null handle was returned. 
By Comparing with its RenderWidgetHostViewWin::GetCompositingSurface 
counterpart. I think it is the place to begin with.

Thanks.

Original comment by vincent...@ubitusinc.com on 21 Jun 2013 at 2:10

[GoogleCodeExporter]

Off-screen rendering is meant to be used for those who need an array of bytes 
containing the webview. If you want to enable hardware acceleration, the web 
content will go to an accelerated surface and you should copy from that surface 
back to your buffer. This is going to be very expensive.
Can you describe the use case, why do you need OSR & hardware acceleration, why 
a non OSR, hardware accelerated browser instance wouldn't be an option.

Original comment by ro...@adobe.com on 21 Jun 2013 at 7:08

[GoogleCodeExporter]

Yes, it is best not to read the data bytes back from GPU for the performance 
concern.
In my use case, I am not really trying to render the web contents to window, 
what really concerns is the data bytes, no matter where they are. 

If they are placed on the GPU, I would make use of these data in the GPU way, 
anyway. As I said, there will be a shared resource in hardware-rendering path. 
I think it would be fine to make use of this resource. 

Original comment by vincent...@ubitusinc.com on 21 Jun 2013 at 8:02

[GoogleCodeExporter]

@comment#120: without hardware acceleration no any css 3d transforms works. So 
if we will see from rendering correctness point - we must no have concern about 
performance penalties, 'cause fast code which work incorrectly - incorrect 
code. ;)
also it is not bad idea have capabilities to rendering in specific surface 
(directx) - in this case it is can be more easy to embedd it in directx-based 
applications.

Original comment by fdd...@gmail.com on 21 Jun 2013 at 8:14

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

@comment#122
It would be really nice to have the capability to render into a specific 
surface, but this sounds to a feature besides OSR.
I wouldn't say that non-accelerated rendering works incorrectly, it's more like 
it has limited capabilities and it works as intended. If you'd like to have an 
off-screen buffer for each frame, you'd probably not be able to replay a smooth 
animations from them. If you want an off-screen buffer with a webview snapshot, 
there should be options to do this without off-screen rendering.

Original comment by ro...@adobe.com on 21 Jun 2013 at 8:46

[GoogleCodeExporter]

@comment#124: Sorry, may be my previous message may be not completely 
understandable. Incorrectness of OSR are not in technical details / 
implementation - it incorrect from default user expectation (i.e. users are 
waiting same behavior as windowed mode). There is not first GPU & OSR 
discussion, and most of them we always speak about lack of performance, but i 
just point, that also other point of view on same problem exists -> it is get 
worked available already implemented features without concern of performance. 
Of course more performant solution is better. :)

Original comment by fdd...@gmail.com on 21 Jun 2013 at 12:14

[GoogleCodeExporter]

The attached patch against trunk revision 1280 adds off-screen rendering 
support on Linux. I've only tested it on Ubuntu 12.04 64-bit so far. The 
cefclient OSR example adds a new dependency on the libgtkglext1-dev package.

It would be great if someone with linux/GTK experience could review this patch. 
I plan to commit it some time next week (trunk and 1453 branch).

Original comment by magreenb...@gmail.com on 21 Jun 2013 at 7:24

Attachments:

• osr_linux_trunk_1280.patch

[GoogleCodeExporter]

Any ideas why CEF3 with offscreen rendering doesn't render flash (e.g. youtube 
videos)? My guess is it needs the same windowless-flash patch as in CEF1

Original comment by predator...@yahoo.com on 22 Jun 2013 at 10:46

[GoogleCodeExporter]

Trunk revision 1285 and 1453 branch revision 1286 force flash and silverlight 
plugins to windowless mode when using off-screen rendering.

Original comment by magreenb...@gmail.com on 24 Jun 2013 at 4:51

[GoogleCodeExporter]

Issue 594 has been merged into this issue.

Original comment by magreenb...@gmail.com on 27 Jun 2013 at 5:37

[GoogleCodeExporter]

Linux off-screen rendering support added in trunk revision 1294 and 1453 branch 
revision 1295.

This issue is now closed:
* File new bugs or feature requests as a separate issue (search first to make 
sure it doesn't already exist).
* Use the CEF Forum for questions related to off-screen rendering.
* Support for hardware acceleration will be discussed in issue #1006.

Original comment by magreenb...@gmail.com on 27 Jun 2013 at 5:53

• Changed state: Fixed