obs-browser: Groundwork to enable Chrome 4430/arm64/Apple silicon

Developer-Ecosystem-Engineering commented 3 years ago

Update for custom obs texture sharing support used in CEF Since these CEF patches are not available on any particularly OBS Project repository, instructions are provided here

PREFIX=/Users/youruser/Documents/cef-4430
mkdir -p $PREFIX
cd $PREFIX
mkdir cef
cd cef
mkdir automate
mkdir chromium_git
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH=$PREFIX/cef/depot_tools/:$PATH
cd automate
curl -O https://bitbucket.org/chromiumembedded/cef/raw/master/tools/automate/automate-git.py
cd ../chromium_git
python ../automate/automate-git.py --download-dir=$PREFIX/cef/chromium_git --depot-tools-dir=$PREFIX/cef/depot_tools --no-distrib --no-build --arm64-build --branch=4430
cd cef 
patch -p1  < viz_osr_d3d_4430.patch
cd ../chromium/src/cef/ 
./cef_create_projects.sh
cd ..
mv out/Release_GN_x64 out/Release_GN_arm64
vim out/Release_GN_arm64/args.gn
**edit** target_cpu="arm64"
ninja -C out/Release_GN_arm64 cef
cd cef/tools 
./make_distrib.sh --ninja-build --minimal --arm64-build
cd ../binary_distrib

This should produce "cef_binary_90.6.7+g19ba721+chromium-90.0.4430.212_macosarm64_minimal.zip" which will be used as a part of the OBS Studio build and will also need these OBS Browser changes.

Description

When using CEF 4430 or later, we should use the texture sharing/gpu compositing feature that OBS Browser relies on, otherwise behaves like previously

Motivation and Context

OBS Studio relies on a custom patch to CEF in order to perform certain features in OBS Browser. Until these patches are integrated into an Apple silicon variant of CEF, OBS Studio isn't at feature parity natively on Apple silicon.

How Has This Been Tested?

Tested on M1 iMac and M1 MBP macOS 11.5.1

Types of changes

Bug fix (non-breaking change which fixes an issue)

Checklist:

[x] My code has been run through clang-format.
[x] I have read the contributing document.
[x] My code is not on the master branch.
[x] The code has been tested.
[x] All commit messages are properly formatted and commits squashed where appropriate.
[x] I have included updates to all appropriate documentation.

WizardCM commented 3 years ago

Please split the changes for newer CEF support to a separate commit from the hardware acceleration related changes.

Additionally, please be sure to run clang-format.

Developer-Ecosystem-Engineering commented 3 years ago

Please split the changes for newer CEF support to a separate commit from the hardware acceleration related changes.

I'm not sure we follow what you want split, what lines are you wanting as their own PR?

We may need to modify this PR further to scope it to only run against the arm64 CEF variant and exclude x86. Will capture your feedback + clang in that subsequent change thank you.

NextNextStep commented 3 years ago

hi @Developer-Ecosystem-Engineering I'm attempting to build your PR, but I'm unable to find the patch you shared in your instructions: viz_osr_d3d_4430.patch. Has it been uploaded as part of a PR across one of the various OBS repositories?

Developer-Ecosystem-Engineering commented 3 years ago

hi @Developer-Ecosystem-Engineering I'm attempting to build your PR, but I'm unable to find the patch you shared in your instructions: viz_osr_d3d_4430.patch. Has it been uploaded as part of a PR across one of the various OBS repositories?

Hi @gnuherdnix, that patch can be found here https://bitbucket.org/chromiumembedded/cef/pull-requests/285, thats not our work but appears to be some custom work to support texture sharing in OBS Studio to improve performance and reduce CPU overhead on x86. It doesn't appear to be technically necessary on Apple silicon, but we are still reviewing some tests and data based on feedback from the team. We will need to rework this PR based on some feedback, that should happen in a bit.

pabloko commented 3 years ago

Hi @Developer-Ecosystem-Engineering, this patch is not a performance enhancement, what it really does, is to bring back the OnAcceleratedPaint from CefRenderHandler, wich allows to access a gpu texture instead having to map raw RGBA pixels in memory, but most importantly, allows that each frame can be requested, this is very important for video in order to prevent tearing. In windows it uses DirectX11 shared handles and IOSurface on apple. Do somebody know if this patch is already on the main windows obs-browser dist?

RytoEX commented 3 years ago

Hi @Developer-Ecosystem-Engineering, this patch is not a performance enhancement, what it really does, is to bring back the OnAcceleratedPaint from CefRenderHandler, wich allows to access a gpu texture instead having to map raw RGBA pixels in memory, but most importantly, allows that each frame can be requested, this is very important for video in order to prevent tearing. In windows it uses DirectX11 shared handles and IOSurface on apple. Do somebody know if this patch is already on the main windows obs-browser dist?

@pabloko On Windows, we use a prebuilt patched version of CEF 3770 (x64, x86). On macOS, we use a prebuilt patched version of CEF 4183.

For all intents and purposes, to the end user, enabling hardware accelerated browser sources is a performance enhancement because it allows them to have fluid 60 FPS complex browser-based sources and overlays. We're not talking about increasing the efficiency of a loop or function when we use the phrase in this context.

NextNextStep commented 3 years ago

@RytoEX out of curiosity, could you point me to where the build infrastructure and patches are stored for the CEF prebuilts used for Mac & Windows? I haven't been able to find them and wanted to try building locally.

@Developer-Ecosystem-Engineering Thanks for the link. Regarding IOSurfaces, in the example at hand where we render onto the IOSurface backed GL texture from the CEF process, and then read from that same texture for rendering to the scene in OBS, are we managing to work with the same pointer to on-device GPU memory and avoid CPU<->GPU memory copies?

My understanding is that IOSurface allows transparent access to the surface data and will copy whether needed from GPU or CPU memory, but I'm wondering if we effectively avoid CPU copies when we are only reading & writing to the GPU memory buffer across the two processes.

RytoEX commented 3 years ago

@RytoEX out of curiosity, could you point me to where the build infrastructure and patches are stored for the CEF prebuilts used for Mac & Windows? I haven't been able to find them and wanted to try building locally.

We don't have a build infrastructure for our CEF prebuilts. They were generously built by contributors so that we could host them so that people don't have to rebuild them when building obs-studio or obs-browser.

I don't believe we have any custom CEF patches publicly posted. As @Developer-Ecosystem-Engineering has said, the texture sharing patch we use is that unmerged CEF PR. It's possible that we modified it further to get it to actually work, but I didn't work on it, so I can't speak to the details.

Developer-Ecosystem-Engineering commented 3 years ago

clang-format re-run on all the changes
CEF version and HW Accel changes are split into two distinct commits
Removed debug log

Developer-Ecosystem-Engineering commented 3 years ago

It looks like the first two commits cancel each other out. Please squash your commits so that only the relevant commits remain.

Apologies, should be resolved now, didn't mean to close the PR!

Developer-Ecosystem-Engineering commented 3 years ago

Some users use many browser sources at once, sometimes running heavy workloads (bit cups used to be the common example). Hardware acceleration and texture sharing reduce CPU load overall in addition to achieving 60 FPS, not just aiming to allow one browser source to reach 60 FPS.

Hardware acceleration is still enabled. If you disable it in CEF, you'll definitely notice even on M1.

RytoEX commented 2 years ago

Does this still need to be open?

Developer-Ecosystem-Engineering commented 2 years ago

I believe @PatTheMav integrated the spirit of these changes into his PR already and CEF is building 'elsewhere' to be included as a binary payload for arm64 variants of OBS. Would be good for him to confirm he's happy with it and it needs no further iteration.

PatTheMav commented 2 years ago

Yeah we've mainly used the CEF build instructions. We still need to revisit the coupling of enabling GPU compositing in CEF and texture sharing (because even with the latter disabled, CEF seems to still do the former on my M1 Mac).

But I'm happy to solve/revisit this in the scope of another PR.

Developer-Ecosystem-Engineering commented 2 years ago

Closing then since this was integrated via #290

Lets see what the feedback is on the M1 OBS build w.r.t. CEF and what needs further work, if anything.

obsproject / obs-browser