Automated chrome cef building and/or cefsharp building dockerfiles and scripts.
While the processes of building CEF and CEFSHARP are not hard they require a very exacting environment and build steps can take a long time so are annoying to repeat. The goal if this repo is a collection of scripts to automate everything to make it easy for anyone to do. We are using Docker to run everything in a container as it makes it much easier to reproduce and won't pollute your dev environment with all the pre-reqs. You can easily tweak the exact versions you want to build, and the build flags. From creating a VM on your cloud provider of choice (or your own machine) it is about 20 minutes of setup, starting a build script, and just waiting a few hours for it to spit out the compiled binaries. It has been tested with chrome 63->100 but would likely work for any modern chrome build without changes (in most cases).
Thanks to the fantastic CEFSharp team, especially @amaitland who works insanely hard on the open source project. @perlun provided some great direction on the Windows building and was also a huge help. Please support CEFSharp if you use it, even if you do a small monthly donation of $10 or $25 it can be a big help: https://github.com/sponsors/amaitland (primary CEFSharp developer, and full time parent).
If you are building in process isolation mode (recommended) make sure the base image file is the same build as your actual OS (or the VM's os). IE if you are on windows Fall 2018 release 1803 (10.0.17134) change VAR_BASE_DOCKER_FILE to the 1803 image. Run ./build.ps1 and it should build the packages. Warning: The Windows 10 Client May 2021/Nov 2021 DOES NOT support the more performant process isolation mode. See HyperV Isolation (for server or Windows 10 client) Mode below for details.
For a super fast start look at the azure auto provision option below. As long as you have an azure account created it can create the entire setup and build in a few commands.
If using Azure create a F32_v2 VM with the image "Datacenter Core 1903 with Containers", if using another machine just install docker for windows (make sure you have 20GB (40GB for chrome < 65) of ram between actual ram + page file). Set the Docker For Windows Config File changing the path to the folder to store data on (suggested local temp drive) and restart docker service. Copy the items from this repo into a folder. Copy the versions_src.ps1 to versions.ps1 and change the variables to what you want: for example $VAR_GN_DEFINES="is_official_build=true proprietary_codecs=true ffmpeg_branding=Chrome";$VAR_DUAL_BUILD="1";
. Only use DUAL_BUILD if you have 30 gigs of ram or more, otherwise leave it at 0 and the build will take an extra 20-40 minutes.
Beware if using the exact same version string as an official CEF Build as it will mean you need to make sure your nuget source is always used before the master source. If you use a slightly different minor build you will not have that problem. For CefSharp you can use a manual higher fake minor version number(ie .99) to not get confused with the official builds (but the CEF build note above still applies).
In part we use the latest version of several installers/build tools if they changed so might the success of these dockerfiles. It does not build the debug versions of CEF or CEFSharp. This could be added as an option pretty easily (but would probably at-least double build times). For some reason I had issues getting the automated build script for CEF to work doing the calls by hand is pretty basic however.
Window 10 Client (Pro) by default with docker uses HyperV isolation, this mode is very non performant vs process isolation mode. Make sure to change it (and see the note below, but beware does not work on 21H1/21H2 builds).
The following requirements are for chrome and the current vs_2019 installer, they may change over time. Compiling is largely CPU bound but linking is largely IO bound.
Due to a bug in 1903 (https://github.com/microsoft/hcsshim/issues/708) a new method had to be found rather than building all the cef source in a normal docker build step. The solution was to move the source building to store data in a volume. This has several benefits now including: builds can be resumed (if you force the same volume to be used), source code can be left behind on the volume as docker will not manually copy this with the storage driver (previously very non-performant). Downsides include the fact it will leave volumes behind that need to be removed from unsuccessful builds, there is a bit of a hacked step to support volumes. To support volumes we have to use docker run
for the cef build step (as docker build does not support volumes during build). This means we needed a way to detect IF we need to run the build step or if it was already completed successfully. To do this we rely on some tag naming tests (see build script).
There is not much in terms of a software requirements other than docker. You can run it on Windows Server or Windows 10 Client. For Windows 10 Client Install it from https://store.docker.com/editions/community/docker-ce-desktop-windows. For server Docker EE from: https://docs.docker.com/install/windows/docker-ee/#docker-universal-control-plane-and-windows (or standard docker for windows for desktops) if docker is not auto installed. If installing on Windows 10 Client make sure to see the Hyper V Notes below.
You will want a docker configuration with options similar to this. On windows client you can use the docker settings (Right-click on the Docker whale icon in the on the task bar, then click "Settings..." then click to advanced mode). For server the file is edited directly (or created if it didn't exist) at C:\ProgramData\docker\config\daemon.json
{
"registry-mirrors": [],
"insecure-registries": [],
"debug": true,
"experimental": false,
"exec-opts": [
"isolation=process"
],
"data-root": "d:/docker_data",
"storage-opts": [
"size=400G"
]
}
If you are new to Azure it is pretty easy to get started and they will give you $200 for your first month free so there will be no costs. Below we even have an auto deploy script if you prefer not to do it by hand. An Azure F32 v2 is pretty good, its only 256 gigs of space but that should be ok. ~$2.72 an hour in WestUS2 running the latest windows image. You can use the prebuilt image "Windows Server 2016 Datacenter - with Containers" or a newer one if it exists. You can either use one with a full shell (pre 1709) or one of the newer builds like "Windows Server 2016 Datacenter - with Containers 1803". Without a full shell you won't have explorer and remote desktop will just open a command prompt. You can launch notepad and manage it all through there (or use remote PS) but a full shell is easier for some people. Use the local SSD as the docker storage folder (note this will likely get wiped if you de-allocate the machine so do the entire build at once). You could potentially hook up a huge number of disks in raid 0 configuration to get somewhat decent speed that way. Create a new resource, search for the prebuilt image noted above. You do not need managed disks, assign a random user/password, new network/storage/etc is all fine. For the size make sure you select one of the F series (F32 recommended). It won't show by default, leave HD type set to SSD put Min CPU's at 32 and Ram at 64 then hit "View all".
I suggest auto-shutdown to make sure you don't leave it running.
If you have an azure account already created you can use the az_create.ps1 script to automatically setup the VM for you. It will create a Standard_F32s_v2 VM by default but this is 32 cpu's. New Azure accounts need to file a support request to increase CPU quota beyond 10. These are often approved within minutes but if you don't want to do so you can edit az_create.ps1 and change the size to something like a Standard_F8s_v2. It will take longer to build but your cost will be roughly the same total cost.
It will create everything under a new "CEFTest" resource group to make cleanup at the end easy. You can adjust the settings at the top if desired but really the only important options you pass as options to it. It will setup the VM and enable remote powershell to make the process very easy. Just launch powershell (or type powershell into the run box in windows). If the first time using powershell with azure you will need to install the tools for azure: Install-Module -Name AzureRM -Scope CurrentUser
. Next change to the folder with all the CefSharpDockerfiles (cd c:\downloads\CefSharpDockerfiles for example).
Login with your azure credentials first:
Connect-AzureRmAccount
Then if you have multiple subscriptions set the one you want with:
Set-AzureRmContext -SubscriptionName "My Subscription"
Next run this and enter a new username and password to configure the new VM with:
$cred = Get-Credential -Message "Enter user and password for remote machine admin"
Next we will run the deploy script, by default it can configure the machine to automatically shutdown at 11:30pm PDT if you provide an email it will do this and notify you. If you do not you need to manually turn the machine off when done. You can adjust time and such in the az_create.ps1 at the top (along with some other items but likely you do not need to adjust them):
./az_create.ps1 -admin_creds $cred -shutdown_email "john@gmail.com"
It should print out when done Public IP: 123.123.123.123
Next set this in a variable $IP_ADDY like:
$IP_ADDY = "123.123.123.123"
Note we are disabling the security checks in the remote powershell session. This could make you vulnerable to MITM attacks if on an unsafe network.
Next create the remote powershell session and copy the files over, and disable anti virus real time scanning (slows down compiling significantly):
$so = New-PsSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck;
$remote = New-PSSession -ComputerName $IP_ADDY -UseSSL -SessionOption $so -Credential $cred;
Get-ChildItem -Path "./" | Copy-Item -ToSession $remote -Destination "C:/CefSharpDockerfiles/"
Copy-Item -ToSession $remote daemon.json -Destination "c:/ProgramData/docker/config/daemon.json";
Invoke-Command -Session $remote -ScriptBlock { Restart-Service Docker;Set-MpPreference -DisableRealtimeMonitoring $true; }
Next we will "enter" the remote machine via powershell:
Enter-PSSession $remote
Then you should see your terminal is like [123.123.123.123]: PS c:\>
showing you are on the remote machine.
Finally we start the build:
cd C:/CefSharpDockerfiles
./build.ps1
When done exit out the remote session by typing: exit
Finally we copy the resulting files locally:
Copy-Item -FromSession $remote "c:/CefSharpDockerfiles/packages_cefsharp.zip" -Destination ".";
You could also expose the docker server to the internet and use remote docker commands rather than running the powershell remotely. When done you can delete the entire CEFTest resource group from the azure portal as well to clean everything up.
With the Azure F32 v2 host above the total estimated build time is about 2-3 hours depending on CEF version (~$6 on azure). Machines are nice 600MB/sec read/write to the local disk. There are larger VM's as well up to 72+ CPU's however some of the build stages are very linear. Note it can vary somewhat dramatically for the not cef build steps based on the luck of the draw (but the cef build is most of the build time). It seems local IO depending on what physical host it is spun up on can cause 30-50% performance fluxes. Most of the build steps make efficient use of the machine however: The git cloning is not very efficient. It is 30 minutes of the cef build time below. It doesn't quite max out network or IO. The linking stage is also not super efficient see the DUAL_BUILD flag below to help with that. Linking will take 20+ minutes per platform (40 total unless run concurrently). Here are the individual build/commit times:
HyperV isolation mode should be avoided if possible. It is slower, and more prone to fail. For Windows 10 clients below 1809 (October 2018 edition) there is not a legal alternative. You must also use Docker Desktop build newer than October of 2018. NOTE: If you are not using process isolation mode you WILL need to set $VAR_HYPERV_MEMORY_ADD
and make sure your page file is properly sized (recommend a page file at least a few gigs bigger as it needs that amount of FREE page file space). It will set the memory on every docker build step to up the default memory limit. Technically this is primarily needed in the CEF build step. NOTE if you stop docker during a build with HyperV it does not properly kill off the hyperV container restart docker to fix this. Warning: The Windows 10 May 2021/Nov 2021 (21H1/21H2 10.0.19043/10.0.19044) DOES NOT support process isolation mode. This is due to an MS failure per micorosft/windows-containers#163. If you want isolation support you must use builds 20H2 (10.0.19042) or lower OR Windows 11. Note Hyper-V isolation can be over 3 times as slow as process isolation mode for Windows 10 clients.
Once docker is setup and running copy this repo to a local folder. Copy versions_src.ps1 to versions.ps1 and change the version strings to match what you want. NOTE BASE_DOCKER_FILE must match the same kernel as the host machine IF you are using process isolation mode. This means you cannot use the 1709 image on an older host and you can use and older image on a 1709 host. Either base file is fine however to use just match it to the host.
Next run build.ps1 and if you are lucky you will end up with a cefsharp_packages.zip file with all the nupkg files you need:) Beware that as docker might be flaky(especially in hyperV mode) you may need to call build.ps1 a few times. It should largely just resume. Once it is done building you will have the cefsharp_packages.zip file. If you want any of the CEF binaries, or symbol files, you can copy them from the CEF image like: docker cp cef:c:/code/binaries/*.zip .
To be safer you can run the biggest build command by hand. The hardest (longest) build step if the CEF build at the start. You can comment out the last step in the dockerfile and manually do that step and commit it. Infact you can just docker run the build image from before than manually call cef_build.ps1 one or more times (it should do a decent job at auto-resuming) until success. If you are using a proper host with enough ram it should be able to automatically build 9 times out of 10 (if not higher) with its current redundant tries. Of course if you prefer to manually run the commands from it you can do that too. To do so comment out the final build step in Dockerfile_cef then run the following:
#So if the autmate-git.py doesn't work (if something errors out it doesn't always stop at the right point) try running the build steps manually that are there.
# From the c:/code/chromium_git/chromium/src folder run the build hooks to download tools: gclient runhooks
# From the c:/code/chromium_git/chromium/src/cef folder run the following to make the projects: ./cef_create_projects.bat
# From the c:/code/chromium_git/chromium/src
# ninja -C out/Release_GN_x64 cefclient
# ninja -C out/Release_GN_x86 cefclient
# cd C:/code/chromium_git/chromium/src/cef/tools/
# C:/code/chromium_git/chromium/src/cef/tools/make_distrib.bat --ninja-build --allow-partial;
# c:/code/chromium_git/chromium/src/cef/tools/make_distrib.bat --ninja-build --allow-partial --x64-build;
# Allow partial needed if not building debug builds, make sure to run it when done or run the cef_build last few commands to create the archive with the result and to clean up the workspace of the source files.
Note the DUAL_BUILD may speed up builds by running x86 and x64 builds concurrently (each with 1/2 as many threads). This is primarily useful during linking. Linking is largely single threaded and takes awhile and is single thread CPU bound (given enough IO). The main issue is memory usage. If both linking steps run at once you may need nearly 30GB of memory at once (in worst case older builds would use up to 50GB). It would be better if they linked at slightly separate times but as every compute system was different there did not seem to be a good way to determine how long to sleep for to make it most efficient.
$VAR_CEF_USE_BINARY_PATH=".\..\CEFBinaries\";
and set extension to the downloaded extension: $VAR_CEF_BINARY_EXT="tar.bz2";
The following were helpful: