dotnet / docfx

Static site generator for .NET API documentation.
https://dotnet.github.io/docfx/
MIT License
4.09k stars 867 forks source link

[Feature Request] Ability to skip browser download #10376

Open plachta11b opened 1 week ago

plachta11b commented 1 week ago

Is your feature request related to a problem? Please describe. Docfx alwawys download browser even when not needed. I have build environment without internet access where only pre-built docker image and documentation is allowed to enter. I can build documentation only with slow workaround currently.

Default behaviour always tries to download chrome with each run: Downloading Chromium 125.0.6422.26 (playwright build v1117) from https://playwright-verizon.azureedge.net/builds/chromium/1117/chromium-linux.zip even when there is INSTALLATION_COMPLETE file available in $PLAYWRIGHT_BROWSERS_PATH directory. It downloads browser with each pdf built when internet access is allowed. That is wasteful when building many files in my opinion.

Before the build the PLAYWRIGHT_BROWSERS_PATH=/.cache dir looks like this:

# /.cache/chromium-1117$ ls
DEPENDENCIES_VALIDATED  INSTALLATION_COMPLETE   chrome-linux/ 

Download failure would break build. But there is simple workaround. Build works when you set the directory that contains browser installation as readonly. This prevent __dirlock directory creation and Docfx proceeds to documentation build without crashing after some time.

How it looks: Error: EACCES: permission denied '<your playwright directory>' '__dirlock'

It always takes some time before Docfx give up on browser installation with this workaround and I need to speed this up. I tried ENV PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 with no effect.

Describe the solution you'd like The environment variable PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD configured to 1 that would skip any browser download attempt.

Describe alternatives you've considered Adding parameter to docfx pdf build command: docfx pdf --skip-browser-download.

Additional context I call docfx pdf Dummy/docfx.json build command on dummy project during image build to be sure that all dependencies are in place.

filzrev commented 4 days ago

Default behaviour always tries to download chrome with each run:

If INSTALLATION_COMPLETE file is exists. Browser download/installation is expected to be skipped by playwright.

What's happens when running docfx pdf command with playwright's debug log.

PowerShell Command

$env:DEBUG="pw:install"  # Enable playwright debug logging by setting environment variable
docfx pdf

Result On my environment. (OS: Windows) Browser download seems to be skipped. And it works on no internet access.

pw:install Chromium 125.0.6422.26 (playwright build v1117) is already downloaded. +0ms pw:install FFMPEG playwright build v1009 is already downloaded. +1ms

plachta11b commented 3 days ago

Default behaviour always tries to download chrome with each run:

If INSTALLATION_COMPLETE file is exists. Browser download/installation is expected to be skipped by playwright.

What's happens when running docfx pdf command with playwright's debug log.

PowerShell Command

$env:DEBUG="pw:install" # Enable playwright debug logging by setting environment variable docfx pdf Result On my environment. (OS: Windows) Browser download seems to be skipped. And it works on no internet access.

pw:install Chromium 125.0.6422.26 (playwright build v1117) is already downloaded. +0ms pw:install FFMPEG playwright build v1009 is already downloaded. +1ms

I did not get any difference with DEBUG="pw:install":

Failed to install browsers
Error: EACCES: permission denied, mkdir '/home/docfx/ms-playwright/__dirlock'
filzrev commented 1 day ago

I've tested with this Dockerfile And run test steps that described at https://github.com/dotnet/docfx/pull/10407.

After docker image created. It works as expected without internet access.

Above Dockerfile install chromium browser and dependent libs with following command.

playwright.ps1 install --with-deps chromium

And I've confirmed playwright browser is installed to ~/.cache/ms-playwright.

plachta11b commented 1 day ago

I've tested with this Dockerfile And run test steps that described at https://github.com/dotnet/docfx/pull/10407.

After docker image created. It works as expected without internet access.

Above Dockerfile install chromium browser and dependent libs with following command.

playwright.ps1 install --with-deps chromium

And I've confirmed playwright browser is installed to ~/.cache/ms-playwright.

@filzrev Thank you for the test. I will try to convert the Dockerfile for our non-root/readonly env.

plachta11b commented 23 hours ago

I am not able to run playwright.ps1 as non root user

ARG DOTNET_IMAGE mcr.microsoft.com/dotnet/sdk:8.0-noble
FROM $DOTNET_IMAGE

# Add dotnet tools to path.
ENV PATH="${PATH}:/root/.dotnet/tools"

# Set Node.js path
ENV PLAYWRIGHT_NODEJS_PATH="/usr/bin/node"

# Set target docfx version.
ARG DOCFX_VERSION=2.78.1

# Install Node.js and browser(chromium) with dependencies
RUN apt-get install -y -qq --update --no-install-recommends nodejs && \
    rm -rf /var/lib/apt/lists/* && \
    rm -rf /tmp/*

RUN useradd -m -d /home/docfx -s /bin/bash -u 1001 docfx
ENV PLAYWRIGHT_BROWSERS_PATH=/home/docfx/ms-playwright

# Install DocFX as a dotnet tool.
RUN dotnet tool install docfx -g --version ${DOCFX_VERSION}

# Install dependencies
RUN pwsh -File /root/.dotnet/tools/.store/docfx/${DOCFX_VERSION}/docfx/${DOCFX_VERSION}/tools/net8.0/any/playwright.ps1 install --with-deps chromium && \
    rm -rf /var/lib/apt/lists/* && \
    rm -rf /tmp/*

USER docfx
WORKDIR /home/docfx
ENV HOME="/home/docfx"
ENV PATH="${PATH}:${HOME}/.dotnet/tools"

RUN dotnet new tool-manifest
RUN dotnet tool install docfx --version ${DOCFX_VERSION} && \
    dotnet tool run docfx --version

RUN dotnet tool run docfx init --yes --output dummy
RUN dotnet tool run docfx build dummy/docfx.json
RUN dotnet tool run docfx pdf dummy/docfx.json

The build waits really long on the RUN dotnet tool run docfx pdf dummy/docfx.json step. So it is caused by readonly fs and network has no role in this. I was confused because it took so long to fail.