search

rocker-org / devcontainer-features

A collection of Dev Container Features.
https://rocker-project.org/images/devcontainer/features.html
MIT License
52 stars 16 forks source link

`quarto-cli` feature install failure when combine with `Latex` feature installation #214

Closed dylanrussellmd closed 7 months ago

dylanrussellmd commented 7 months ago

I built a simple Codespaces container with the following devcontainer.json:

{
    "name": "Pandoc Development Container",
    // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
    "image": "mcr.microsoft.com/devcontainers/base:bullseye",

    // Features to add to the dev container. More info: https://containers.dev/features.
    "features": {
        "ghcr.io/rocker-org/devcontainer-features/quarto-cli:1": {
            "version": "latest",
            "installChromium": true
        },
        "ghcr.io/rocker-org/devcontainer-features/apt-packages:1": {
            "packages": "chromium"
        }, 
        "ghcr.io/devcontainers-contrib/features/pandoc:1": {
            "version": "latest"
        }, 
        "ghcr.io/prulloac/devcontainer-features/latex:1": {
            "scheme": "full",
            "mirror": "https://ctan.math.utah.edu/ctan/tex-archive/systems/texlive/tlnet/"
        }
    }
}

This failed with the following error:

#18 0.465 ===========================================================================
#18 0.465 Feature       : Quarto CLI
#18 0.465 Description   : Installs the Quarto CLI. Auto-detects latest version.
#18 0.465 Id            : ghcr.io/rocker-org/devcontainer-features/quarto-cli
#18 0.465 Version       : 1.1.2
#18 0.465 Documentation : ********/rocker-org/devcontainer-features/tree/main/src/quarto-cli
#18 0.465 Options       :
#18 0.465     VERSION="latest"
#18 0.465     INSTALLTINYTEX="false"
#18 0.465     INSTALLCHROMIUM="********"
#18 0.465 ===========================================================================#18 0.484 Downloading Quarto CLI...
#18 0.532 /tmp/quarto /tmp/dev-container-features/quarto-cli_3
#18 1.585 Selecting previously unselected package quarto.#18 1.608 (Reading database ... 32464 files and directories currently installed.)
#18 1.609 Preparing to unpack quarto.deb ...
#18 1.633 Unpacking quarto (1.4.553) ...
#18 7.432 Setting up quarto (1.4.553) ...
#18 7.568 /tmp/dev-container-features/quarto-cli_3
#18 7.580 Installing chromium...
#18 7.580 (!) Quarto CLI installs headless Chromium via Puppeteer. The bundled Chromium that Puppeteer installs may not work on Docker containers.
#18 7.580     Please check the Puppeteer document: ********/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#running-puppeteer-in-docker
#18 7.594 bash: line 1: quarto: command not found
#18 7.596 ERROR: Feature "Quarto CLI" (ghcr.io/rocker-org/devcontainer-features/quarto-cli) failed to install! Look at the documentation at ********/rocker-org/devcontainer-features/tree/main/src/quarto-cli for help troubleshooting this error.
#18 ERROR: process "/bin/sh -c cp -ar /tmp/build-features-src/quarto-cli_3 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/quarto-cli_3  && cd /tmp/dev-container-features/quarto-cli_3  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/quarto-cli_3" did not complete successfully: exit code: 127
------
 > [dev_containers_target_stage 8/8] RUN --mount=type=bind,from=dev_containers_feature_content_source,source=quarto-cli_3,target=/tmp/build-features-src/quarto-cli_3     cp -ar /tmp/build-features-src/quarto-cli_3 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/quarto-cli_3  && cd /tmp/dev-container-features/quarto-cli_3  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/quarto-cli_3:
1.608 (Reading database ... 32464 files and directories currently installed.)
1.609 Preparing to unpack quarto.deb ...
1.633 Unpacking quarto (1.4.553) ...
7.432 Setting up quarto (1.4.553) ...
7.568 /tmp/dev-container-features/quarto-cli_3
7.580 Installing chromium...
7.580 (!) Quarto CLI installs headless Chromium via Puppeteer. The bundled Chromium that Puppeteer installs may not work on Docker containers.
7.580     Please check the Puppeteer document: ********/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#running-puppeteer-in-docker
7.594 bash: line 1: quarto: command not found
7.596 ERROR: Feature "Quarto CLI" (ghcr.io/rocker-org/devcontainer-features/quarto-cli) failed to install! Look at the documentation at ********/rocker-org/devcontainer-features/tree/main/src/quarto-cli for help troubleshooting this error.
------
Dockerfile.extended:51
--------------------
  50 |     
  51 | >>> RUN --mount=type=bind,from=dev_containers_feature_content_source,source=quarto-cli_3,target=/tmp/build-features-src/quarto-cli_3 \
  52 | >>>     cp -ar /tmp/build-features-src/quarto-cli_3 /tmp/dev-container-features \
  53 | >>>  && chmod -R 0755 /tmp/dev-container-features/quarto-cli_3 \
  54 | >>>  && cd /tmp/dev-container-features/quarto-cli_3 \
  55 | >>>  && chmod +x ./devcontainer-features-install.sh \
  56 | >>>  && ./devcontainer-features-install.sh \
  57 | >>>  && rm -rf /tmp/dev-container-features/quarto-cli_3
  58 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c cp -ar /tmp/build-features-src/quarto-cli_3 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/quarto-cli_3  && cd /tmp/dev-container-features/quarto-cli_3  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/quarto-cli_3" did not complete successfully: exit code: 127
Stop: Run: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-root/container-features/0.56.1-1713598698526 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/base:bullseye --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-pandoc-43a4e104c62be3219495884aaff9b9fb4c77f05fc457d7e11c7aeb730b3de265-features -f /tmp/devcontainercli-root/container-features/0.56.1-1713598698526/Dockerfile.extended /var/lib/docker/codespacemount/.persistedshare/empty-folder
Error: Command failed: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-root/container-features/0.56.1-1713598698526 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/base:bullseye --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-pandoc-43a4e104c62be3219495884aaff9b9fb4c77f05fc457d7e11c7aeb730b3de265-features -f /tmp/devcontainercli-root/container-features/0.56.1-1713598698526/Dockerfile.extended /var/lib/docker/codespacemount/.persistedshare/empty-folder
    at O$ (/.codespaces/agent/bin/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:464:1253)
{"outcome":"error","message":"Command failed: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-root/container-features/0.56.1-1713598698526 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/base:bullseye --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-pandoc-43a4e104c62be3219495884aaff9b9fb4c77f05fc457d7e11c7aeb730b3de265-features -f /tmp/devcontainercli-root/container-features/0.56.1-1713598698526/Dockerfile.extended /var/lib/docker/codespacemount/.persistedshare/empty-folder","description":"An error occurred setting up the container."}
    at iK (/.codespaces/agent/bin/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:464:997)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async gAA (/.codespaces/agent/bin/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:481:3660)
    at async BC (/.codespaces/agent/bin/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:481:4775)
    at async xeA (/.codespaces/agent/bin/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:614:11265)
    at async UeA (/.codespaces/agent/bin/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:614:11006)
devcontainer process exited with exit code 1
Failed to create container.
Error: Command failed: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-root/container-features/0.56.1-1713598698526 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/base:bullseye --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-pandoc-43a4e104c62be3219495884aaff9b9fb4c77f05fc457d7e11c7aeb730b3de265-features -f /tmp/devcontainercli-root/container-features/0.56.1-1713598698526/Dockerfile.extended /var/lib/docker/codespacemount/.persistedshare/empty-folder
Error code: 1302 (UnifiedContainersErrorFatalCreatingContainer)
Container creation failed.
Jobs failed, exiting the agent. Error code: 1302 (UnifiedContainersErrorFatalCreatingContainer)

When built with this devcontainer.json...

{
    "name": "Pandoc Development Container",
    // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
    "image": "mcr.microsoft.com/devcontainers/base:bullseye",

    // Features to add to the dev container. More info: https://containers.dev/features.
    "features": {
        "ghcr.io/rocker-org/devcontainer-features/quarto-cli:1": {
            "version": "latest",
            "installChromium": true
        },
        "ghcr.io/rocker-org/devcontainer-features/apt-packages:1": {
            "packages": "chromium"
        }, 
        "ghcr.io/devcontainers-contrib/features/pandoc:1": {
            "version": "latest"
        }//, 
        //"ghcr.io/prulloac/devcontainer-features/latex:1": {
        //  "scheme": "full",
        //  "mirror": "https://ctan.math.utah.edu/ctan/tex-archive/systems/texlive/tlnet/"
        //}
    }
}

the build failed. The only difference is the presence of the latex feature. Any idea why these two are coming in to conflict in such a way that this error is produced: #18 7.594 bash: line 1: quarto: command not found

eitsupi commented 7 months ago

You mean the container creation is succeed without ghcr.io/prulloac/devcontainer-features/latex:1 but failed with that?

If you do not set "installChromium": true, the quarto command should not be executed during container creation, so the container creation will succeed. In that case, what happens when I run the quarto command inside the created container? Or what happens if you manually install the Quarto CLI deb package in a container with only ghcr.io/prulloac/devcontainer-features/latex:1 installed?

My guess is that this is because some directory used by Quarto CLI has been rewritten by ghcr.io/prulloac/devcontainer-features/latex:1 and Quarto CLI is no longer working. In other words, there is nothing that can be done here and it is a bug in the Quarto CLI.

eitsupi commented 7 months ago

This may be due to PATH being overwritten. In that case this is a bug of ghcr.io/prulloac/devcontainer-features/latex:1. https://github.com/prulloac/devcontainer-features/blob/6d7fb8926bf924ca515bba370c139c98f6ce4f55/src/latex/install.sh#L39

eitsupi commented 7 months ago

I tried ghcr.io/prulloac/devcontainer-features/latex:1 but in my area I could not build the container because the latex installation fails or takes too long to install from mirror.

eitsupi commented 7 months ago

I did a few more tests and I am still convinced this is ghcr.io/prulloac/devcontainer-features/latex:1's problem because after ghcr.io/prulloac/devcontainer-features/latex:1, ghcr.io/meaningful-ooo/devcontainer-features/fish:1 also failed to find fish.

Details ```log > [dev_containers_target_stage 7/7] RUN --mount=type=bind,from=dev_containers_feature_content_source,source=fish_2,target=/tmp/build-features-src/fish_2 cp -ar /tmp/build-features-src/fish_2 /tmp/dev-container-features && chmod -R 0755 /tmp/dev-container-features/fish_2 && cd /tmp/dev-container-features/fish_2 && chmod +x ./devcontainer-features-install.sh && ./devcontainer-features-install.sh && rm -rf /tmp/dev-container-features/fish_2: 0.386 =========================================================================== 0.386 Feature : fish 0.386 Description : Installs fish shell and Fisher plugin manager (optionally) 0.386 Id : ghcr.io/meaningful-ooo/devcontainer-features/fish 0.386 Version : 1.2.2 [2024-04-21T04:36:25.991Z] 0.386 Documentation : https://github.com/meaningful-ooo/devcontainer-features/tree/main/src/fish 0.386 Options : 0.386 FISHER="true" 0.386 =========================================================================== 0.415 Installing fish shell... 0.419 deb http://download.opensuse.org/repositories/shells:/fish:/release:/3/Debian_12/ / 2.111 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB] 2.138 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB] 2.142 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB] 2.260 Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8786 kB] 2.633 Get:5 http://download.opensuse.org/repositories/shells:/fish:/release:/3/Debian_12 InRelease [1556 B] 2.803 Get:6 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [12.7 kB] 2.804 Get:7 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [155 kB] 3.118 Get:8 http://download.opensuse.org/repositories/shells:/fish:/release:/3/Debian_12 Packages [1841 B] 4.495 Fetched 9211 kB in 2s (3760 kB/s) 4.495 Reading package lists... 5.669 Reading package lists... 6.729 Building dependency tree... 7.001 Reading state information... 7.352 The following additional packages will be installed: 7.353 libpython3-stdlib libpython3.11-minimal libpython3.11-stdlib media-types 7.353 python3 python3-minimal python3.11 python3.11-minimal 7.357 Suggested packages: 7.357 xdg-utils python3-doc python3-tk python3-venv python3.11-venv python3.11-doc 7.357 binfmt-support 7.357 Recommended packages: 7.357 xsel 7.493 The following NEW packages will be installed: 7.495 fish libpython3-stdlib libpython3.11-minimal libpython3.11-stdlib 7.496 media-types python3 python3-minimal python3.11 python3.11-minimal [2024-04-21T04:36:25.992Z] 7.611 0 upgraded, 9 newly installed, 0 to remove and 1 not upgraded. 7.611 Need to get 7896 kB of archives. 7.611 After this operation, 40.1 MB of additional disk space will be used. 7.611 Get:1 http://deb.debian.org/debian bookworm/main amd64 libpython3.11-minimal amd64 3.11.2-6 [813 kB] 7.671 Get:2 http://deb.debian.org/debian bookworm/main amd64 python3.11-minimal amd64 3.11.2-6 [2064 kB] 7.788 Get:3 http://deb.debian.org/debian bookworm/main amd64 python3-minimal amd64 3.11.2-1+b1 [26.3 kB] 7.789 Get:4 http://deb.debian.org/debian bookworm/main amd64 media-types all 10.0.0 [26.1 kB] 7.792 Get:5 http://deb.debian.org/debian bookworm/main amd64 libpython3.11-stdlib amd64 3.11.2-6 [1796 kB] 7.897 Get:6 http://deb.debian.org/debian bookworm/main amd64 python3.11 amd64 3.11.2-6 [572 kB] 7.937 Get:7 http://deb.debian.org/debian bookworm/main amd64 libpython3-stdlib amd64 3.11.2-1+b1 [9312 B] 7.938 Get:8 http://deb.debian.org/debian bookworm/main amd64 python3 amd64 3.11.2-1+b1 [26.3 kB] 9.251 Get:9 http://ftp.lysator.liu.se/pub/opensuse/repositories/shells%3A/fish%3A/release%3A/3/Debian_12 fish 3.7.1-1 [2562 kB] 12.59 Fetched 7896 kB in 4s (2094 kB/s) 12.65 Selecting previously unselected package libpython3.11-minimal:amd64. 12.65 (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 25412 files and directories currently installed.) 12.67 Preparing to unpack .../libpython3.11-minimal_3.11.2-6_amd64.deb ... 12.68 Unpacking libpython3.11-minimal:amd64 (3.11.2-6) ... 12.90 Selecting previously unselected package python3.11-minimal. 12.91 Preparing to unpack .../python3.11-minimal_3.11.2-6_amd64.deb ... [2024-04-21T04:36:25.992Z] 12.93 Unpacking python3.11-minimal (3.11.2-6) ... 13.32 Setting up libpython3.11-minimal:amd64 (3.11.2-6) ... 13.36 Setting up python3.11-minimal (3.11.2-6) ... 14.66 Selecting previously unselected package python3-minimal. 14.66 (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 25719 files and directories currently installed.) 14.69 Preparing to unpack .../python3-minimal_3.11.2-1+b1_amd64.deb ... 14.70 Unpacking python3-minimal (3.11.2-1+b1) ... 14.78 Selecting previously unselected package media-types. 14.78 Preparing to unpack .../media-types_10.0.0_all.deb ... 14.79 Unpacking media-types (10.0.0) ... 14.87 Selecting previously unselected package libpython3.11-stdlib:amd64. 14.87 Preparing to unpack .../libpython3.11-stdlib_3.11.2-6_amd64.deb ... 14.88 Unpacking libpython3.11-stdlib:amd64 (3.11.2-6) ... 15.21 Selecting previously unselected package python3.11. 15.22 Preparing to unpack .../python3.11_3.11.2-6_amd64.deb ... 15.23 Unpacking python3.11 (3.11.2-6) ... 15.31 Selecting previously unselected package libpython3-stdlib:amd64. 15.31 Preparing to unpack .../libpython3-stdlib_3.11.2-1+b1_amd64.deb ... [2024-04-21T04:36:25.992Z] 15.32 Unpacking libpython3-stdlib:amd64 (3.11.2-1+b1) ... 15.39 Setting up python3-minimal (3.11.2-1+b1) ... 15.67 Selecting previously unselected package python3. 15.67 (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 26128 files and directories currently installed.) 15.70 Preparing to unpack .../python3_3.11.2-1+b1_amd64.deb ... 15.72 Unpacking python3 (3.11.2-1+b1) ... 15.82 Selecting previously unselected package fish. 15.82 Preparing to unpack .../fish_3.7.1-1_amd64.deb ... 15.84 Unpacking fish (3.7.1-1) ... 16.58 Setting up media-types (10.0.0) ... 16.62 Setting up libpython3.11-stdlib:amd64 (3.11.2-6) ... 16.65 Setting up libpython3-stdlib:amd64 (3.11.2-1+b1) ... 16.67 Setting up python3.11 (3.11.2-6) ... 18.01 Setting up python3 (3.11.2-1+b1) ... 18.19 Setting up fish (3.7.1-1) ... 18.24 Processing triggers for man-db (2.11.2-2) ... 19.10 fish, version 3.7.1 19.10 Installing Fisher... 19.46 fisher install version 4.4.4(B 19.48 Fetching https://api.github.com/repos/jorgebucaran/fisher/tarball/HEAD(B 20.16 Installing jorgebucaran/fisher(B 20.16 /root/.config/fish/functions/fisher.fish 20.16 /root/.config/fish/completions/fisher.fish 20.17 Installed 1 plugin/s 20.18 bash: line 1: fish: command not found [2024-04-21T04:36:25.992Z] 20.18 ERROR: Feature "fish" (ghcr.io/meaningful-ooo/devcontainer-features/fish) failed to install! Look at the documentation at https://github.com/meaningful-ooo/devcontainer-features/tree/main/src/fish for help troubleshooting this error. ------ [2024-04-21T04:36:25.995Z] Dockerfile.extended:42 -------------------- 41 | 42 | >>> RUN --mount=type=bind,from=dev_containers_feature_content_source,source=fish_2,target=/tmp/build-features-src/fish_2 \ 43 | >>> cp -ar /tmp/build-features-src/fish_2 /tmp/dev-container-features \ 44 | >>> && chmod -R 0755 /tmp/dev-container-features/fish_2 \ [2024-04-21T04:36:25.996Z] 45 | >>> && cd /tmp/dev-container-features/fish_2 \ 46 | >>> && chmod +x ./devcontainer-features-install.sh \ 47 | >>> && ./devcontainer-features-install.sh \ 48 | >>> && rm -rf /tmp/dev-container-features/fish_2 49 | -------------------- ```
dylanrussellmd commented 7 months ago

I did a few more tests and I am still convinced this is ghcr.io/prulloac/devcontainer-features/latex:1's problem because after ghcr.io/prulloac/devcontainer-features/latex:1, ghcr.io/meaningful-ooo/devcontainer-features/fish:1 also failed to find fish.

Details

Thanks for helping me figure this out - I posted the issue on their repository. I'm surprised this hasn't been reported before, seems like two features that would be called together often.

eitsupi commented 7 months ago

As a workaround, it may be possible to use overrideFeatureInstallOrder to change the installation order and install ghcr.io/prulloac/devcontainer-features/latex:1 last.