PermissionError: [Errno 13] Permission denied: 'logs'

[yaozhang09]

v0.3 Action was working till about a week ago. Now container fails to pass the postcreatecommand after build with the error

PermissionError: [Errno 13] Permission denied: 'logs' [2023-08-28T17:44:17.307Z] postCreateCommand failed with exit code 2. Skipping any further user-provided commands.

I've tried pinning it to the latest devcontainers/ci@v0.3.1900000328 but to no avail. Our devcontainer.json:

{
    "name": "dbt",
    "build": {
        "dockerfile": "../dbt/Dockerfile",
        "args": { "VARIANT": "3.10-bullseye" }
    },
    "workspaceMount": "src=${localWorkspaceFolder},target=/usr/app,type=bind,consistency=cached",
    "mounts": [
        "src=${localEnv:HOME}/.dbt/profiles.yml,target=/home/vscode/.dbt/profiles.yml,type=bind,readonly",
        "src=${localEnv:HOME}/.config/gcloud/application_default_credentials.json,target=/var/secrets/google/client_secrets.json,type=bind,readonly"
    ],
    "workspaceFolder": "/usr/app/dbt",
    "containerEnv": {
        "GOOGLE_APPLICATION_CREDENTIALS": "${localEnv:GOOGLE_APPLICATION_CREDENTIALS:/var/secrets/google/client_secrets.json}",
        "TARGET": "${localEnv:TARGET:dev}",
        "GCP_PROJECT_ID": "${localEnv:GCP_PROJECT_ID:apollo-warehouse-dev}",
        "DBT_PROFILES_DIR": "${localEnv:DBT_PROFILES_DIR:~/.dbt/}",
        "DATASET": "${localEnv:DATASET}"
    },
    "forwardPorts": [
        8000
    ],
    "postCreateCommand": "dbt deps && dbt debug && dbt seed && pre-commit install && pre-commit install --hook-type post-checkout && cp target/manifest.json .", //Note: dbt deps is needed here and in Dockerfile so prod airflow works
    "postStartCommand": "git config --global --add safe.directory ${containerWorkspaceFolder}",
    "settings": {
        "terminal.integrated.defaultProfile.linux": "zsh",
        "python.pythonPath": "/usr/local/bin/python",
        "python.languageServer": "Pylance",
        "files.associations": {
            "*.sql": "jinja-sql"
        },
        "dbt.bigquery.projectId": "apollo-warehouse-dev",
        "dbt.bigquery.runnerTheme": "light",
        "sqlfluff.config": "${workspaceFolder}/.sqlfluff",
        "sqlfluff.linter.run": "onSave",
        "sqlfluff.experimental.format.executeInTerminal": true,
        "editor.formatOnSave": false
    },
    "remoteUser": "vscode",
    "customizations": {
        "vscode": {
            "extensions": [
                "eamodio.gitlens",
                "mhutchie.git-graph",
                "ms-azuretools.vscode-docker",
                "GitHub.vscode-pull-request-github",
                "MarkLarah.pre-commit-vscode",
                "dorzey.vscode-sqlfluff",
                "bastienboutonnet.vscode-dbt",
                "butchland.vscode-dbt-bigquery-power-user"
            ]
        }
    }
}

Workflow:

- name: Run git hooks on changed files
        uses: devcontainers/ci@v0.3.1900000328
        env: 
          GOOGLE_APPLICATION_CREDENTIALS: /var/secrets/google/client_secrets.json
        with:
          imageName: ${{ env.IMAGE_NAME }}
          imageTag: ${{ env.DATASET }}.gh-${{ github.run_id }}.${{ needs.pre-build.outputs.run_attempt }}
          runCmd: |
              cp ../artifacts/manifest.json manifest.json
              pre-commit run --files ${{ steps.parsed-changed-paths.outputs.paths }}

      - name: Run dbt test
        uses: devcontainers/ci@v0.3
        env: 
          GOOGLE_APPLICATION_CREDENTIALS: /var/secrets/google/client_secrets.json
        with:
          imageName: ${{ env.IMAGE_NAME }}
          imageTag: ${{ env.DATASET }}.gh-${{ github.run_id }}.${{ needs.pre-build.outputs.run_attempt }}
          runCmd: |
              cp ../artifacts/manifest.json manifest.json
              dbt test -m state:modified+ --defer --state .

[jlian]

I think we have the same issue. Our postCreateCommand fails when it tries to run npm install:

  [2023-08-25T22:03:44.202Z] Container started

  [2023-08-25T22:03:44.497Z] Running the postCreateCommand from devcontainer.json...

  [2023-08-25T22:03:52.155Z] npm ERR! code EACCES
  [2023-08-25T22:03:52.155Z] npm ERR! syscall mkdir
  npm ERR! path /workspaces/my-repo/node_modules
  npm ERR! errno -13

  [2023-08-25T22:03:52.158Z] npm ERR! Error: EACCES: permission denied, mkdir '/workspaces/my-repo/node_modules'
  npm ERR!  [Error: EACCES: permission denied, mkdir '/workspaces/my-repo/node_modules'] {
  npm ERR!   errno: -13,
  npm ERR!   code: 'EACCES',
  npm ERR!   syscall: 'mkdir',
  npm ERR!   path: '/workspaces/my-repo/node_modules'
  npm ERR! }
  npm ERR! 
  npm ERR! The operation was rejected by your operating system.
  npm ERR! It is likely you do not have the permissions to access this file as the current user
  npm ERR! 
  npm ERR! If you believe this might be a permissions issue, please double-check the
  npm ERR! permissions of the file and its containing directories, or try running
  npm ERR! the command again as root/Administrator.

  [2023-08-25T22:03:52.160Z] 
  npm ERR! A complete log of this run can be found in: /home/node/.npm/_logs/2023-08-25T22_03_44_982Z-debug-0.log

  [2023-08-25T22:03:52.179Z] postCreateCommand failed with exit code 243. Skipping any further user-provided commands.

  Error: Command failed: /bin/sh -c npm install
      at YY (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:235:130)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Rl (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:227:4396)
      at async yl (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:227:3741)
      at async Sl (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:227:2945)
      at async Js (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:227:2389)
      at async sAA (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:462:1346)
      at async mK (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:462:964)
      at async mAA (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:479:3660)
      at async LC (/usr/local/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:479:4775)

[bamurtaugh]

Thanks for opening. @samruddhikhandale have you seen any similar errors recently?

[ferronn-dev]

I'd like to pile on and say that my project has also started failing recently, also with permission errors when trying to write files in the checkout directory. Removing remoteUser=vscode from devcontainer.json did not help.

[ferronn-dev]

I also meant to say that the failure is the same with devcontainers/ci@0.2 or @0.3. I suspect something changed with the underlying GitHub runner.

[samruddhikhandale]

Thanks for looping me in. It definitely looks like an issue with the upstream action runners. https://github.com/actions/runner-images/issues/8160 and https://github.com/actions/runner-images/issues/8157 looks relevant.

Does following instructions from https://github.com/actions/runner-images/issues/8160#issuecomment-1697455235 fix your issues?

[ferronn-dev]

Worked like a charm! Thanks for finding that!

Is this something that's more permanently resolvable on the devcontainers side, or are we stuck with these workarounds long-term?

[samruddhikhandale]

Is this something that's more permanently resolvable on the devcontainers side, or are we stuck with these workarounds long-term?

We have https://github.com/devcontainers/images/issues/723 open on the images side which tracks this. We will definitely have some way out soon, then you won't have to use these workarounds. Thank you for your patience!

[samruddhikhandale]

Crpss-posting from https://github.com/devcontainers/images/issues/723#issuecomment-1701546899

Quick update: Investigated this issue with the dev containers team, few findings -

• The dev container CLI should have automatic UID/GID syncing, unless the UID or GIDs were already in use
• From https://github.com/actions/runner-images/issues/8157, Docker's group was 999, and they then update the user's default group to that rather than adding the docker group to the user
• This caused the issue, hence, we necessarily don't have to update anything on the images side
• However, we could update our Features to use system groups with lower IDs to try to avoid this in the future. We do this pretty frequently to deal with priv issues
• https://github.com/actions/runner-images/pull/8201 which fixes docker group issues is already merged. Waiting for the action-runners team to release new images. Looks like that would fix the entire issue.

[samruddhikhandale]

Newer ubuntu (action runner) images were released last week with https://github.com/actions/runner-images/pull/8201 changes. This has fixed the ID problems we have been seeing.

Also, on the CLI side, we have https://github.com/devcontainers/cli/pull/635 opened which fixes https://github.com/microsoft/vscode-remote-release/issues/7284

Feel free to reopen if the issue resurfaces.