Unable to run codspeed inside docker container

[finsberg]

Hi!

I am trying to use codspeed for the first time using GitHub action, but for some reason I cannot get it to work. One difference from the example online is that I need to run it inside a docker container. Here is the relevant PR: https://github.com/finsberg/fenicsx-pulse/pull/39

Running the tests locally works fine.

Here is my GitHub actions .yml file

name: codspeed-benchmarks

on:
  push:
    branches:
      - "main" # or "master"
  pull_request:
  # `workflow_dispatch` allows CodSpeed to trigger backtest
  # performance analysis in order to generate initial data.
  workflow_dispatch:

jobs:
  benchmarks:
    runs-on: ubuntu-22.04
    container: ghcr.io/fenics/dolfinx/dolfinx:nightly
    env:
      DEB_PYTHON_INSTALL_LAYOUT: deb_system

    steps:
      - uses: actions/checkout@v4

      - name: Install package needed by codspeed
        run: apt-get update && apt-get install -y curl

      - name: Install package
        run: python3 -m pip install -e .[benchmark]

      - name: Run benchmarks
        uses: CodSpeedHQ/action@v2
        with:

            token: ${{ secrets.CODSPEED_TOKEN }}
            run: python3 -m pytest tests/ --codspeed

Also, here are the logs from the run,

logs.txt

and it looks like the command

head_status=$(curl -I -fsSL -w "%{http_code}" -o /dev/null https://github.com/CodSpeedHQ/runner/releases/download/v$RUNNER_VERSION/codspeed-runner-installer.sh)

returns 404, which probably means that $RUNNER_VERSION is not set correctly. Any ideas on how to proceed?

[adriencaccia]

Hey!

I just looked at https://github.com/finsberg/fenicsx-pulse/pull/39, and as you have figured out, the problem was due to the fact that there were some dependencies missing in the image you are using.

We quite heavily rely on the pre-installed software on the runner: https://github.com/actions/runner-images/blob/ubuntu22/20240630.1/images/ubuntu/Ubuntu2204-Readme.md

[finsberg]

Indeed, but it might be a good idea to add a section about this in the docs. In particular, that you need to run the container with the option --privileged, e.g (in my case)

jobs:
  benchmarks:
    runs-on: ubuntu-22.04
    container:
      image: ghcr.io/fenics/dolfinx/dolfinx:v0.8.0
      options: --privileged

This was not easy to figure out.

Also it would be nice if installing curl and lsb-release was also part of the action. Figuring out that I was missing curl was quite straight forward, but that I was missing lsb-release was not obvious to me.

[finsberg]

Looks like the problem is still not resolved. This is the output from the logs

2024-07-11T14:35:59.3399682Z ========== 1 passed, 73 deselected, 2 warnings in 2208.81s (0:36:48) ===========
2024-07-11T14:35:59.9642717Z ##[endgroup]
2024-07-11T14:35:59.9643331Z ##[group]Upload the results
2024-07-11T14:36:00.0987151Z ##[error]Error Failed to open repository at path: /__w/fenicsx-pulse/fenicsx-pulse/
2024-07-11T14:36:00.0997393Z ##[error]Process completed with exit code 1.
2024-07-11T14:36:00.1087509Z Post job cleanup.
2024-07-11T14:36:00.1093772Z ##[command]/usr/bin/docker exec  073c7ad45a4332f7416a302ad293a80977fa4b42213576d1205162ae3d7bb439 sh -c "cat /etc/*release | grep ^ID"
2024-07-11T14:36:00.3096645Z [command]/usr/bin/git version
2024-07-11T14:36:00.3138545Z git version 2.34.1
2024-07-11T14:36:00.3184394Z Temporarily overriding HOME='/__w/_temp/4fd6889d-1fd4-4675-9dfe-ea7acd493452' before making global git config changes
2024-07-11T14:36:00.3186146Z Adding repository directory to the temporary git global config as a safe directory
2024-07-11T14:36:00.3192415Z [command]/usr/bin/git config --global --add safe.directory /__w/fenicsx-pulse/fenicsx-pulse
2024-07-11T14:36:00.3228681Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2024-07-11T14:36:00.3261260Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2024-07-11T14:36:00.3527375Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2024-07-11T14:36:00.3552722Z http.https://github.com/.extraheader
2024-07-11T14:36:00.3566195Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2024-07-11T14:36:00.3598635Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2024-07-11T14:36:00.3992313Z Stop and remove container: 67352b11bbd5453ea936becb4c1eb710_ghcriofenicsdolfinxdolfinxv080_8a5e96
2024-07-11T14:36:00.3997778Z ##[command]/usr/bin/docker rm --force 073c7ad45a4332f7416a302ad293a80977fa4b42213576d1205162ae3d7bb439
2024-07-11T14:36:00.6497579Z 073c7ad45a4332f7416a302ad293a80977fa4b42213576d1205162ae3d7bb439
2024-07-11T14:36:00.6527041Z Remove container network: github_network_c8a11f66ebc144f296fc52a8a76a0b09
2024-07-11T14:36:00.6532007Z ##[command]/usr/bin/docker network rm github_network_c8a11f66ebc144f296fc52a8a76a0b09
2024-07-11T14:36:00.9109536Z github_network_c8a11f66ebc144f296fc52a8a76a0b09
2024-07-11T14:36:00.9287842Z Cleaning up orphan processes

Any ideas on what is missing here?

[adriencaccia]

Looks like the problem is still not resolved. This is the output from the logs

2024-07-11T14:35:59.9643331Z ##[group]Upload the results
2024-07-11T14:36:00.0987151Z ##[error]Error Failed to open repository at path: /__w/fenicsx-pulse/fenicsx-pulse/
2024-07-11T14:36:00.0997393Z ##[error]Process completed with exit code 1.

Any ideas on what is missing here?

I investigated and added a way to add more details about the error. I found this: https://github.com/adriencaccia/fenicsx-pulse/actions/runs/9905400860/job/27365652535#step:7:131 The user in the docker container does not own the repository, so when we want to open it in the CLI, it fails.

You can fix this by adding the following step in your workflow: adriencaccia/fenicsx-pulse@a595830 (#1)

I released https://github.com/CodSpeedHQ/action/releases/tag/v2.4.3 that allows displaying causes of errors when debug logging is enabled in actions

[finsberg]

Thanks a lot for the help 🙏