search

nektos / act

Run your GitHub Actions locally 🚀
https://nektosact.com
MIT License
53.74k stars 1.34k forks source link

act git module fails process repos that use `.git/packed-refs` #1369

Closed Josverl closed 1 year ago

Josverl commented 1 year ago

Bug report info

act version:            0.2.30
GOOS:                   windows
GOARCH:                 amd64
NumCPU:                 8
Docker host:            DOCKER_HOST environment variable is unset/empty.
Sockets found:
        \\.\pipe\docker_engine
Config files:
        C:\Users\josverl\.actrc:
                -P ubuntu-latest=catthehacker/ubuntu:full-latest
                -P ubuntu-latest-m=catthehacker/ubuntu:act-latest
                -P ubuntu-20.04=catthehacker/ubuntu:act-20.04
                -P ubuntu-18.04=catthehacker/ubuntu:act-18.04
Build info:
        Go version:            go1.18.4
        Module path:           github.com/nektos/act
        Main version:          (devel)
        Main path:             github.com/nektos/act
        Main checksum:
        Build settings:
                -compiler:            gc
                -ldflags:             -s -w -X main.version=0.2.30 -X main.commit=3364f95569d8a39d440b18af69d29421fcc77b55 -X main.date=2022-08-01T02:56:59Z -X main.builtBy=goreleaser
                CGO_ENABLED:          0
                GOARCH:               amd64
                GOOS:                 windows
                GOAMD64:              v1
                vcs:                  git
                vcs.revision:         3364f95569d8a39d440b18af69d29421fcc77b55
                vcs.time:             2022-07-29T19:43:24Z
                vcs.modified:         false
Docker Engine:
        Engine version:        20.10.17
        Engine runtime:        runc
        Cgroup version:        1
        Cgroup driver:         cgroupfs
        Storage driver:        overlay2
        Registry URI:          https://index.docker.io/v1/
        OS:                    Docker Desktop
        OS type:               linux
        OS version:
        OS arch:               x86_64
        OS kernel:             5.10.102.1-microsoft-standard-WSL2
        OS CPU:                8
        OS memory:             7907 MB
        Security options:
                name=seccomp,profile=default

Command used with act

act workflow_dispatch -j get-micropython-doc-stubs

Describe issue

act reports errors getting the git ref and git revision, and hangs on the associated cleanup tasks at the end of the job(s)

the used git module appears to be assuming thet the gist refs are in a simple format , and do not process the '.git/packed-refs'

When repository histories start getting really long, having hundreds of tags starts to slow things down, so git pack-refs will put them all in the packed-refs file.

Link to GitHub repository

https://github.com/Josverl/micropython-stubs

Workflow content

# Check out repos in this structure 
# micropython-stubs
# +-- stubs 
# +-- repos 
#     +-- micropython
#     +-- micropython-lib
# repro structure needed to allow automatic PR creation to work
########################################################################################################################################
name: get-doc-stubs

on:
  workflow_dispatch:
  schedule: 
    - cron: "0 1 * * 5"   # Run every Friday at 01:00

jobs:
  ########################################################################################################################################
  list_versions:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      # no need to install python 
      - run: pip install pygithub packaging
      - run: python .github/workflows/list_versions.py
        id: dynamic
    outputs:
      job_versions: ${{ steps.dynamic.outputs.versions }}

  ########################################################################################################################################
  get-micropython-doc-stubs:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    continue-on-error: true
    needs: list_versions
    strategy:
      matrix: ${{ fromJSON(needs.list_versions.outputs.job_versions) }}

    # Most scripts will be run from the stubber, set that as default
    defaults:
      run:
        shell: bash

    steps:

      - name: Checkout stubs repo
        uses: actions/checkout@v3

      # make Python work
      - name: Set up Python
        uses: actions/setup-python@v3
        with:
          python-version: 3.9
          cache: 'pip'

      - name: Install Python dependencies (stubber)
        run: |
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

      # micropython as a sub-module 
      - name: 'clone and checkout micropython ${{ matrix.version }}'
        run: |
          stubber clone
          stubber switch --tag ${{ matrix.version }}

      ######################################
      # This is where the actual work starts
      ######################################

      - name: Generate document stubs for ${{ matrix.version }}
        run: |
          stubber get-docstubs

      - name: check for changes other than modules.py
        shell: pwsh
        id: changed
        run: |
          $changed = @(git whatchanged -1 --format=oneline  | ?{ $_.Contains(".py") })
          write-host "Changed stub files:"
          write-host $changed
          write-host "::set-output name=count::$($changed.count)"

      - name: Create Pull Request ( if relevant changes) 
        if: ${{ steps.changed.outputs.count }} >= 1
        id: cpr-doc
        uses: peter-evans/create-pull-request@v3
        # ref: https://github.com/peter-evans/create-pull-request
        with:
          token: ${{ secrets.CREATE_PR_ACTION_TOKEN }} 
          title: "Add/update MicroPython ${{ matrix.version }} doc-stubs"
          commit-message: add/update doc-stubs
          body : |
            update MicroPython ${{ matrix.version }} doc-stubs
            based on micropython commit ${{ steps.mpycommit.outputs.info }}

          branch: "micropython-${{ matrix.version }}-doc-stubs"
          labels: |
            automated pr
            Doc stubs 
            MicroPython
            ${{ matrix.version }}
          delete-branch: true

      - name: done
        if: always()
        run: |
          echo "Done..."

Relevant log output

[get-doc-stubs/list_versions] [DEBUG] Loading slug from git directory 'C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git'
[get-doc-stubs/list_versions] [DEBUG] Loading revision from git directory 'C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git'
[get-doc-stubs/list_versions] unable to get git ref: open C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git\refs\heads\main: The system cannot find the file specified.
[get-doc-stubs/list_versions] unable to get git revision: open C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git\refs\heads\main: The system cannot find the file specified.
[get-doc-stubs/list_versions] [DEBUG] Loading slug from git directory 'C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git'
[get-doc-stubs/list_versions] [DEBUG] Loading revision from git directory 'C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git'
[get-doc-stubs/list_versions] unable to get git ref: open C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git\refs\heads\main: The system cannot find the file specified.
[get-doc-stubs/list_versions] unable to get git revision: open C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git\refs\heads\main: The system cannot find the file specified.
[get-doc-stubs/list_versions] [DEBUG] Loading slug from git directory 'C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git'
[get-doc-stubs/list_versions] [DEBUG] Loading revision from git directory 'C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git'
[get-doc-stubs/list_versions] unable to get git ref: open C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git\refs\heads\main: The system cannot find the file specified.
[get-doc-stubs/list_versions] unable to get git revision: open C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git\refs\heads\main: The system cannot find the file specified.
...
...

| Pull request branch target repository set to josverl/micropython-stubs.git
[get-doc-stubs/get-micropython-doc-stubs]   ❓  ::group::Configuring credential for HTTPS authentication
[get-doc-stubs/get-micropython-doc-stubs]   ⚙  ***
| [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
[get-doc-stubs/get-micropython-doc-stubs]   ❓  ::endgroup::
[get-doc-stubs/get-micropython-doc-stubs]   ❓  ::group::Checking the base repository state
| [command]/usr/bin/git symbolic-ref HEAD --short
| main
| Working base is branch 'main'
| [command]/usr/bin/git remote prune origin
************************************************************
* At this point ACT hangs until Ctrl-C is pressed - possibly unrelated to this issue 
************************************************************
[get-doc-stubs/get-micropython-doc-stubs]   ❌  Failure - Main Create Pull Request ( if relevant changes)
[get-doc-stubs/get-micropython-doc-stubs] context canceled
time="2022-10-01T18:28:43+02:00" level=debug msg="Loading slug from git directory 'C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git'" stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=debug msg="Loading revision from git directory 'C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git'" stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=warning msg="unable to get git ref: open C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git\\refs\\heads\\main: The system cannot find the file specified." stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=warning msg="unable to get git revision: open C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git\\refs\\heads\\main: The system cannot find the file specified." stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=debug msg="Loading slug from git directory 'C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git'" stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=debug msg="Loading revision from git directory 'C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git'" stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=warning msg="unable to get git ref: open C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git\\refs\\heads\\main: The system cannot find the file specified." stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=warning msg="unable to get git revision: open C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git\\refs\\heads\\main: The system cannot find the file specified." stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=debug msg="setupEnv => map[ACCEPT_EULA:Y ACT:true AGENT_TOOLSDIRECTORY:/opt/hostedtoolcache ANDROID_HOME:/usr/local/lib/android/sdk ANDROID_NDK_HOME:/usr/local/lib/android/sdk/ndk-bundle ANDROID_NDK_LATEST_HOME:/usr/local/lib/android/sdk/ndk/22.1.7171670 ANDROID_NDK_ROOT:/usr/local/lib/android/sdk/ndk-bundle ANDROID_SDK_ROOT:/usr/local/lib/android/sdk ANT_HOME:/usr/share/ant AZURE_EXTENSION_DIR:/opt/az/azcliextensions BOOTSTRAP_HASKELL_NONINTERACTIVE:1 CHROMEWEBDRIVER:/usr/local/share/chrome_driver CHROME_BIN:/usr/bin/google-chrome CI:true CONDA:/usr/share/miniconda DEBIAN_FRONTEND:noninteractive DEPLOYMENT_BASEPATH:/opt/runner DOTNET_MULTILEVEL_LOOKUP:0 DOTNET_NOLOGO:1 DOTNET_SKIP_FIRST_TIME_EXPERIENCE:1 GECKOWEBDRIVER:/usr/local/share/gecko_driver GITHUB_ACTION:post-1 GITHUB_ACTIONS:true GITHUB_ACTION_PATH: GITHUB_ACTION_REF: GITHUB_ACTION_REPOSITORY: GITHUB_ACTOR:nektos/act GITHUB_API_URL:https://api.github.com GITHUB_BASE_REF: GITHUB_ENV:/var/run/act/workflow/envs.txt GITHUB_EVENT_NAME:workflow_dispatch GITHUB_EVENT_PATH:/var/run/act/workflow/event.json GITHUB_GRAPHQL_URL:https://api.github.com/graphql GITHUB_HEAD_REF: GITHUB_JOB:get-micropython-doc-stubs GITHUB_PATH:/var/run/act/workflow/paths.txt GITHUB_REF:master GITHUB_REF_NAME: GITHUB_REF_TYPE: GITHUB_REPOSITORY:josverl/micropython-stubs GITHUB_REPOSITORY_OWNER:josverl GITHUB_RETENTION_DAYS:0 GITHUB_RUN_ID:1 GITHUB_RUN_NUMBER:1 GITHUB_SERVER_URL:https://github.com GITHUB_SHA: GITHUB_TOKEN: GITHUB_WORKFLOW:get-doc-stubs GITHUB_WORKSPACE:/mnt/c/develop/MyPython/micropython-stubber/repos/micropython-stubs GOROOT_1_14_X64:/opt/hostedtoolcache/go/1.14.15/x64 GOROOT_1_15_X64:/opt/hostedtoolcache/go/1.15.15/x64 GOROOT_1_16_X64:/opt/hostedtoolcache/go/1.16.7/x64 GRAALVM_11_ROOT:/usr/local/graalvm/graalvm-ce-java11-21.2.0 GRADLE_HOME:/usr/share/gradle-7.2 HOMEBREW_CELLAR:/home/linuxbrew/.linuxbrew/Cellar HOMEBREW_CLEANUP_PERIODIC_FULL_DAYS:3650 HOMEBREW_NO_AUTO_UPDATE:1 HOMEBREW_PREFIX:/home/linuxbrew/.linuxbrew HOMEBREW_REPOSITORY:/home/linuxbrew/.linuxbrew/Homebrew HOMEBREW_SHELLENV_PREFIX:/home/linuxbrew/.linuxbrew IMAGE_OS:ubuntu20 INPUT_ARCHITECTURE: INPUT_CACHE:pip INPUT_CACHE-DEPENDENCY-PATH: INPUT_PYTHON-VERSION:3.9 INPUT_TOKEN: ImageOS:ubuntu20 ImageVersion:dev JAVA_HOME:/usr/lib/jvm/adoptopenjdk-11-hotspot-amd64 JAVA_HOME_11_X64:/usr/lib/jvm/adoptopenjdk-11-hotspot-amd64 JAVA_HOME_8_X64:/usr/lib/jvm/adoptopenjdk-8-hotspot-amd64 LD_LIBRARY_PATH:/opt/hostedtoolcache/Python/3.9.6/x64/lib LEIN_HOME:/usr/local/lib/lein LEIN_JAR:/usr/local/lib/lein/self-installs/leiningen-2.9.6-standalone.jar LSB_OS_VERSION:2004 LSB_RELEASE:20.04 NVM_DIR:/home/runner/.nvm PATH:/opt/hostedtoolcache/Python/3.9.6/x64/bin:/opt/hostedtoolcache/Python/3.9.6/x64:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/runner/.local/bin:/opt/pipx_bin:/usr/share/rust/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/snap/bin:/opt/hostedtoolcache/node/14.17.5/x64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PIPX_BIN_DIR:/opt/pipx_bin PIPX_HOME:/opt/pipx RUNNER_PERFLOG:/dev/null RUNNER_TEMP:/home/runner/work/_temp RUNNER_TRACKING_ID: RUNNER_USER:runner RUN_TOOL_CACHE:/opt/hostedtoolcache SELENIUM_JAR_PATH:/usr/share/java/selenium-server-standalone.jar SWIFT_PATH:/usr/share/swift/usr/bin USER:runner VCPKG_INSTALLATION_ROOT:/usr/local/share/vcpkg XDG_CONFIG_HOME:/home/runner/.config pythonLocation:/opt/hostedtoolcache/Python/3.9.6/x64]" stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=debug msg="Loading slug from git directory 'C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git'" stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=debug msg="Loading revision from git directory 'C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git'" stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=warning msg="unable to get git ref: open C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git\\refs\\heads\\main: The system cannot find the file specified." stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=warning msg="unable to get git revision: open C:\\develop\\MyPython\\micropython-stubber\\repos\\micropython-stubs\\.git\\refs\\heads\\main: The system cannot find the file specified." stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=debug msg="evaluating expression 'success()'" stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=debug msg="expression 'success()' evaluated to 'false'" stage=Post step="Set up Python" stepID="[1]"
time="2022-10-01T18:28:43+02:00" level=debug msg="Skipping step 'Set up Python' due to 'success()'" stage=Post step="Set up Python" stepID="[1]" stepResult=skipped
time="2022-10-01T18:28:43+02:00" level=debug msg="skipping post step for 'Checkout stubs repo': no action model available" stage=Post step="Checkout stubs repo" stepID="[0]" stepResult=skipped
time="2022-10-01T18:28:43+02:00" level=info msg="🏁  Job succeeded" jobResult=success
[get-doc-stubs/get-micropython-doc-stubs] [DEBUG] Loading slug from git directory 'C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git'
[get-doc-stubs/get-micropython-doc-stubs] [DEBUG] Loading revision from git directory 'C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git'
[get-doc-stubs/get-micropython-doc-stubs] unable to get git ref: open C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git\refs\heads\main: The system cannot find the file specified.
[get-doc-stubs/get-micropython-doc-stubs] unable to get git revision: open C:\develop\MyPython\micropython-stubber\repos\micropython-stubs\.git\refs\heads\main: The system cannot find the file specified.
Error: context canceled

Additional information

the same issue is seen with almost all workflows in this and other repos in which git pack-refs has been used.

Josverl commented 1 year ago

it may be better to use it show-ref refs/heads/<branch> , or to enhance the algorithm in func FindGitRevision to

github-actions[bot] commented 1 year ago

Issue is stale and will be closed in 14 days unless there is new activity

AlexRuiz7 commented 1 year ago

Facing the same problem here.

@Josverl were you able to find a solution?

Josverl commented 1 year ago

No, not really The only dirty workaround is to create a copy of the repo without packed refs and then trying to keep that up to date

My go skill are read-only