ratchet update exit with errors

[ruzickap]

New issue?

• [X] I have reviewed the Known Issues and confirm this is not a known issue.

TL;DR

I'm getting error panic: runtime error: index out of range [40] with length 40 when running ratchet update mega-linter.yml

Expected behavior

It should update the GitHub Action file or give me the "understandable" error.

Observed behavior

No response

Version

0.9.1

On what operating system are you using ratchet?

Mac OS

CI/CD workflow file

---
name: mega-linter

on:
  workflow_dispatch:
  push:
    branches-ignore:
      - main

permissions: read-all

jobs:
  mega-linter:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - name: Checkout Code
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

      - name: Restore lychee cache
        uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
        with:
          path: .lycheecache
          key: cache-lychee-${{ github.sha }}
          restore-keys: cache-lychee-

      - name: Extract commands from markdown files
        run: |
          set -euxo pipefail
          echo '#!/usr/bin/env bash' > README.sh
          find . -name '*.md' -print0 | while IFS= read -r -d '' FILE; do
            # Extract:  ... 
            sed -n "/^\`\`\`\(bash\|shell\)$/,/^\`\`\`$/p" "${FILE}" | sed '/^```*/d' >> README.sh
            # Extract:   ```bash ... ```
            sed -n "/^  \`\`\`\(bash\|shell\)$/,/^  \`\`\`$/p" "${FILE}" | sed '/^  ```*/d; s/^  //' >> README.sh
          done
          ls -la README.sh
          chmod a+x README.sh

      - name: 💡 MegaLinter
        uses: oxsecurity/megalinter@a7a0163b6c8ff7474a283d99a706e27483ddd80f # v7.10.0
        env:
          GITHUB_COMMENT_REPORTER: false
          # Disabled due to error: [GitHub Status Reporter] Error posting Status for REPOSITORY with ...: 403
          GITHUB_STATUS_REPORTER: false
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Relevant log output

❯ ratchet update mega-linter.yml
panic: runtime error: index out of range [40] with length 40

goroutine 1 [running]:
github.com/sethvargo/ratchet/command.fixIndentation(0x1400011e0f0)
    github.com/sethvargo/ratchet/command/command.go:180 +0x404
github.com/sethvargo/ratchet/command.loadYAMLFiles({0x104df0358, 0x14000200080}, {0x14000200050, 0x1, 0x3?}, 0x1)
    github.com/sethvargo/ratchet/command/command.go:147 +0x1d4
github.com/sethvargo/ratchet/command.(*UpdateCommand).Run(0x14000198060, {0x104df30b0, 0x1400003b180}, {0x140000122c0, 0x1, 0x1})
    github.com/sethvargo/ratchet/command/update.go:72 +0x144
github.com/sethvargo/ratchet/command.Run({0x104df30b0, 0x1400003b180}, {0x140000122b0?, 0x2?, 0x2?})
    github.com/sethvargo/ratchet/command/command.go:58 +0x278
main.realMain(...)
    github.com/sethvargo/ratchet/main.go:26
main.main()
    github.com/sethvargo/ratchet/main.go:18 +0xc0

Additional information

Steps to reproduce the error:

$ cd /tmp/
$ wget https://raw.githubusercontent.com/ruzickap/my-git-projects/main/gh-repo-defaults/my-defaults/.github/workflows/mega-linter.yml
$ ratchet update mega-linter.yml
panic: runtime error: index out of range [40] with length 40

goroutine 1 [running]:
github.com/sethvargo/ratchet/command.fixIndentation(0x140001981e0)
    github.com/sethvargo/ratchet/command/command.go:180 +0x404
github.com/sethvargo/ratchet/command.loadYAMLFiles({0x104970358, 0x14000110080}, {0x14000110050, 0x1, 0x3?}, 0x1)
    github.com/sethvargo/ratchet/command/command.go:147 +0x1d4
github.com/sethvargo/ratchet/command.(*UpdateCommand).Run(0x14000198060, {0x1049730b0, 0x1400003b180}, {0x140000122c0, 0x1, 0x1})
    github.com/sethvargo/ratchet/command/update.go:72 +0x144
github.com/sethvargo/ratchet/command.Run({0x1049730b0, 0x1400003b180}, {0x140000122b0?, 0x2?, 0x2?})
    github.com/sethvargo/ratchet/command/command.go:58 +0x278
main.realMain(...)
    github.com/sethvargo/ratchet/main.go:26
main.main()
    github.com/sethvargo/ratchet/main.go:18 +0xc0

[sethvargo]

I know why this is happening, but I don't know WHY this is happening. The document separator (---) is messing up the line parsing. @ruzickap you can remove the document separator and everything will be happy. I need to think about how to fix this edge case though

[ruzickap]

Thank you, but starting the yaml files with --- is something considered as "good practice" and various yaml linters reports it (https://yamllint.readthedocs.io/en/latest/rules.html#module-yamllint.rules.document_start).

I will fait for the final fix...

Thank you

[sethvargo]

Yes, I know. You can remove it, run ratchet, and manually re-add it.

[sethvargo]

https://github.com/sethvargo/ratchet/pull/81

[stdedos]

I am having a number of unrelated errors with most of these files - even after removing ---: https://github.com/pylint-dev/pylint-pytest/tree/master/.github

[sethvargo]

@stdedos I just cloned and ran ratchet against that repo at HEAD using the code in #81 and it ran without errors:

➜  pylint-pytest master ❃ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   .github/workflows/release.yml

no changes added to commit (use "git add" and/or "git commit -a")
➜  pylint-pytest master ❃ git diff
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index 10b8cd1..4003613 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -21,10 +21,10 @@ jobs:
       url: https://pypi.org/project/pylint-pytest/
     steps:
       - name: Check out code from Github
-        uses: actions/checkout@v4
+        uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # ratchet:actions/checkout@v4
       - name: Set up Python ${{ env.DEFAULT_PYTHON }}
         id: python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # ratchet:actions/setup-python@v5
         with:
           python-version: ${{ env.DEFAULT_PYTHON }}
           check-latest: true

[stdedos]

Did you, by any chance, happen to "just pick the one that works"? 😅

$ find .github/ -name '*.yml' -or -name '*.yaml' | xargs -I{} ./ratchet pin {}
panic: runtime error: index out of range [95] with length 95

goroutine 1 [running]:
github.com/sethvargo/ratchet/command.fixIndentation(0xc0001121b0)
    github.com/sethvargo/ratchet/command/command.go:180 +0x4ed
github.com/sethvargo/ratchet/command.loadYAMLFiles({0x82c500, 0xc00010c070}, {0xc00010c040, 0x1, 0x416113?}, 0x1)
    github.com/sethvargo/ratchet/command/command.go:147 +0x255
github.com/sethvargo/ratchet/command.(*PinCommand).Run(0xc000015da0, {0x82f248, 0xc0000ab140}, {0xc000014140, 0x1, 0x1})
    github.com/sethvargo/ratchet/command/pin.go:83 +0x1cb
github.com/sethvargo/ratchet/command.Run({0x82f248, 0xc0000ab140}, {0xc000014130?, 0x2?, 0x2?})
    github.com/sethvargo/ratchet/command/command.go:58 +0x212
main.realMain(...)
    github.com/sethvargo/ratchet/main.go:26
main.main()
    github.com/sethvargo/ratchet/main.go:18 +0xba
panic: runtime error: index out of range [9] with length 9

goroutine 1 [running]:
github.com/sethvargo/ratchet/command.fixIndentation(0xc00028c0c0)
    github.com/sethvargo/ratchet/command/command.go:180 +0x4ed
github.com/sethvargo/ratchet/command.loadYAMLFiles({0x82c500, 0xc00011c070}, {0xc00011c040, 0x1, 0x416113?}, 0x1)
    github.com/sethvargo/ratchet/command/command.go:147 +0x255
github.com/sethvargo/ratchet/command.(*PinCommand).Run(0xc000191d70, {0x82f248, 0xc0001a3140}, {0xc000190110, 0x1, 0x1})
    github.com/sethvargo/ratchet/command/pin.go:83 +0x1cb
github.com/sethvargo/ratchet/command.Run({0x82f248, 0xc0001a3140}, {0xc000190100?, 0x2?, 0x2?})
    github.com/sethvargo/ratchet/command/command.go:58 +0x212
main.realMain(...)
    github.com/sethvargo/ratchet/main.go:26
main.main()
    github.com/sethvargo/ratchet/main.go:18 +0xba
panic: runtime error: index out of range [65] with length 65

goroutine 1 [running]:
github.com/sethvargo/ratchet/command.fixIndentation(0xc0000141e0)
    github.com/sethvargo/ratchet/command/command.go:180 +0x4ed
github.com/sethvargo/ratchet/command.loadYAMLFiles({0x82c500, 0xc0000320b0}, {0xc000032080, 0x1, 0x100000000000000?}, 0x1)
    github.com/sethvargo/ratchet/command/command.go:147 +0x255
github.com/sethvargo/ratchet/command.(*PinCommand).Run(0xc00011bd70, {0x82f248, 0xc00012b140}, {0xc00011a110, 0x1, 0x1})
    github.com/sethvargo/ratchet/command/pin.go:83 +0x1cb
github.com/sethvargo/ratchet/command.Run({0x82f248, 0xc00012b140}, {0xc00011a100?, 0x2?, 0x2?})
    github.com/sethvargo/ratchet/command/command.go:58 +0x212
main.realMain(...)
    github.com/sethvargo/ratchet/main.go:26
main.main()
    github.com/sethvargo/ratchet/main.go:18 +0xba

[sethvargo]

I'm using ratchet compiled from #81:

➜  Development  git clone https://github.com/pylint-dev/pylint-pytest
Cloning into 'pylint-pytest'...
remote: Enumerating objects: 1039, done.
remote: Counting objects: 100% (393/393), done.
remote: Compressing objects: 100% (230/230), done.
remote: Total 1039 (delta 223), reused 251 (delta 163), pack-reused 646
Receiving objects: 100% (1039/1039), 229.28 KiB | 14.33 MiB/s, done.
Resolving deltas: 100% (596/596), done.
➜  Development  cd pylint-pytest
➜  pylint-pytest master ✔ ratchet -v
ratchet (devel) (d905c9cd9c2f8741d02765f56b4d31681599e0fd, darwin/arm64)
➜  pylint-pytest master ✔ ratchet pin .github/workflows/*.{yml,yaml}
➜  pylint-pytest master ❃ git diff
diff --git a/.github/workflows/checks.yaml b/.github/workflows/checks.yaml
index 2ee5938..a66daa0 100644
--- a/.github/workflows/checks.yaml
+++ b/.github/workflows/checks.yaml
@@ -28,10 +28,10 @@ jobs:
       pre-commit-key: ${{ steps.generate-pre-commit-key.outputs.key }}
     steps:
       - name: Check out code from GitHub
-        uses: actions/checkout@v4
+        uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # ratchet:actions/checkout@v4
       - name: Set up Python ${{ env.DEFAULT_PYTHON }}
         id: python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # ratchet:actions/setup-python@v5
         with:
           python-version: ${{ env.DEFAULT_PYTHON }}
           check-latest: true
@@ -42,7 +42,7 @@ jobs:
             hashFiles('pyproject.toml', 'requirements/**.txt') }}" >> $GITHUB_OUTPUT
       - name: Restore Python virtual environment
         id: cache-venv
-        uses: actions/cache@v4.0.2
+        uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # ratchet:actions/cache@v4.0.2
         with:
           path: .venv
           key: >-
@@ -55,7 +55,7 @@ jobs:
             hashFiles('.pre-commit-config.yaml') }}" >> $GITHUB_OUTPUT
       - name: Restore pre-commit environment
         id: cache-precommit
-        uses: actions/cache@v4.0.2
+        uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # ratchet:actions/cache@v4.0.2
         with:
           path: ${{ env.PRE_COMMIT_CACHE }}
           key: >-
@@ -71,23 +71,23 @@ jobs:
     needs: prepare-base
     steps:
       - name: Check out code from GitHub
-        uses: actions/checkout@v4
+        uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # ratchet:actions/checkout@v4
       - name: Set up Python ${{ env.DEFAULT_PYTHON }}
         id: python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # ratchet:actions/setup-python@v5
         with:
           python-version: ${{ env.DEFAULT_PYTHON }}

# ...

Also note, you don't need xargs - see the command I ran above:

ratchet pin .github/workflows/*.{yml,yaml}

I'm pretty confident that merging #81 and releasing will resolve your issue.

[stdedos]

On a tanget (and since you don't have Discussions), what is the logic of # ratchet:actions/cache@v4.0.2 vs # ratchet:actions/setup-python@v5 supposed to be?

I think GHA parses "partial tags" (@v5) to support "auto-updating minors etc". However, any sha1 you may write can possibly only be a specific version.