[Bug]: Missing PR comments

alberto-miranda commented 9 months ago

What happened?

After some time of using the action with great success, the PR comments suddenly disappeared. The logs from the action run don't contain any explicit errors, and all steps seem to work as expected. We tried deleting the coverage branch and updating to the latest version of the action but the behavior persists.

The only potentially-related messages I was able to find in the logs (even with Debug mode enabled) are the following:

fatal: a branch named '_xml_coverage' already exists:

▼ Run git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
  git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
  git config --global user.name "github-actions[bot]"
  # Switch to the branch if it already exists
  git switch _xml_coverage || true
  git pull origin _xml_coverage || true
  # Create the branch if it doesn't exist yet
  git checkout --orphan _xml_coverage || true
  # Ensure that the bare minimum components exist in the branch
  mkdir -p data
  touch README.md data/.gitkeep
  # Copy necessary files and folders to a temporary location
  mkdir -p /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8
  echo "Copying data to /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8"
  cp -r .git README.md data /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8
  # Remove everything else
  # Attribution: https://unix.stackexchange.com/a/77313
  rm -rf ..?* .[!.]* *
  # Restore files from the temporary location
  echo "Copying data from /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8"
  cp -r /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8/.git /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8/README.md /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8/data .
  rm -rf /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8
  git add --all -f
  git commit -m "Update storage branch: $(date)" || true
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.11.7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib
##[debug]/usr/bin/bash --noprofile --norc -e -o pipefail /home/runner/work/_temp/d06e431e-a16d-4966-9068-38b881147dd1.sh
Switched to a new branch '_xml_coverage'
branch '_xml_coverage' set up to track 'origin/_xml_coverage'.
From https://github.com/exampleorg/repo
 * branch            _xml_coverage -> FETCH_HEAD
Already up to date.
fatal: a branch named '_xml_coverage' already exists
Copying data to /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8
Copying data from /tmp/cfd24555d4f2483afc5f83f742e1eb02bfe120a8
On branch _xml_coverage
Your branch is up to date with 'origin/_xml_coverage'.

nothing to commit, working tree clean

no pull request numbers given: skip step:

▼ Run marocchino/sticky-pull-request-comment@v2
with:
header: coverage/e2e/merged/coverage-e2e.xml
path: .coverage-output.final
append: false
recreate: false
delete: false
only_create: false
only_update: false
hide_details: false
hide: false
hide_and_recreate: false
hide_classify: OUTDATED
ignore_empty: false
skip_unchanged: false
follow_symbolic_links: false
GITHUB_TOKEN: ***
env:
pythonLocation: /opt/hostedtoolcache/Python/3.11.7/x64
PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib/pkgconfig
Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib
diff_storage_branch: 387-missing-coverage-comments-in-pull-requests
no pull request numbers given: skip step

Relevant log output

No response

Code of Conduct

[X] I agree to follow this project's Code of Conduct.

Contribution Guidelines

[X] I agree to follow this project's Contribution Guidelines.

Security Policy

[X] I agree to follow this project's Security Policy.

cicdguy commented 9 months ago

Thanks for the detailed report. We will investigate this.

cicdguy commented 9 months ago

@alberto-miranda - Do you know (approximately) when you started seeing this issue? We made a release yesterday v2.5.1 but we did test that it was working okay on other repositories, including this one. Eg: https://github.com/insightsengineering/coverage-action/pull/37

Given the error in the sticky PR comment (no pull request numbers given: skip step:), we might need to just add the following to this action: https://github.com/marocchino/sticky-pull-request-comment?tab=readme-ov-file#comment-from-push

alberto-miranda commented 9 months ago

I just check and the last PR where we had comments is from two weeks ago. Interestingly, around that time we merged a PR that modified our build.yml and removed the on: pull_request: keys:

on:
  push:
    branches:
-      - "*"
+      - "**"
      - "!refs/tags/.*"
    tags-ignore:
-      - "*"
-    paths-ignore:
-      - "docs/**"
-      - mkdocs.yml
-      - .github/workflows/website_preview.yml
-      - .github/workflows/website_deploy.yml
-  pull_request:
-    types: [opened, reopened, synchronize]
+      - "**"
    paths-ignore:
      - "docs/**"
      - mkdocs.yml

I'm starting to think that this might be the root cause of our problem.

cicdguy commented 9 months ago

Yes that could explain it. It'd be imperative to have the pull_request event type for the comments to be created during PRs.

alberto-miranda commented 9 months ago

Makes sense. I'll find out why this was removed and whether we can reinstate it. Not saying you have to do it, but it might be a nice addition to propagate this option https://github.com/marocchino/sticky-pull-request-comment?tab=readme-ov-file#comment-from-push as a coverage-action option :wink:

cicdguy commented 9 months ago

Indeed, let me go ahead and do that anyway.

insightsengineering / coverage-action