codecov / feedback

A place to discuss feedback about the pull request and web product experience.
37 stars 7 forks source link

Base commit on main stuck to some months ago on Tanstack #576

Open Balastrong opened 1 week ago

Balastrong commented 1 week ago

Describe the bug On TanStack Query and Form all coverage reports are compared against really old base commits leading to wrong results when the Github Action writes comments in the PR (also makes the PR checks fail if below coverage threshold!)

Query -> Base commit is 2fe16e5 Form -> Base commit is 4e54c09

To Reproduce Unknown

Expected behavior Base commit is up to date, if a branch is opened from main's latest commit that's the expected base.

Screenshots Image Image

drazisil-codecov commented 1 day ago

Hi @Balastrong ,

It looks like, for https://app.codecov.io/gh/TanStack/form, the most recent SHA on the main branch, 4e54c0940b751e44a0a4a1b944fbfcccc206c761, was indeed months ago. Do you have a newer SHA, and possibly the CL output from the coverage upload step?

For https://app.codecov.io/gh/TanStack/query, the most recent commit on main is https://app.codecov.io/gh/TanStack/query/commit/4a2e838fc09b910656aee6b214ab37557c7064ef, which was 5 days ago. This was compared to https://app.codecov.io/gh/TanStack/query/commit/0df28830a756c0d8b7545e7a6f75c1eacfb2df54, also from 5 days ao.

Can you further detail what you are seeing?

Balastrong commented 1 day ago

Hey @drazisil-codecov, thanks for the answer!

For form let's use this comment as example.

As per Codecov's comment "Report is 167 commits behind head on main." but that branch has been created on main's head and immediately merged back. I was expecting Codecov to compare the PR with this commit instead that is the base commit for that branch.

drazisil-codecov commented 1 day ago

Hi @Balastrong It looks like https://github.com/TanStack/form/commit/04238b14535d3e8be43e259384b8ddeb53060a69 didn't upload coverage. Is it possible that https://app.codecov.io/gh/TanStack/form/commit/5473bb8be2e046dccc49a26c3f2ed85491aabb60 is the nst recent commit on main where coverage was uoloaded?

Balastrong commented 1 day ago

We do have an action with a step that uploads the coverage but now that you mentioned that I had a closer look

https://github.com/TanStack/form/actions/runs/11920712618/job/33223266275#step:8:66

info - 2024-11-19 19:59:09,748 -- Process Upload complete
error - 2024-11-19 19:59:09,749 -- Upload failed: {"message":"Token required because branch is protected"}

The step is marked as successful but this log seems to indicate it wasn't, I'm now learning about the existence of the fail_ci_if_error flag :D

This is how we upload the result

      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        with:
          directory: packages
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

I'll check if we have a problem with the token from the secrets, thanks!