Bug Report - Githubissues

superlopuh commented 1 month ago

I recently switched to codecov/codecov-action@v4, and since then I've not been seeing any line by line coverage, on the website or in any of the actions.

Here is a recent example:

https://github.com/xdslproject/xdsl/pull/2621

Here's the relevant action for that PR:

https://github.com/xdslproject/xdsl/actions/runs/9188980377/job/25269976878

And the config:

    - name: Upload coverage to Codecov
      if: matrix.python-version == '3.10'
      uses: Wandalen/wretry.action@v1
      with:
        action: codecov/codecov-action@v4
        attempt_delay: 10000
        attempt_limit: 10
        with: |
          fail_ci_if_error: true
          verbose: true
          files: coverage.xml
          token: ${{ secrets.CODECOV_TOKEN }}

On the codecov website, I see this for every file:

There was a problem getting the source code from your provider. Unable to show line by line coverage.

How do we get code coverage back?

drazisil-codecov commented 1 month ago

Hi @superlopuh ,

Can you try changing the fetch depth, as explained here https://docs.codecov.com/docs/environment-specific-requirements#github-actions ?

(That's newly documented as of a few days ago, it was missed when we switched to v4.

superlopuh commented 1 month ago

Thank you for the suggestion. I tried it out in these two PRs:

https://github.com/xdslproject/xdsl/pull/2650 https://github.com/xdslproject/xdsl/pull/2651

And the diff is still missing.

drazisil-codecov commented 1 month ago

Hi @superlopuh ,

Interesting , it still isn't finding your repo files. How are you calling the action? Is it in a normal job with the repo checked out? A standalone job? A Docker container? Some other way? :)

superlopuh commented 1 month ago

I'm not sure that I understand your question, here's the action file: https://github.com/xdslproject/xdsl/blob/bd7ffdfa722561bb5bb501a5fc400229c3aff366/.github/workflows/ci-mlir.yml

It runs on every PR and when merged to main. It runs on the GitHub CI, runs a bunch of tests, combines the coverage, and then uploads to CodeCov.

superlopuh commented 1 month ago

One of the things I'm a little confused by is this message in the CodeCov website:

There was a problem getting the source code from your provider. Unable to show line by line coverage.

Which suggests that there's a different problem to the PR not being able to display the diff, but rather in how our coverage data is collected and uploaded.

drazisil-codecov commented 1 month ago

Hi @superlopuh

You actually have two issues:

Codecov can't match your coverage files to any files in your repo, due to not being able to get your repo file list at upload time
You are seeing an error in codecov.io because we are unable to fetch your repo file (at the time you are trying to view it) This is usually due to either a bad path, or bad access permissions from the token we use (should be yours at that time)

Let me address one at a time, please 🙏

drazisil-codecov commented 1 month ago

For the first, can you please add root_dir: xdsl to your codecov-action values? Let's see if that helps.

superlopuh commented 1 month ago

Thank you! What is the best way to see whether it's helped? Is it to merge the PR and see if the changes are reflected on the codecov.io website? Or is there something I can see in the logs?

Here are the logs for the run with the root_dir: xdsl included:

info - 2024-05-29 15:01:29,236 -- ci service found: github-actions
debug - 2024-05-29 15:01:29,238 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
debug - 2024-05-29 15:01:29,239 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
warning - 2024-05-29 15:01:29,240 -- No config file could be found. Ignoring config.
debug - 2024-05-29 15:01:29,240 -- No codecov_yaml found
debug - 2024-05-29 15:01:29,241 -- Starting create commit process --- {"commit_sha": "d6020e7bafcd221934dac65df9d4d29a6c361ec5", "parent_sha": null, "pr": "2650", "branch": "sasha/codecov", "slug": "xdslproject/xdsl", "token": "2******************", "service": "github", "enterprise_url": null}
info - 2024-05-29 15:01:29,367 -- Process Commit creating complete
debug - 2024-05-29 15:01:29,368 -- Commit creating result --- {"result": "RequestResult(error=None, warnings=[], status_code=201, text='{\"message\":null,\"timestamp\":\"2024-05-29T15:01:29.332896Z\",\"ci_passed\":null,\"state\":null,\"repository\":{\"name\":\"xdsl\",\"is_private\":false,\"active\":true,\"language\":\"python\",\"yaml\":{\"fixes\":[\"xdsl/::\"],\"ignore\":[\"(?s:.*/[^\\\\\\\\/]*\\\\\\\\.ipynb)\\\\\\\\Z\"],\"codecov\":{\"notify\":{\"wait_for_ci\":true},\"require_ci_to_pass\":true},\"coverage\":{\"range\":[80.0,90.0],\"round\":\"down\",\"status\":{\"patch\":{\"default\":{\"enabled\":false}},\"project\":{\"default\":{\"target\":1.0,\"enabled\":true,\"threshold\":0.1}}},\"precision\":2}}},\"author\":null,\"commitid\":\"d6020e7bafcd221934dac65df9d4d29a6c361ec5\",\"parent_commit_id\":null,\"pullid\":2650,\"branch\":\"sasha/codecov\"}')"}
==> Running command '/home/runner/work/_actions/Wandalen/codecov-action/dist/codecov -v create-report'
/home/runner/work/_actions/Wandalen/codecov-action/dist/codecov -v create-report --git-service github -C d6020e7bafcd221934dac65df9d4d29a6c361ec5 -Z
info - 2024-05-29 15:01:30,072 -- ci service found: github-actions
debug - 2024-05-29 15:01:30,075 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
debug - 2024-05-29 15:01:30,076 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
warning - 2024-05-29 15:01:30,077 -- No config file could be found. Ignoring config.
debug - 2024-05-29 15:01:30,077 -- No codecov_yaml found
debug - 2024-05-29 15:01:30,078 -- Starting create report process --- {"commit_sha": "d6020e7bafcd221934dac65df9d4d29a6c361ec5", "code": "default", "slug": "xdslproject/xdsl", "service": "github", "enterprise_url": null, "token": "2******************"}
info - 2024-05-29 15:01:30,182 -- Process Report creating complete
debug - 2024-05-29 15:01:30,182 -- Report creating result --- {"result": "RequestResult(error=None, warnings=[], status_code=201, text='{\"external_id\":\"b07a7dfe-d631-4e0c-9d52-5ea978106109\",\"created_at\":\"2024-05-29T15:01:30.147324Z\",\"commit_sha\":\"d6020e7bafcd221934dac65df9d4d29a6c361ec5\",\"code\":null}')"}
info - 2024-05-29 15:01:30,182 -- Finished creating report successfully --- {"response": "{\"external_id\":\"b07a7dfe-d631-4e0c-9d52-5ea978106109\",\"created_at\":\"2024-05-29T15:01:30.147324Z\",\"commit_sha\":\"d6020e7bafcd221934dac65df9d4d29a6c361ec5\",\"code\":null}"}
==> Running command '/home/runner/work/_actions/Wandalen/codecov-action/dist/codecov -v do-upload'
/home/runner/work/_actions/Wandalen/codecov-action/dist/codecov -v do-upload -Z -f coverage.xml --git-service github -C d6020e7bafcd221934dac65df9d4d29a6c361ec5 --network-root-folder xdsl
info - 2024-05-29 15:01:30,885 -- ci service found: github-actions
debug - 2024-05-29 15:01:30,887 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
debug - 2024-05-29 15:01:30,889 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
warning - 2024-05-29 15:01:30,889 -- No config file could be found. Ignoring config.
debug - 2024-05-29 15:01:30,889 -- No codecov_yaml found
debug - 2024-05-29 15:01:30,890 -- Starting upload processing --- {"branch": "sasha/codecov", "build_code": "9288421232", "build_url": "https://github.com/xdslproject/xdsl/actions/runs/9288421232", "commit_sha": "d6020e7bafcd221934dac65df9d4d29a6c361ec5", "disable_file_fixes": false, "disable_search": false, "enterprise_url": null, "env_vars": {}, "files_search_exclude_folders": [], "files_search_explicitly_listed_files": ["coverage.xml"], "files_search_root_folder": "/home/runner/work/xdsl/xdsl", "flags": [], "git_service": "github", "handle_no_reports_found": false, "job_code": "CI - MLIR-based Testing", "name": "9288421232", "network_filter": null, "network_prefix": null, "network_root_folder": "xdsl", "plugin_names": ["xcode", "gcov", "pycoverage"], "pull_request_number": "2650", "report_code": "default", "slug": "xdslproject/xdsl", "token": "2******************", "upload_file_type": "coverage"}
debug - 2024-05-29 15:01:30,891 -- Selected preparation plugins --- {"selected_plugins": ["<class 'codecov_cli.plugins.xcode.XcodePlugin'>", "<class 'codecov_cli.plugins.gcov.GcovPlugin'>", "<class 'codecov_cli.plugins.pycoverage.Pycoverage'>"]}
debug - 2024-05-29 15:01:30,891 -- Running preparation plugin: <class 'codecov_cli.plugins.xcode.XcodePlugin'>
debug - 2024-05-29 15:01:30,891 -- Running xcode plugin...
warning - 2024-05-29 15:01:30,891 -- xcrun is not installed or can't be found.
debug - 2024-05-29 15:01:30,891 -- Running preparation plugin: <class 'codecov_cli.plugins.gcov.GcovPlugin'>
debug - 2024-05-29 15:01:30,891 -- Running gcov plugin...
warning - 2024-05-29 15:01:30,979 -- No gcov data found.
debug - 2024-05-29 15:01:30,979 -- Running preparation plugin: <class 'codecov_cli.plugins.pycoverage.Pycoverage'>
info - 2024-05-29 15:01:31,051 -- Generating coverage.xml report in /home/runner/work/xdsl/xdsl/xdsl

I'm a little concerned by the fact that it can't find the codecov.yml in the root folder.

drazisil-codecov commented 1 month ago

@thomasrockhu-codecov I think I need to tag you in here. It's unclear to me why the CLI can't find the file network. Using the action.

superlopuh commented 3 weeks ago

Hi @drazisil-codecov @thomasrockhu-codecov, any follow-up on this? I'd be happy with any pointers, I'm just not sure where to start looking.

thomasrockhu-codecov commented 3 weeks ago

hi @superlopuh sorry I haven't gotten a chance to dig into this. I'll take a look over the weekend

superlopuh commented 2 weeks ago

Thank you!

codecov / feedback

Bug Report #388