[bug][github actions] codecov base commit does not match the "merge commit" that the branch used to generate coverage

On codecov reports (github actions), we see the base commit used to compare the PR is rather stale. example PR is https://github.com/Ryang20718/cflag/pull/28

i.e if my branch was branched off commit C whilst A is head, if I submit a pull request to run in github actions (for CI), we run on the merged commit.

A --> main head	B

C --> user/ryang/user_branch

Thus, the state of our branch is actually the following diagram.

A --> main head --> user/ryang/user_branch	B

This leads to a couple of unexpected bugs found in code coverage status checks. I.e because we run all of our tests on the merged commit, there may be a decent amount of files changed between commits C and A which we have inherited because we are running on the merged commit.

Thus, project level coverage may differ drastically even though the files we have touched have 100% patch coverage.

I have tried a couple of ways of bypassing this by overriding the commit to use the merged commit however, that doesn't work. If you manually rebase your PR and resubmit CI, then the base commit for the codecov report is accurate again. However, this is not a great developer experience as we'd like to make our codecov status checks blocking, but currently this is a bug

It seems like https://community.codecov.com/t/selected-base-commit-when-pr-is-based-on-an-older-commit-of-the-target-branch/4101 is a similar issue

I tried setting commit_parent in codecov-action similar to https://github.com/codecov/codecov-action/issues/597, but to no avail and also setting commit_parent to the HEAD of main, but to no avail even though main has a more recent commits with coverage

Logs from attempt to use parent_commit

=> Running git config --global --add safe.directory /home/runner/work/cflag/cflag
/usr/bin/git config --global --add safe.directory /home/runner/work/cflag/cflag
==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov --codecov-yml-path .codecov.yml -v create-commit'
/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov --codecov-yml-path .codecov.yml -v create-commit --parent-sha de19c68b335e6e17e1e5b3204935d3c5e5d5b716 --git-service github -C 37d7897188310ffc193ea3340b682505f9ca2db8
info - 2024-11-16 00:[48](https://github.com/Ryang20718/cflag/actions/runs/11865673830/job/33071259742#step:4:49):48,182 -- ci service found: github-actions
debug - 2024-11-16 00:48:48,185 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-11-16 00:48:48,185 -- Loading config from .codecov.yml
debug - 2024-11-16 00:48:48,188 -- Starting create commit process --- {"codecov_yml_path": ".codecov.yml", "verbose": true, "auto_load_params_from": null, "enterprise_url": null, "version": "cli-0.9.4", "command": "create-commit", "parent_sha": "de19c68b335e6e17e1e5b320[49](https://github.com/Ryang20718/cflag/actions/runs/11865673830/job/33071259742#step:4:50)35d3c5e5d5b716", "git_service": "github", "commit_sha": "37d7897188310ffc193ea3340b682505f9ca2db8", "pull_request_number": "28", "branch": "ryang/rebase", "fail_on_error": false, "slug": "Ryang20718/cflag"}
info - 2024-11-16 00:48:48,597 -- Process Commit creating complete
debug - 2024-11-16 00:48:48,597 -- Commit creating result --- {"result": "RequestResult(error=None, warnings=[], status_code=202, text='{\"status\":\"queued\"}\\n')"}
==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov --codecov-yml-path .codecov.yml -v create-report'
/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov --codecov-yml-path .codecov.yml -v create-report --git-service github -C 37d7897188310ffc193ea3340b682[50](https://github.com/Ryang20718/cflag/actions/runs/11865673830/job/33071259742#step:4:51)5f9ca2db8
info - 2024-11-16 00:48:49,317 -- ci service found: github-actions
debug - 2024-11-16 00:48:49,320 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-11-16 00:48:49,320 -- Loading config from .codecov.yml
debug - 2024-11-16 00:48:49,324 -- Starting create report process --- {"codecov_yml_path": ".codecov.yml", "verbose": true, "auto_load_params_from": null, "enterprise_url": null, "version": "cli-0.9.4", "command": "create-report", "git_service": "github", "commit_sha": "37d7897188310ffc193ea3340b682505f9ca2db8", "code": "default", "pull_request_number": "28", "fail_on_error": false, "slug": "Ryang20718/cflag"}
info - 2024-11-16 00:48:49,806 -- Process Report creating complete
debug - 2024-11-16 00:48:49,806 -- Report creating result --- {"result": "RequestResult(error=None, warnings=[], status_code=202, text='{\"status\":\"queued\"}\\n')"}
info - 2024-11-16 00:48:49,806 -- Finished creating report successfully --- {"response": "{\"status\":\"queued\"}\n"}
==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov --codecov-yml-path .codecov.yml -v do-upload'
/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov --codecov-yml-path .codecov.yml -v do-upload -f torchscript_input_files_coverage.dat -F torchscript_input_files --git-service github -C 37d7897188310ffc193ea3340b682505f9ca2db8
info - 2024-11-16 00:48:50,523 -- ci service found: github-actions
debug - 2024-11-16 00:48:50,526 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-11-16 00:48:50,526 -- Loading config from .codecov.yml
debug - 2024-11-16 00:48:50,530 -- Starting upload processing --- {"codecov_yml_path": ".codecov.yml", "verbose": true, "auto_load_params_from": null, "enterprise_url": null, "version": "cli-0.9.4", "command": "do-upload", "flags": ["torchscript_input_files"], "git_service": "github", "commit_sha": "37d7897188310ffc193ea3340b682505f9ca2db8", "report_code": "default", "network_root_folder": "/home/runner/work/cflag/cflag", "files_search_root_folder": "/home/runner/work/cflag/cflag", "files_search_exclude_folders": [], "disable_search": false, "disable_file_fixes": false, "build_code": "11865673830", "build_url": "https://github.com/Ryang20718/cflag/actions/runs/11865673830", "job_code": "ci", "name": "11865673830", "branch": "ryang/rebase", "pull_request_number": "28", "env_vars": {}, "plugin_names": ["xcode", "gcov", "pycoverage"], "dry_run": false, "use_legacy_uploader": false, "handle_no_reports_found": false, "report_type": "coverage", "network_filter": null, "network_prefix": null, "gcov_args": null, "gcov_ignore": null, "gcov_include": null, "gcov_executable": null, "swift_project": null, "fail_on_error": false, "slug": "Ryang20718/cflag"}
debug - 2024-11-16 00:48:50,530 -- Selected preparation plugins --- {"selected_plugins": ["<class 'codecov_cli.plugins.xcode.XcodePlugin'>", "<class 'codecov_cli.plugins.gcov.GcovPlugin'>", "<class 'codecov_cli.plugins.pycoverage.Pycoverage'>"], "cli_config": {}}
debug - 2024-11-16 00:48:50,530 -- Running preparation plugin: <class 'codecov_cli.plugins.xcode.XcodePlugin'>
debug - 2024-11-16 00:48:50,530 -- Running xcode plugin...
warning - 2024-11-16 00:48:50,531 -- xcrun is not installed or can't be found.
debug - 2024-11-16 00:48:50,531 -- Running preparation plugin: <class 'codecov_cli.plugins.gcov.GcovPlugin'>
debug - 2024-11-16 00:48:50,531 -- Running gcov plugin...
warning - 2024-11-16 00:48:50,532 -- No gcov data found.
debug - 2024-11-16 00:48:50,532 -- Running preparation plugin: <class 'codecov_cli.plugins.pycoverage.Pycoverage'>
warning - 2024-11-16 00:48:50,533 -- coverage.py is not installed or can't be found.
debug - 2024-11-16 00:48:50,533 -- Collecting relevant files
info - 2024-11-16 00:48:50,542 -- Found 2 coverage files to report
info - 2024-11-16 00:48:50,542 -- > /home/runner/work/cflag/cflag/onboard_av_binary_coverage.dat
info - 2024-11-16 00:48:50,542 -- > /home/runner/work/cflag/cflag/torchscript_input_files_coverage.dat
debug - 2024-11-16 00:48:50,545 -- Selected uploader to use: <class 'codecov_cli.services.upload.upload_sender.UploadSender'>
debug - 2024-11-16 00:48:50,546 -- Sending upload request to Codecov
info - 2024-11-16 00:48:50,983 -- Your upload is now processing. When finished, results will be available at: https://app.codecov.io/github/Ryang20718/cflag/commit/37d7897188310ffc193ea3340b682505f9ca2db8
debug - 2024-11-16 00:48:50,983 -- Upload request to Codecov complete. --- {"response": {"raw_upload_location": "https://storage.googleapis.com/codecov/shelter/github/Ryang20718%3A%3A%3A%3Acflag/37d7897188310ffc193ea3340b682505f9ca2db8/c39a133b-565c-46e3-b6bc-d590a640d796.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJWQHUGIBILH4J7Q6ZUSCIFNEOLYSNDS7L3B4N5SIBQ2J4YLYE5CRFCD%2F20241116%2Fus%2Fs3%2Faws4_request&X-Amz-Date=20241116T004850Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=abf385f33cff4a5bfb2bf19cacdd6f640cf9e1d629645a9bd3f65b1dea1ce3d5", "url": "https://app.codecov.io/github/Ryang20718/cflag/commit/37d7897188310ffc193ea3340b682505f9ca2db8"}}
debug - 2024-11-16 00:48:50,983 -- Sending upload to storage
info - 2024-11-16 00:48:[51](https://github.com/Ryang20718/cflag/actions/runs/11865673830/job/33071259742#step:4:52),164 -- Process Upload complete
debug - 2024-11-16 00:48:51,164 -- Upload result --- {"result": "RequestResult(error=None, warnings=[], status_code=200, text='')"}

codecov / codecov-cli

[bug][github actions] codecov base commit does not match the "merge commit" that the branch used to generate coverage #564