search

StanfordBDHG / .github

Collection of default community health files, GitHub Action workflows, templates
https://biodesigndigitalhealth.stanford.edu
3 stars 2 forks source link

Upgrade to new CodeCov action #65

Open Supereg opened 6 months ago

Supereg commented 6 months ago

Problem

We are currently using the v3 version of the codecov action. A newer release v4 removed a feature that we relied on, namely converting .xcresult coverage format to one that codecov understands. This was removed without a proper replacement. We have reported our situation here: https://github.com/codecov/codecov-action/issues/1367. Our motivation to upgrade to v4 is mainly driven by the fact that tokenless uploads were randomly broken in v3.

Their new approach uses llvm-cov which uses .profdata files sourced from the derived data folder (the path is hardcoded to ~/Library/Developer/Xcode/DerivedData), see https://github.com/codecov/uploader/pull/968/files. They scan for all .app, .framework and .test folders in the derived data subfolders.

Solution

There are two ways forward here:

We could use a tool like xcresultparser (available via brew) to do the conversion ourself into the Cobertura XML format. Drawback would be that this introduces yet another tool, not a 100% conversion (authors note Note that some data in this file is currently fake) and relying on the xccov format which might randomly break between Xcode releases (seemingly also the reason why there is not a large adoption for that?). Another approach could be to execute the xcrun llvm-cov ... commands ourself?

If we wanted to adapt our infrastructure for the new codecov infrastructure. We would need to make sure that the necessary profdata files are uploaded as artifacts (such that the collect coverage reports action can use it). We are probably fine uploading a .xctest archive (not sure how it differs from .xcresult).

Additional context

Related PRs around cutest coverage conversion:

Code of Conduct

PSchmiedmayer commented 6 months ago

Thank you @Supereg for documenting this. My gut assumption is that running xcrun llvm-cov ... might be the best approach here ... might require a small script that does that.

In the end it might just be a thing that needs to be fixed as part of your bug report in https://github.com/codecov/codecov-action/issues/1367