search

codecov / feedback

A place to discuss feedback about the pull request and web product experience.
37 stars 7 forks source link

[BUG][UX] Codecov web UI incorrectly claims that multiple flags is an error #567

Open webknjaz opened 2 weeks ago

webknjaz commented 2 weeks ago

Describe the bug

My typical setup is to add flags that correspond to the current runtime/OS/Python version and testing type (pytest vs. MyPy). The uploads are coming from multiple jobs in the matrix on each CI run. This helps me distinguish between what env/tool covered which lines.

Here's a message that shows up in red under each upload:

Multiple flags detected. Please ensure one flag per upload.

Both the action and cli still document this as supported: https://github.com/codecov/codecov-action#arguments / https://github.com/codecov/codecov-cli#do-upload.

The CI log shows that the invocation includes multiple -F options: https://github.com/ansible/awx-plugins/actions/runs/11672508220/job/32501323212?pr=52#step:23:61. This is no different from how it worked before. The change is in how the web service displays data now.

Environment (please complete the following information):

GHA but shouldn't be important.

To Reproduce

  1. Configure GHA to pass a list of a few comma-separated flags on upload.
  2. Run the respective workflow.
  3. Observe the list of uploads on the specific commit web page on Codecov filled with red error messages.

Expected behavior

There should not be any message implying that marking uploads with multiple flags is incorrect.

Screenshots

An example of this can be seen @ https://app.codecov.io/github/ansible/awx-plugins/commit/3cd6520e2e0fd211366a5b91d509577a5d588bc0.

Additional context

Specific configuration example: https://github.com/ansible/awx-plugins/blob/3cd6520/.github/workflows/reusable-tox.yml#L394-L405

drazisil-codecov commented 2 weeks ago

@codecovdesign see above ^

jagerber48 commented 1 week ago

I have this same use case. I test my code on various OS and python versions and tag each upload with two flags, one for OS and one for python version. The documentation warns against this: Image but I don't really understand the warning.

My expectation is that

jagerber48 commented 1 week ago

Actually, on nth thought, maybe this is a weird way to do things. Maybe it's weird to include all python versions under the "windows" flag. Probably better to make compound flags like windows-3.12 instead of double flags like windows and 3.12.