Proper yaml setup to skip files in coverage testing

[glwhart]

I've tried multiple times to set my yaml file for github actions to skip coverage testing for some files, but nothing I have tried works. paths-ignore early in the yaml file doesn't change what actually happens, and putting it later results in a syntax error. Here is my current file with two added lines, commented to show what I'm trying to do:

name: Runtests
on: [push, pull_request]
# needed to allow julia-actions/cache to delete old caches that it has created
permissions: 
  actions: write
  contents: read
jobs: 
  test: 
    runs-on: ${{ matrix.os }}
    strategy:
      matrix: 
        julia-version: ['1.9']
        julia-arch: [x64]
        #os: [ubuntu-latest, macOS-latest, windows-latest]
        os: [macOS-latest]
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@latest
        with: 
          version: ${{ matrix.julia-version }}
      - uses: julia-actions/cache@v2
      - run: |
          julia -e '
            import Pkg;
            Pkg.activate(".");  # Activate the project environment
            # Not sure why removing and reinstalling the SNF package is necessary, but it is
            Pkg.rm("SmithNormalForm");  # Remove any existing installation of SNFer
            Pkg.add(url="https://github.com/glwhart/Spacey.jl"); # Re-add Spacey from the URL
            Pkg.add(url="https://github.com/wildart/SmithNormalForm.jl");  # Add the unregistered SNF package
          '
      - uses: julia-actions/julia-buildpkg@latest
      - uses: julia-actions/julia-runtest@latest
      - uses: julia-actions/julia-processcoverage@v1
        paths-ignore: # These are the two lines showing what I want to do, but this syntax doesn't work
        - "src/proto*.jl"
      - uses: codecov/codecov-action@v4
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}      

Any advice or help would be greatly appreciated. I've read the docs extensively, and tried myriad things, but without success.

[glwhart]

Adding a .codecov.yml file (as below), doesn't affect what the workflow specified by the CI.yml file does. I got additional reports now, but they don't show that the coverage is skipping the specified files.

coverage:
    ignore:
      - "src/proto*.jl"  

and adding these commands to the CI.yml file just gives me syntax errors...

[drazisil-codecov]

Hi @glwhart . To clarify , you are looking to exclude Codecov from reporting coverage from these files, or only run Codecov in certain cases?

[glwhart]

Just exclude certain files from being included when coverage is computed

[drazisil-codecov]

@glwhart

Then ignore in your codecov.yml should be what you are looking for. Can you share a SHA where that didn't work and I can take a look?

coverage:
    ignore:
      - "src/proto*.jl"  

[glwhart]

From the codecov report here: https://app.codecov.io/gh/glwhart/JuCE.jl/tree/main/src You can see that 48 lines from the prototype3_correlations.jl file are being included in the report as missing lines even though the .codecov.yml file said to ignore that file...

The actual repo JuCE is private until the paper is ready to be submitted, but I've added you as a collaborator so you can take a look.

[glwhart]

Here is the last CI workflow run: https://github.com/glwhart/JuCE.jl/actions/runs/10569614625/job/29282732441

[drazisil-codecov]

@thomasrockhu-codecov can you help here? Or @rohan-at-sentry ?

[drazisil-codecov]

@glwhart I will be out of office for the next 7 days or so. I've left you in good hands.

[glwhart]

Thanks! I've invited both as collaborators on JuCE.jl repo

[rohan-at-sentry]

@glwhart I'm looping in @calvin-codecov and @giovanni-guidini to help. Can you invite them into the repo as well so they can take a look?

[glwhart]

Both have been invited as collaborators.

[giovanni-guidini]

Hello @glwhart , thanks for the invite.

I reviewed the CI run and also our logs for the commit. I noticed that we have We were not able to find the yaml on the provider API as one of the logs.

This indicates that the codecov.yaml was not found, so any configuration you added there would not have been picked up (particularly the ignore paths).

Could you try moving the codecov.yml file to the root of your repository?

UPDATE Also looking at the ignore docs those paths should be placed in the root level of the YAML, so something like:

ignore:
  - "src/proto*.jl"

[glwhart]

I've tried a lot of things---the github actions docs and codecov docs don't seem consistent. Based on your advice right here, I did the following: 1) renamed .codecov.yml to codecov.yml 2) and put a copy both in .github/workflows directory (their docs) and in the repo root directory (your suggestion) 3) Edited codecov.yml to match your suggestion (removed coverage:)

On the latest commit, the coverage % report changed and the three src/proto*.jl files are not in the list! Yay!

[glwhart]

I deleted the codecov.yml from the .github/workflows folder and the ignore list still worked, so the yml file in the root of the repo must be the one that is actually getting used.

I'll write up the successful setup and post it on discourse.julialang.org so others can benefit from your help as well. Thanks so much for helping me sort that out.

[drazisil-codecov]

Ah. It will work in .github, not workflows,

https://docs.codecov.com/docs/codecov-yaml#can-i-name-the-file-codecovyml

We should make ir clearer that does not include subdirectories, @rohan-at-sentry