codecov / codecov-ats

GitHub Action that uploads returns selected test labels to CI ☂️
MIT License
3 stars 3 forks source link

Label Anaysis doesn't use the Python installed via mamba-org/setup-micromamba in GitHub Actions #18

Closed seisman closed 4 months ago

seisman commented 1 year ago

Our project is trying to use the codecov/ats action in https://github.com/GenericMappingTools/pygmt/pull/2792, but it doesn't work as expected.

The codecov/ats actions produces the following errors (https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792):

Run codecov/codecov-ats@v0
  with:
  env:
    OS: ubuntu-latest
    PYTHON: 3.9
    NUMPY: 1.[2](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:2)2
    MAMBA_ROOT_PREFIX: /home/runner/micromamba
    MAMBA_EXE: /home/runner/micromamba-bin/micromamba
    CONDARC: /home/runner/micromamba-bin/.condarc
    CODECOV_STATIC_TOKEN: ***
    CODECOV_TOKEN: ***
==> linux OS detected
https://cli.codecov.io/latest/linux/codecov.SHA256SUM
==> SHASUM file signed by key id 806bb28aed779869
==> CLI SHASUM verified (fc46f8c696a28[3](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:3)8d0d2c88c[4](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:4)93daf9a40479b[5](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:5)f90[6](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:6)5fa0b5502[7](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:7)fa9407311aa1  codecov)
==> Running version latest
==> Running version v0.4.1
==> Running command '/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov create-commit'
/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov create-commit -C de621613[8](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:8)447[9](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:9)f54df1c69bc69436c6bc4705215
info - 2023-11-03 02:58:15,[10](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:10)4 -- ci service found: github-actions
info - 2023-[11](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:11)-03 02:58:15,305 -- Process Commit creating complete
==> Running command '/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov create-report'
/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov create-report -C de62161384479f54df1c69bc69436c6bc4705215
info - 2023-11-03 02:58:16,086 -- ci service found: github-actions
info - 2023-11-03 02:58:16,292 -- Process Report creating complete
info - 2023-11-03 02:58:16,292 -- Finished creating report successfully --- {"response": "{\"external_id\":\"0eaeb298-6516-427d-a544-b2290e7c955c\",\"created_at\":\"2023-11-03T02:58:16.240066Z\",\"commit_sha\":\"de62161384479f54df1c69bc69436c6bc4705215\",\"code\":null}"}
==> Running command '/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov static-analysis'
/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov static-analysis --commit-sha de62161384479f54df1c69bc69436c6bc4705215
info - 2023-11-03 02:58:17,053 -- ci service found: github-actions
info - 2023-11-03 02:58:17,071 -- Running the analyzer on 1049 files
Analyzing files
info - 2023-11-03 02:58:17,891 -- All files have been processed
info - 2023-11-03 02:58:17,891 -- Sending files fingerprints to Codecov --- {"files_effectively_analyzed": 280}
info - 2023-11-03 02:58:18,573 -- Received response from server --- {"time_taken": 0.608914}
info - 2023-11-03 02:58:18,573 -- 280 files VALID; 0 files CREATED
info - 2023-11-03 02:58:18,573 -- All files are already uploaded!
info - 2023-11-03 02:58:18,744 -- Received response with status code 204 from server --- {"time_taken": 0.16841}
Parent commit: d6b6d032f392646a9546bdf7b4932ee6c948a6ba
PRBase commit: f079866a379261a6b23ee903f11d0ae54904e76a
d6b6d032f392646a9546bdf7b4932ee6c948a6ba,f079866a379261a6b23ee903f11d0ae54904e76a
Attempting label analysis on d6b6d032f392646a9546bdf7b4932ee6c948a6ba
==> Running command '/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov label-analysis'
/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov label-analysis --dry-run --dry-run-format space-separated-list --head-sha de62161384479f54df1c69bc69436c6bc4705215 --base-sha d6b6d032f392646a9546bdf7b4932ee6c948a6ba
info - 2023-11-03 02:58:19,488 -- ci service found: github-actions
info - 2023-11-03 02:58:19,491 -- Requesting set of labels to run... --- {"with_labels": false}
info - 2023-11-03 02:58:19,691 -- Label Analysis request successful --- {"request_id": "9dab017d-6c29-428d-9025-384c7062a78e"}
info - 2023-11-03 02:58:19,692 -- Collecting labels...
info - 2023-11-03 02:58:19,693 -- Collecting tests --- {"pytest_command": ["python", "-m", "pytest"], "pytest_options": ["-q", "--collect-only"]}
Error: Pytest exited with non-zero code 1.
This is likely not a problem with label-analysis. Check pytest's output and options.
PYTEST OUTPUT:
/usr/bin/python: No module named pytest

Warning: Codecov: Failed to properly retrieve labels: The process '/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov' failed with exit code 1
Attempting label analysis on f079866a379261a6b23ee903f11d0ae54904e76a
==> Running command '/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov label-analysis'
/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov label-analysis --dry-run --dry-run-format space-separated-list --head-sha de62161384479f54df1c69bc69436c6bc4705215 --base-sha f079866a379261a6b23ee903f11d0ae54904e76a
info - 2023-11-03 02:58:20,434 -- ci service found: github-actions
info - 2023-11-03 02:58:20,437 -- Requesting set of labels to run... --- {"with_labels": false}
warning - 2023-11-03 02:58:20,647 -- Got a 4XX status code back from Codecov --- {"status_code": 400, "response_json": {"base_commit": ["No possible commits have static analysis sent. Attempted too many commits: f079866a379261a6b23ee903f11d0ae54904e76a,728c9166d6825a13c5bfd31ea6c1d8a6daa27c[12](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:13),90a12077d8082da31bb45fc0520bc05d745469[13](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:14),2be332dad1cb049134be591bc84cc4d62d89853d,dfe25544df5132d91b515af01cc8e15513c83e42,69b3cd717a210d6eb4dfabe3b443ec22ec401085,58efca053c1d97306c24dc37f849b1a37a45192e,6aa3ff87b99b3ec55495e9947647db369a5919a3,a9bd688f2d56f50126fd6bb6fe3ed57b86537956,c42def[14](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:15)5c994e48545ebfd3[19](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:20)64b6a68c1492f8"]}}
Error: There is some problem with the submitted information
Warning: Codecov: Failed to properly retrieve labels: The process '/home/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov' failed with exit code 1
Codecov: Could not find labels from commits: d6b6d032f392646a9546bdf7b4932ee6c948a6ba,f079866a379261a6b[23](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:24)ee903f11d0ae5[49](https://github.com/GenericMappingTools/pygmt/actions/runs/6740742232/job/18324277077?pr=2792#step:8:50)04e76a Defaulting to run all tests.

We install Python and all dependencies (including pytest-cov) using the setup-micromamba action, but it seems codecov lable-analysis call the system one /usr/bin/python.

thomasrockhu-codecov commented 1 year ago

@seisman thanks! Going to take a look into this, I suspect moving to a composite will help solve this

thomasrockhu-codecov commented 1 year ago

@seisman, so we released 0.2.5 which is a composite action. Would you be able to see if that works for you?

seisman commented 1 year ago

It still doesn't work. Now the error messages are:

Running ATS Action version: 0.2.5
==> Token of length 36 detected
==> Static token of length 40 detected
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting codecov-cli
  Downloading codecov-cli-0.3.3.tar.gz (269 kB)
    ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3s/vfzpb5r51gs6y328rmlgzm7c0000gn/T/pip-install-bPb9_m/codecov-cli/setup.py'"'"'; __file__='"'"'/private/var/folders/3s/vfzpb5r51gs6y328rmlgzm7c0000gn/T/pip-install-bPb9_m/codecov-cli/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/3s/vfzpb5r51gs6y328rmlgzm7c0000gn/T/pip-pip-egg-info-UorflE
         cwd: /private/var/folders/3s/vfzpb5r51gs6y328rmlgzm7c0000gn/T/pip-install-bPb9_m/codecov-cli/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/3s/vfzpb5r51gs6y328rmlgzm7c0000gn/T/pip-install-bPb9_m/codecov-cli/setup.py", line 8, in <module>
        with open(path.join(here, "README.md"), encoding="utf-8") as f:
    TypeError: 'encoding' is an invalid keyword argument for this function
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
/Users/runner/work/_actions/codecov/codecov-ats/v0/dist/codecov_ats.sh: line 73: codecovcli: command not found
Codecov: Failed to properly create commit
Error: Process completed with exit code 1.
thomasrockhu-codecov commented 1 year ago

@seisman are you using python2? Or is this aliasing from somewhere else? Codecov CLI requires python3

thomasrockhu-codecov commented 12 months ago

Closing this out as the CLI does not support python2

seisman commented 12 months ago

Sorry for the late response. The real issue is that, we install Python and its dependencies via the setup-micromamba action, and run our tests using this Python interpreter. However, codecov-ats action uses the system one, which is Python 3 on Linux and Python 2 on macOS.

rohan-at-sentry commented 11 months ago

@seisman we're looking into this. Given that the action is running in it's own shell we suspect we're defaulting to the system python instead of the micromamba one. We suspect that to be the issue.

I'll have an update next week

thomasrockhu-codecov commented 11 months ago

@seisman, I'm having a hard time discovering the reason here. Sometimes, it runs python2, sometimes it runs python3. It doesn't seem particularly consistent.

You can see in this run, that the macos 3.9 run seems to be in python2 here.

However, this is not true of other builds in the run, and it's also not true in other runs of the workflow. Do you have any idea why this might be?

seisman commented 11 months ago

Sometimes, it runs python2, sometimes it runs python3.

I think that's because the Linux system provides Python 3 while macOS provides Python 2 by defaults.

in the Linux run, the following message shows that codecov-ats still uses the system Python, rather than /home/runner/micromamba/envs/pygmt/bin/python:

Requirement already satisfied: idna in /usr/lib/python3/dist-packages (from rfc3986[idna2008]<2,>=1.3->httpx==0.23.*->codecov-cli) (3.3)
seisman commented 11 months ago

Given that the action is running in it's own shell we suspect we're defaulting to the system python instead of the micromamba one.

I'm wondering if it's easier if the codecov-ats action can accept an option like PYTHON_EXE_PATH. If it's undefined, then use the system one or the first one found in the PATH. If defined (in my case, PYTHON_EXE_PATH=/home/runner/micromamba/envs/pygmt/bin/python), then use the one specified by the user.

giovanni-guidini commented 11 months ago

@seisman that is a good idea. We'll consider that, thanks for the suggestion.

You can kinda achieve the same thing by changing the configuration for the CLI (i.e. bypassing the action). The full documentation is here, but let me give the TL;DR

The step failing is label-analysis, and that runs using the PytestStandardRunner. You can configure the runner with the pytest command to use.

To do that you can add to your codecov.yml the following:

cli:
  runners:
    pytest:
      pytest_command: ["/home/runner/micromamba/envs/pygmt/bin/python", "-m", "pytest"]

This will instruct the runner to use the appropriate Python interpreter. But I agree that it's not as clean as adding that to the action, so let's call it a workaround for now 😅

rohan-at-sentry commented 11 months ago

@seisman did this work for you?

seisman commented 11 months ago

The path /home/runner/micromamba/envs/pygmt/bin/python is system-dependent, so it may work for Linux, but not on macOS/Windows.

rohan-at-sentry commented 11 months ago

@seisman thanks for your patience here. This is something we're looking to tackle imminently to help unblock you. Here's the issue https://github.com/codecov/engineering-team/issues/407

Feel free to weigh in on the approach described. Once this ships, I'll reach out again

thomasrockhu-codecov commented 4 months ago

@seisman as it's been some time, I'm not sure the current state of your usage. I believe a fix may have gone in, but I will close this issue for now. Please re-open if you are still running into problems.