apache / submarine

Submarine is Cloud Native Machine Learning Platform.
https://submarine.apache.org/
Apache License 2.0
691 stars 254 forks source link

SUBMARINE-1365. Fix failure python unit test environment #1040

Closed FatalLin closed 1 year ago

FatalLin commented 1 year ago

What is this PR for?

https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929 we got following error in python unit test. [24|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:25]Collecting wheel [25|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:26] Downloading wheel-0.38.4-py3-none-any.whl (36 kB) [26|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:27]Installing collected packages: wheel, setuptools [27|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:28] Attempting uninstall: setuptools [28|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:29] Found existing installation: setuptools 65.5.0 [29|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:30] Uninstalling setuptools-65.5.0: [30|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:31] Successfully uninstalled setuptools-65.5.0 [31|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:32]Successfully installed setuptools-66.0.0 wheel-0.38.4 [32|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:33]Obtaining file:///home/runner/work/submarine/submarine/submarine-sdk/pysubmarine [33|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:34] Preparing metadata (setup.py): started [34|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:35] Preparing metadata (setup.py): finished with status 'error' [35|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:36] error: subprocess-exited-with-error [36|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:37] [37|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:38] × python setup.py egg_info did not run successfully. [38|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:39] │ exit code: 1 [39|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:40] ╰─> [27 lines of output] [40|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:41] /opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/dist.py:543: UserWarning: The version specified ('0.8.0-SNAPSHOT') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details. [41|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:42] warnings.warn( [42|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:43] running egg_info [43|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:44] Traceback (most recent call last): [44|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:45] File "<string>", line 2, in <module> [45|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:46] File "<pip-setuptools-caller>", line 34, in <module> [46|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:47] File "/home/runner/work/submarine/submarine/submarine-sdk/pysubmarine/setup.py", line 21, in <module> [47|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:48] setup( [48|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:49] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup [49|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:50] return distutils.core.setup(**attrs) [50|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:51] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup [51|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:52] return run_commands(dist) [52|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:53] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands [53|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:54] dist.run_commands() [54|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:55] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands [55|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:56] self.run_command(cmd) [56|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:57] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/dist.py", line 1208, in run_command [57|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:58] super().run_command(command) [58|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:59] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command [59|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:60] cmd_obj.ensure_finalized() [60|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:61] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized [61|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:62] self.finalize_options() [62|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:63] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 219, in finalize_options [63|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:64] parsed_version = parse_version(self.egg_version) [64|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:65] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/version.py", line 266, in __init__ [65|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:66] raise InvalidVersion(f"Invalid version: '\{version}'") [66|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:67] pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '0.8.0-SNAPSHOT' [67|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:68] [end of output] [68|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:69] [69|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:70] note: This error originates from a subprocess, and is likely not a problem with pip. [70|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:71]error: metadata-generation-failed [71|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:72] [72|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:73]× Encountered error while generating package metadata. [73|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:74]╰─> See above for output. I found a similar issue and it looks like there's some trouble on setuptools.

https://github.com/pypa/setuptools/issues/3772

I'd like to take their adoption to fix the setuptools to 65.7.0 as a workaround here.

What type of PR is it?

Hot fix

Todos

N/A

What is the Jira issue?

https://issues.apache.org/jira/browse/SUBMARINE-1365

How should this be tested?

should run python unit test as expected.

Screenshots (if appropriate)

Questions:

codecov[bot] commented 1 year ago

Codecov Report

Merging #1040 (e2fe9b5) into master (a7e2c54) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1040   +/-   ##
=======================================
  Coverage   75.98%   75.98%           
=======================================
  Files         119      119           
  Lines        5000     5000           
=======================================
  Hits         3799     3799           
  Misses       1201     1201           
Flag Coverage Δ
python-integration 59.72% <ø> (ø)
python-unit 52.48% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more