oracle / graal

GraalVM compiles Java applications into native executables that start instantly, scale fast, and use fewer compute resources 🚀
https://www.graalvm.org
Other
20.36k stars 1.64k forks source link

Upgrade to python3 for stylecheck in Github Actions #3802

Open SwapnilGaikwad opened 3 years ago

SwapnilGaikwad commented 3 years ago

Describe your environment :

Describe the issue

  1. Github Actions specify MX_PYTHON_VERSION=3 however, the python packages used for stylechecks, e.g., pylint, are using python2.7. Graal is using pylint 1.9.3 at the moment that fails when installed with python3. Therefore, while switching to python3 fully in GitHub Actions, we will need to update pylint to newer version.
  2. Switching to newer version of pylint 2.2.0 also fails with AttributeError: 'Import' object has no attribute 'infer_name_module' error.
  3. Switching to pylint 2.4.0 works but flags two styling issues in compiler/mx.compiler/mx_compiler.py.

Code snippet or code repository that reproduces the issue

  1. Run pylint 1.9.3 with python 3.8.10
    Output
$ curl https://bootstrap.pypa.io/pip/get-pip.py --output get-pip.py
$ python3 get-pip.py
$ pip3 install pylint==1.9.3
$ pylint --version
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.8/site-packages/astroid/astpeephole.py", line 17, in <module>
    _TYPES = (_ast.Str, _ast.Bytes)
AttributeError: module '_ast' has no attribute 'Str'

During handling of the above exception, another exception occurred:

...
  File "/home/user/.local/lib/python3.8/site-packages/astroid/astpeephole.py", line 19, in <module>
    _TYPES = (_ast.Str, )
AttributeError: module '_ast' has no attribute 'Str'

  1. Run with pylint 2.2.0 with python 3.8.10
    Output
$ pip3 install pylint==2.2.0
$ mx -v --primary-suite-path $(pwd)/compiler --java-home=/usr/lib/jvm/openjdk-8u292+09-jvmci-21.1-b04 --J @ pylint --primary
Traceback (most recent call last):
  File "/home/user/.local/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
...
  File "/home/user/.local/lib/python3.8/site-packages/pylint/checkers/variables.py", line 1582, in visit_import
    module = next(node.infer_name_module(parts[0]))
AttributeError: 'Import' object has no attribute 'infer_name_module'
[exit code: 1]
...

  1. Run with pylint 2.2.4 with python 3.8.10
    Output
...
************* Module mx_compiler
compiler/mx.compiler/mx_compiler.py:818:8: E0702: Raising NoneType while only classes or instances are allowed (raising-bad-type)
compiler/mx.compiler/mx_compiler.py:820:8: E0702: Raising NoneType while only classes or instances are allowed (raising-bad-type)
[exit code: 2]
...

Expected behavior It would be helpful if Graal uses python3 throughout its CI system including mx & the gates it executes. Current GitHub actions extract python package dependencies from mx/common.json. Therefore, switching to python3 may require updates to both mx and Graal in sync to avoid CI failures.

munishchouhan commented 3 years ago

@SwapnilGaikwad thanks for reporting the issue @dougxc please advise

dougxc commented 3 years ago

@sanzinger is this something already being taking care of with the work to migrate mx and Graal to python 3?

rschatz commented 3 years ago

FWIW, I recently changed some of our internal gates to use python3 for pylint, and it seems to work fine with the current version. Not sure what exact python version we're using there, it might be older than 3.8.10.

See https://github.com/oracle/graal/commit/e90430bfc11dd77ac718931fc6215fb95333292e for the change. Might make sense to do this for all our gates, including the GH actions ones.

I just had to explicitly install isort from pip to work around https://bugzilla.redhat.com/show_bug.cgi?id=1710221, which we seem to be hitting at least in some of our internal CI machines.

That said, I also wouldn't be opposed to updating to a newer pylint version ;)