search

palantir / pyspark-style-guide

This is a guide to PySpark code style presenting common situations and the associated best practices based on the most frequent recurring topics across the PySpark repos we've encountered.
MIT License
1.02k stars 131 forks source link

Pylint checkers are not working (basestring is not defined) #9

Open robertkossendey opened 3 years ago

robertkossendey commented 3 years ago

pylint 2.7.2 astroid 2.5.1 Python 3.9.2 (default, Feb 24 2021, 13:26:01)

When running pylint with those checkers enabled and with a file that has a unnecessary split statement, I encounter this error. It seems that the pylint_utils is currently bugged.

Stacktrace:

export PYTHONPATH="$PWD/pylint/palantir-pyspark" && pylint src/ --load-plugins=palantir_module

Traceback (most recent call last):
  File "/usr/local/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/usr/local/lib/python3.9/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "/usr/local/lib/python3.9/site-packages/pylint/lint/run.py", line 358, in __init__
    linter.check(args)
  File "/usr/local/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 862, in check
    self._check_files(
  File "/usr/local/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 896, in _check_files
    self._check_file(get_ast, check_astroid_module, name, filepath, modname)
  File "/usr/local/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 922, in _check_file
    check_astroid_module(ast_node)
  File "/usr/local/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1054, in check_astroid_module
    retval = self._check_astroid_module(
  File "/usr/local/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1099, in _check_astroid_module
    walker.walk(ast_node)
  File "/usr/local/lib/python3.9/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    self.walk(child)
  File "/usr/local/lib/python3.9/site-packages/pylint/utils/ast_walker.py", line 72, in walk
    callback(astroid)
  File "/WebstormProjects/data-infrastructure/pylint/palantir-pyspark/statement_call_checker.py", line 48, in visit_assign
    if get_length(node) + base_length <= 120:
  File "/WebstormProjects/data-infrastructure/pylint/palantir-pyspark/pylint_utils.py", line 104, in get_length
    value_length = get_length(arg.value)
  File "/WebstormProjects/data-infrastructure/pylint/palantir-pyspark/pylint_utils.py", line 89, in get_length
    compute_arguments_length(arg.args) + base_length
  File "/WebstormProjects/data-infrastructure/pylint/palantir-pyspark/pylint_utils.py", line 13, in compute_arguments_length
    args_length += get_length(arg)
  File "/WebstormProjects/data-infrastructure/pylint/palantir-pyspark/pylint_utils.py", line 77, in get_length
    if isinstance(arg.pytype(), basestring):
NameError: name 'basestring' is not defined
SebasDrewes commented 2 years ago

Basestring is no longer available in Python 3. From What’s New In Python 3.0:

The builtin basestring abstract type was removed. They should change basestring for str in pylint_utils file.