[BUG]: LOCAL LINT CODE NOT WORKING

[SomanathSankaran]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

I have a code base which I downloaded and try to run ucx lint-local-code

On running with debug mode it is failing in the pip install line

Databricks notebook source

!pip install fernet

COMMAND ----------

MAGIC %run ./commons

COMMAND ----------

Expected Behavior

It should be able to skip non python lines and parse remaining content

Steps To Reproduce

No response

Cloud

Azure

Operating System

macOS

Version

latest via Databricks CLI

Relevant log output

ERROR [src/databricks/labs/ucx.lint-local-code] Failed to call lint-local-code: Traceback (most recent call last):
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\state\venv\Lib\site-packages\astroid\builder.py", line 181, in _data_build
    node, parser_module = _parse_string(
                          ^^^^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\state\venv\Lib\site-packages\astroid\builder.py", line 480, in _parse_string
    parsed = parser_module.parse(
             ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\state\venv\Lib\site-packages\astroid\_ast.py", line 30, in parse
    return ast.parse(string, type_comments=type_comments)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<unknown>", line 1
    !pip install fernet
    ^
SyntaxError: invalid syntax

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\state\venv\Lib\site-packages\databricks\labs\blueprint\cli.py", line 109, in _route
    cmd.fn(**kwargs)
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\cli.py", line 489, in lint_local_code
    ctx.local_code_linter.lint(prompts, None if path is None else Path(path))
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 110, in lint
    located_advices = list(self.lint_path(path))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 127, in lint_path
    yield from self._lint_one(child_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 134, in _lint_one
    return [advice.for_path(path) for advice in linter.lint()]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 134, in <listcomp>
    return [advice.for_path(path) for advice in linter.lint()]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\notebooks\sources.py", line 192, in lint
    yield from self._lint_notebook()
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\notebooks\sources.py", line 218, in _lint_notebook
    yield from notebook_linter.lint()
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\notebooks\sources.py", line 102, in lint
    for advice in linter.lint(cell.original_code):
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\base.py", line 172, in lint
    yield from linter.lint(code)
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\dbfs.py", line 81, in lint
    tree = Tree.parse(code)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\python_ast.py", line 22, in parse
    root = parse(code)
           ^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\state\venv\Lib\site-packages\astroid\builder.py", line 303, in parse
    return builder.string_build(code, modname=module_name, path=path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\state\venv\Lib\site-packages\astroid\builder.py", line 151, in string_build
    module, builder = self._data_build(data, modname, path)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\somanath.sankara\.databricks\labs\ucx\state\venv\Lib\site-packages\astroid\builder.py", line 185, in _data_build
    raise AstroidSyntaxError(
astroid.exceptions.AstroidSyntaxError: Parsing Python code failed:

[asnare]

I think this has been fixed in #1905, which will be in the next release.

[SomanathSankaran]

hi @asnare but I am still wondering why a issue with a single code file is not allowing other files to run basically a logging mechanism to skip error files and complete the code fix for other code files will be a better solution Just for thought in-case if it is not approached like this in the fix as I could see an error in a single code files results in error for the entire process of 100 valid files