Fail to parse single line string with comment

[nickdiego]

I'm using bashlex to parse build log files to extract compilation commands. I've just realized that when single line strings with comments are passed to the parser, it fails raising the exception below:

Traceback (most recent call last):
  File "/bin/compiledb", line 11, in <module>
    load_entry_point('compiledb', 'console_scripts', 'compiledb')()
  File "/usr/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 1043, in invoke
    return Command.invoke(self, ctx)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/nick/projects/compiledb/compiledb-generator/compiledb/cli.py", line 74, in cli
    done = generate(infile, outfile, build_dir, exclude_files, verbose, overwrite, not no_strict)
  File "/home/nick/projects/compiledb/compiledb-generator/compiledb/__init__.py", line 78, in generate
    r = generate_json_compdb(infile, proj_dir=build_dir, verbose=verbose, exclude_files=exclude_files)
  File "/home/nick/projects/compiledb/compiledb-generator/compiledb/__init__.py", line 34, in generate_json_compdb
    result = parse_build_log(instream, proj_dir, exclude_files, verbose)
  File "/home/nick/projects/compiledb/compiledb-generator/compiledb/parser.py", line 103, in parse_build_log
    commands = CommandProcessor.process(line, working_dir)
  File "/home/nick/projects/compiledb/compiledb-generator/compiledb/parser.py", line 163, in process
    trees = bashlex.parser.parse(line)
  File "/home/nick/sandbox/bashlex/bashlex/parser.py", line 611, in parse
    ef.visit(parts[-1])
  File "/home/nick/sandbox/bashlex/bashlex/ast.py", line 35, in visit
    k = n.kind
AttributeError: 'NoneType' object has no attribute 'kind'

Patch coming..

[samlikins]

The issue was not fully documented, so not certain what string was used which caused the issue. Based on the description, I've attempted to provide several iterations of comment strings.

In a Python interactive session with the following setup:

Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bashlex
>>> bashlex.parse('# comment')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python3.10/site-packages/bashlex/parser.py", line 620, in parse
    ef.visit(parts[-1])
  File "/home/user/.local/lib/python3.10/site-packages/bashlex/ast.py", line 38, in visit
    k = n.kind
AttributeError: 'str' object has no attribute 'kind'. Did you mean: 'find'?

A comment on it's own fails with this error, adding space before it fails also:

>>> bashlex.parse(' # comment')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python3.10/site-packages/bashlex/parser.py", line 620, in parse
    ef.visit(parts[-1])
  File "/home/user/.local/lib/python3.10/site-packages/bashlex/ast.py", line 38, in visit
    k = n.kind
AttributeError: 'str' object has no attribute 'kind'. Did you mean: 'find'?
>>> bashlex.parse('\t# comment')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python3.10/site-packages/bashlex/parser.py", line 620, in parse
    ef.visit(parts[-1])
  File "/home/user/.local/lib/python3.10/site-packages/bashlex/ast.py", line 38, in visit
    k = n.kind

Comments after commands don't fail:

>>> bashlex.parse('STRING="string with" # comment')
[CommandNode(parts=[AssignmentNode(parts=[] pos=(0, 20) word='STRING=string with')] pos=(0, 20))]

Comments starting on new lines after commands don't fail:

>>> bashlex.parse('STRING="string with"\n# comment')
[CommandNode(parts=[AssignmentNode(parts=[] pos=(0, 20) word='STRING=string with')] pos=(0, 20))]
>>> bashlex.parse('STRING="string with" \
... \
... # comment')
[CommandNode(parts=[AssignmentNode(parts=[] pos=(0, 20) word='STRING=string with')] pos=(0, 20))]