search

JelleZijlstra / autotyping

Automatically add simple type annotations to your code
241 stars 20 forks source link

Crashes on zip(* #57

Closed MarcoGorelli closed 1 year ago

MarcoGorelli commented 1 year ago

Here's an example:

(.venv) marcogorelli@DESKTOP-U8OKFP3:~/tmp$ cat t.py
def foo(iterables):
    zip(*(bar(it) for it in iterables))

(.venv) marcogorelli@DESKTOP-U8OKFP3:~/tmp$ pip freeze | grep -E 'autotyping|libcst'
autotyping==23.2.0
libcst==0.4.9
(.venv) marcogorelli@DESKTOP-U8OKFP3:~/tmp$ python --version
Python 3.8.16
(.venv) marcogorelli@DESKTOP-U8OKFP3:~/tmp$ python -m libcst.tool codemod autotyping.AutotypeCommand t.py --aggressive --no-format
Calculating full-repo metadata...
Executing codemod...
Failed to determine module name for /home/marcogorelli/tmp/t.py: '/home/marcogorelli/tmp/t.py' does not start with '/home/marcogorelli/tmp/src'
Codemodding /home/marcogorelli/tmp/t.py
Traceback (most recent call last):
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/codemod/_cli.py", line 279, in _execute_transform
    output_tree = transformer.transform_module(input_tree)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/codemod/_command.py", line 71, in transform_module
    tree = super().transform_module(tree)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/codemod/_codemod.py", line 108, in transform_module
    return self.transform_module_impl(tree_with_metadata)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/codemod/_visitor.py", line 32, in transform_module_impl
    return tree.visit(self)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/module.py", line 90, in visit
    result = super(Module, self).visit(visitor)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/base.py", line 228, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/module.py", line 74, in _visit_and_replace_children
    body=visit_body_sequence(self, "body", self.body, visitor),
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 227, in visit_body_sequence
    return tuple(visit_body_iterable(parent, fieldname, children, visitor))
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 193, in visit_body_iterable
    new_child = child.visit(visitor)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/base.py", line 228, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 1792, in _visit_and_replace_children
    params=visit_required(self, "params", self.params, visitor),
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 81, in visit_required
    result = node.visit(visitor)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/base.py", line 228, in visit
    _CSTNodeSelfT, self._visit_and_replace_children(visitor)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 1975, in _visit_and_replace_children
    params=visit_sequence(self, "params", self.params, visitor),
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 177, in visit_sequence
    return tuple(visit_iterable(parent, fieldname, children, visitor))
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/internal.py", line 159, in visit_iterable
    new_child = child.visit(visitor)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_nodes/base.py", line 237, in visit
    leave_result = visitor.on_leave(self, with_updated_children)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/matchers/_visitors.py", line 522, in on_leave
    retval = CSTTransformer.on_leave(self, original_node, updated_node)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/libcst/_visitors.py", line 71, in on_leave
    updated_node = leave_func(original_node, updated_node)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/autotyping/autotyping.py", line 562, in leave_Param
    guessed_type, containers = guess_type_from_argname(parameter_name)
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/autotyping/guess_type.py", line 90, in guess_type_from_argname
    elems, container = guess_type_from_argname(name[:-1])
  File "/home/marcogorelli/tmp/.venv/lib/python3.8/site-packages/autotyping/guess_type.py", line 50, in guess_type_from_argname
    if elems in names or (elems[-1] == "s" and elems[:-1] in names):
IndexError: string index out of range

Failed to codemod /home/marcogorelli/tmp/t.py

Finished codemodding 1 files!
 - Transformed 0 files successfully.
 - Skipped 0 files.
 - Failed to codemod 1 files.
 - 0 warnings were generated.

Noticed this when trying to upgrade autotyping in pandas

JelleZijlstra commented 1 year ago

Thanks for reporting and proposing a fix! I don't understand why we would be visiting a splat expression at all since it's not a function parameter.

I think the problem instead is that the parameter is called iterables. That means the container regex at https://github.com/JelleZijlstra/autotyping/blob/master/autotyping/guess_type.py#L17 will match with elems being the empty string. The solution should be to make elems always nonempty by using + instead of * in the two regexes in https://github.com/JelleZijlstra/autotyping/blob/master/autotyping/guess_type.py#L36.