AttributeError: Attribute/Subscript/Name object has no attribute func

[simisimon]

Hi, the code line below causes an AttributeError in many ML projects that I analyze. Such projects are for example Stellargraph, seq2seq-EVC or, mmdetection

https://github.com/SMAT-Lab/Scalpel/blob/f09b868ff7762f14ba6bf9a589980f6b7532c714/scalpel/core/func_call_visitor.py#L27

Here are two a shortened stack trace for two file in the projects Stellargraph.

stellargraph/stellargraph/layer/hinsage.py:

Traceback (most recent call last):
  File "/home/simisimon/github/digital-bauhaus/CfgNet/src/cfgnet/plugins/source_code/ml_plugin.py", line 107, in _parse_config_file
    self.cfg = Cfg(code_str=code_str)
  File "/home/simisimon/github/digital-bauhaus/CfgNet/src/cfgnet/utility/cfg.py", line 26, in __init__
    self.cfg: CFG = CFGBuilder().build_from_src(name="", src=code_str)
  File "/home/simisimon/.cache/pypoetry/virtualenvs/cfgnet--7Rsn6Rj-py3.9/lib/python3.9/site-packages/scalpel/cfg/builder.py", line 128, in build_from_src
...
  File "/home/simisimon/.cache/pypoetry/virtualenvs/cfgnet--7Rsn6Rj-py3.9/lib/python3.9/site-packages/scalpel/core/func_call_visitor.py", line 92, in visit_Call
    call_info["params"] += [self.param2str(arg)]
  File "/home/simisimon/.cache/pypoetry/virtualenvs/cfgnet--7Rsn6Rj-py3.9/lib/python3.9/site-packages/scalpel/core/func_call_visitor.py", line 48, in param2str
    return get_func(param)
  File "/home/simisimon/.cache/pypoetry/virtualenvs/cfgnet--7Rsn6Rj-py3.9/lib/python3.9/site-packages/scalpel/core/func_call_visitor.py", line 34, in get_func
    mid = get_func(node.func.value)
  File "/home/simisimon/.cache/pypoetry/virtualenvs/cfgnet--7Rsn6Rj-py3.9/lib/python3.9/site-packages/scalpel/core/func_call_visitor.py", line 27, in get_func
    if type(node.func) is ast.Name:
AttributeError: 'Subscript' object has no attribute 'func'

stellargraph/stellargraph/layer/graphsage.py.

Traceback (most recent call last):
  File "/home/simisimon/github/digital-bauhaus/CfgNet/src/cfgnet/plugins/source_code/ml_plugin.py", line 107, in _parse_config_file
    self.cfg = Cfg(code_str=code_str)
  File "/home/simisimon/github/digital-bauhaus/CfgNet/src/cfgnet/utility/cfg.py", line 26, in __init__
    self.cfg: CFG = CFGBuilder().build_from_src(name="", src=code_str)
  File "/home/simisimon/.cache/pypoetry/virtualenvs/cfgnet--7Rsn6Rj-py3.9/lib/python3.9/site-packages/scalpel/cfg/builder.py", line 128, in build_from_src
   ...
  File "/home/simisimon/.cache/pypoetry/virtualenvs/cfgnet--7Rsn6Rj-py3.9/lib/python3.9/site-packages/scalpel/core/func_call_visitor.py", line 48, in param2str
    return get_func(param)
  File "/home/simisimon/.cache/pypoetry/virtualenvs/cfgnet--7Rsn6Rj-py3.9/lib/python3.9/site-packages/scalpel/core/func_call_visitor.py", line 34, in get_func
    mid = get_func(node.func.value)
  File "/home/simisimon/.cache/pypoetry/virtualenvs/cfgnet--7Rsn6Rj-py3.9/lib/python3.9/site-packages/scalpel/core/func_call_visitor.py", line 27, in get_func
    if type(node.func) is ast.Name:
AttributeError: 'Attribute' object has no attribute 'func'

[Jarvx]

Thanks for the issue. I will push a fix tomorrow.

[Jarvx]

Hi, the problem has been fixed. If you are interested, you can simply modify the test script here for your code base to test if there are still problems. Due to the complex syntax of the Python language, we are not able to test all the possible syntax sugars. Any test case will be appreciated.

[simisimon]

I found some other issues for which I would like to provide some test cases. However, I am not sure which test script you are referring to. Can you point me to the correct test script?

[Jarvx]

Here is the test script, you can simply add your own target dir, it will list all the source files

https://github.com/SMAT-Lab/Scalpel/blob/master/tests/test_cfg_robust.py

[simisimon]

Thanks, for pointing me to the corresponding test script. Since I'm not able to create an pull request, I will describe my changes here.

First, I added the following files into the directory tests/tests-cases/cfg_test_cases_issue45:

• create_elmo_embeddings_from_vocab.py
• generate_video.py
• action_selector.py
• modeling_utils.py

Building the CFG for these files, fails due to an exception raised in scalpel/core/func_call_visitor.py.

I also modified the test script as follows:

from scalpel.cfg.builder import CFGBuilder
from scalpel.util import get_path_by_ext
import os

def test_all():
    target_dir = "./test-cases/cfg_test_cases_issue45/"
    all_files = get_path_by_ext(target_dir)
    print(f"In total, there are {len(all_files)} files to be tested!")
    for fn in all_files:
        builder = CFGBuilder()
        try:
            cfg = builder.build_from_file(os.path.basename(fn), fn)
        except Exception as error:
            print(f"Failed to build CFG for {os.path.basename(fn)} due to {error}")

def main():
    test_all()

if __name__ == '__main__':
    main() 

[Jarvx]

@simisimon The bug has been fixed. But as call chains are really complex in Python language, we will just ignore many cases such as (X==Y).fun(). We are still working on a better algorithm for analysis. Please just give the link for projects or scripts that Scalpel still has trouble to deal with

Not sure about the problem with PR. I will make it available as soon as possible.
Thanks for your contribution!

[simisimon]

I totally understand your point. However, if such cases are ignored, the tool should at least not fail to build a CFG.

The bug still exists for me. With the latest version, Scalpel still fails to build the CFGs for the files mentioned above.

[Jarvx]

Okay, this is strange. It passed at my side. If you look at the source code Some changes have been made. Can you double check with latest version?

[simisimon]

I pulled the latest commit and run the test script again. It still fails. Did you add the mentioned files and then run test_cfg_robust.py?

If I run python tests/test_cfg_robust.py, I get the following output:

In total, there are 6 files to be tested!
Fail to build CFG for tests/test-cases/cfg_test_cases_issue45/joint-coref-srl-allennlp-allennlp-tools-create_elmo_embeddings_from_vocab.py due to <class 'ast.JoinedStr'>
Fail to build CFG for tests/test-cases/cfg_test_cases_issue45/maua-stylegan2-generate_video.py due to <class 'ast.IfExp'>
Fail to build CFG for tests/test-cases/cfg_test_cases_issue45/ROMA-src-components-action_selectors.py due to <class 'ast.Compare'>
Fail to build CFG for tests/test-cases/cfg_test_cases_issue45/rosita-pruning-hg_transformers-modelung_utils.py due to <class 'ast.UnaryOp'>

[Jarvx]

@simisimon Yes, I solved all of them as you can see from https://github.com/SMAT-Lab/Scalpel/blob/1022200043f2d9e8c24256821b863997ab34dd49/scalpel/core/func_call_visitor.py#L50

This is strange. Can you attach trace stack output?

[simisimon]

You're right, it is already fixed. It was a problem on my side. I pulled the the latest version, but I did not install it. I thought this was not necessary. Sorry for the confusion.

[Jarvx]

That's okay. I feel like the bugs like this will occur again when encountering more complex syntax sugar in a larger codebase. Please feel free to reopen it again as we are glad you can test it for us.