SMAT-Lab / Scalpel

Scalpel: The Python Static Analysis Framework
Apache License 2.0
278 stars 42 forks source link

how run the tool #31

Closed alaalial closed 2 years ago

alaalial commented 2 years ago

I cannot install the tool for type inference I use pip install I got error Using cached scalpel-0.8.2.tar.gz (96 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [8 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "C:\Users\00\AppData\Local\Temp\pip-install-ussx3gzj\scalpel_0c4ee3a7afd74153a91022a0f8e95f23\setup.py", line 4, in import scalpel File "C:\Users\00\AppData\Local\Temp\pip-install-ussx3gzj\scalpel_0c4ee3a7afd74153a91022a0f8e95f23\scalpel__init.py", line 36, in from constants import version ImportError: cannot import name 'version__' from 'constants' (C:\Users\00\AppData\Local\Programs\Python\Python39\lib\site-packages\constants.py) [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

After that I download the source code and I also got error Traceback (most recent call last): File "C:\Users\00\Desktop\Hityper\Scalpel-master\Scalpel-master\ex.py", line 1, in from scalpel.typeinfer.typeinfer import TypeInference File "C:\Users\00\Desktop\Hityper\Scalpel-master\Scalpel-master\scalpel\typeinfer\typeinfer.py", line 14, in from scalpel.typeinfer.analysers import ( File "C:\Users\00\Desktop\Hityper\Scalpel-master\Scalpel-master\scalpel\typeinfer\analysers.py", line 9, in from typed_ast import ast3 ModuleNotFoundError: No module named 'typed_ast'

khatchad commented 2 years ago

I think we need either python setup.py build or sudo python setup.py install but, for the latter, I am receiving an error about nbconvert. I am on be392671a5f55f5da8b2ca0a079207c55261ab03:

Installed /usr/local/lib/python3.10/dist-packages/nbformat-5.4.0-py3.10.egg
Searching for nbconvert
Reading https://pypi.org/simple/nbconvert/
Downloading https://files.pythonhosted.org/packages/4c/37/e28dc270697710eb7bbb66620652e3242c538ff4e56ff61378a1f4f8d5cb/nbconvert-7.0.0rc0-py3-none-any.whl#sha256=adaa1e94e2a398dd7f1bc99b4c75ba189766318a33e16318e2a9efec0c4e3c40
Best match: nbconvert 7.0.0rc0
Processing nbconvert-7.0.0rc0-py3-none-any.whl
Installing nbconvert-7.0.0rc0-py3-none-any.whl to /usr/local/lib/python3.10/dist-packages
Traceback (most recent call last):
  File "/home/rk1424/git/Scalpel/./setup.py", line 2, in <module>
    setup( 
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 124, in do_egg_install
    cmd.run(show_deprecation=False)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 439, in run
    self.easy_install(spec, not self.no_deps)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 681, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 728, in install_item
    self.process_distribution(spec, dist, deps)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 775, in process_distribution
    distros = WorkingSet([]).resolve(
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 771, in resolve
    dist = best[req.key] = env.best_match(
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1056, in best_match
    return self.obtain(req, installer)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1068, in obtain
    return installer(requirement)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 700, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 726, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 886, in install_eggs
    return [install_dist(dist_filename, tmpdir)]
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1101, in install_wheel
    self.execute(
  File "/usr/lib/python3.10/distutils/cmd.py", line 335, in execute
    util.execute(func, args, msg, dry_run=self.dry_run)
  File "/usr/lib/python3.10/distutils/util.py", line 305, in execute
    func(*args)
  File "/usr/lib/python3/dist-packages/setuptools/wheel.py", line 95, in install_as_egg
    self._install_as_egg(destination_eggdir, zf)
  File "/usr/lib/python3/dist-packages/setuptools/wheel.py", line 103, in _install_as_egg
    self._convert_metadata(zf, destination_eggdir, dist_info, egg_info)
  File "/usr/lib/python3/dist-packages/setuptools/wheel.py", line 114, in _convert_metadata
    wheel_metadata = get_metadata('WHEEL')
  File "/usr/lib/python3/dist-packages/setuptools/wheel.py", line 110, in get_metadata
    with zf.open(posixpath.join(dist_info, name)) as fp:
  File "/usr/lib/python3.10/zipfile.py", line 1512, in open
    zinfo = self.getinfo(name)
  File "/usr/lib/python3.10/zipfile.py", line 1439, in getinfo
    raise KeyError(
KeyError: "There is no item named 'nbconvert/tests/exporter_entrypoint/eptest-0.1.dist-info/WHEEL' in the archive"
billquan commented 2 years ago

Hi, Could you please provide your execution environment information? A simple python -m pip install . at the root directory of the project should do the job.

khatchad commented 2 years ago
Processing /home/rk1424/git/Scalpel
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: astor~=0.8.1 in /usr/local/lib/python3.10/dist-packages (from Scalpel==1.0b0) (0.8.1)
Requirement already satisfied: astunparse~=1.6.3 in /usr/local/lib/python3.10/dist-packages (from Scalpel==1.0b0) (1.6.3)
Requirement already satisfied: dataclasses in /usr/local/lib/python3.10/dist-packages (from Scalpel==1.0b0) (0.6)
Requirement already satisfied: graphviz~=0.17 in /usr/local/lib/python3.10/dist-packages (from Scalpel==1.0b0) (0.20)
Requirement already satisfied: nbconvert in /usr/local/lib/python3.10/dist-packages (from Scalpel==1.0b0) (6.5.0)
Requirement already satisfied: nbformat in /usr/local/lib/python3.10/dist-packages/nbformat-5.4.0-py3.10.egg (from Scalpel==1.0b0) (5.4.0)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages/networkx-2.8-py3.10.egg (from Scalpel==1.0b0) (2.8)
Requirement already satisfied: pycg in /usr/local/lib/python3.10/dist-packages/pycg-0.0.5-py3.10.egg (from Scalpel==1.0b0) (0.0.5)
Requirement already satisfied: regex in /usr/local/lib/python3.10/dist-packages/regex-2022.4.24-py3.10-linux-x86_64.egg (from Scalpel==1.0b0) (2022.4.24)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from Scalpel==1.0b0) (59.6.0)
Requirement already satisfied: typed-ast in /usr/local/lib/python3.10/dist-packages/typed_ast-1.5.3-py3.10-linux-x86_64.egg (from Scalpel==1.0b0) (1.5.3)
Requirement already satisfied: typeshed-client in /usr/local/lib/python3.10/dist-packages/typeshed_client-2.0.5-py3.10.egg (from Scalpel==1.0b0) (2.0.5)
Requirement already satisfied: six<2.0,>=1.6.1 in /usr/lib/python3/dist-packages (from astunparse~=1.6.3->Scalpel==1.0b0) (1.16.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/lib/python3/dist-packages (from astunparse~=1.6.3->Scalpel==1.0b0) (0.37.1)
Requirement already satisfied: mistune<2,>=0.8.1 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (0.8.4)
Requirement already satisfied: jupyter-core>=4.7 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (4.10.0)
Requirement already satisfied: bleach in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (5.0.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (21.3)
Requirement already satisfied: jupyterlab-pygments in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (0.2.2)
Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (4.11.1)
Requirement already satisfied: pygments>=2.4.1 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (2.12.0)
Requirement already satisfied: tinycss2 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (1.1.1)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib/python3/dist-packages (from nbconvert->Scalpel==1.0b0) (2.0.1)
Requirement already satisfied: traitlets>=5.0 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (5.2.1.post0)
Requirement already satisfied: pandocfilters>=1.4.1 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (1.5.0)
Requirement already satisfied: entrypoints>=0.2.2 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (0.4)
Requirement already satisfied: jinja2>=3.0 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (3.1.2)
Requirement already satisfied: nbclient>=0.5.0 in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (0.6.3)
Requirement already satisfied: defusedxml in /usr/local/lib/python3.10/dist-packages (from nbconvert->Scalpel==1.0b0) (0.7.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.10/dist-packages (from nbformat->Scalpel==1.0b0) (2.15.3)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.10/dist-packages (from nbformat->Scalpel==1.0b0) (4.5.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.10/dist-packages (from typeshed-client->Scalpel==1.0b0) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat->Scalpel==1.0b0) (0.18.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat->Scalpel==1.0b0) (21.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.10/dist-packages (from nbclient>=0.5.0->nbconvert->Scalpel==1.0b0) (7.3.1)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.10/dist-packages (from nbclient>=0.5.0->nbconvert->Scalpel==1.0b0) (1.5.5)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->nbconvert->Scalpel==1.0b0) (2.3.2.post1)
Requirement already satisfied: webencodings in /usr/local/lib/python3.10/dist-packages (from bleach->nbconvert->Scalpel==1.0b0) (0.5.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/lib/python3/dist-packages (from packaging->nbconvert->Scalpel==1.0b0) (2.4.7)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.5.0->nbconvert->Scalpel==1.0b0) (2.8.2)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.10/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.5.0->nbconvert->Scalpel==1.0b0) (22.3.0)
Requirement already satisfied: tornado>=6.0 in /usr/local/lib/python3.10/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.5.0->nbconvert->Scalpel==1.0b0) (6.1)
Building wheels for collected packages: Scalpel
  Building wheel for Scalpel (setup.py): started
  Building wheel for Scalpel (setup.py): finished with status 'done'
  Created wheel for Scalpel: filename=Scalpel-1.0b0-py3-none-any.whl size=62244 sha256=9b16a07ad4e511d402ac904dda73a0c8736191d29fce99752417be5e22c4d5e9
  Stored in directory: /tmp/pip-ephem-wheel-cache-klph2au3/wheels/cb/2d/fc/e54d89757a9cc6515c8ab485452e4c012004990030bd23146b
Successfully built Scalpel
Installing collected packages: Scalpel
  Attempting uninstall: Scalpel
    Found existing installation: Scalpel 1.0b0
    Uninstalling Scalpel-1.0b0:
      Successfully uninstalled Scalpel-1.0b0
Successfully installed Scalpel-1.0b0
billquan commented 2 years ago

Hi khatchad,

Is that an output of successful installation, if I understand correctly?

khatchad commented 2 years ago

That's the output after receiving this error.

alaalial commented 2 years ago

when I run the tool for import_graph _tutorial.py import sys sys.path.append("..")

from scalpel.import_graph.import_graph import Tree,ImportGraph

root_node = Tree("import_graph_example_pkg") import_graph = ImportGraph() import_graph.build_dir_tree(root_node) module_dict = import_graph.parse_import(root_node)

I got the error import_graph = ImportGraph() TypeError: init() missing 1 required positional argument: 'entry_point'

billquan commented 2 years ago

when I run the tool for import_graph _tutorial.py import sys sys.path.append("..")

from scalpel.import_graph.import_graph import Tree,ImportGraph

root_node = Tree("import_graph_example_pkg") import_graph = ImportGraph() import_graph.build_dir_tree(root_node) module_dict = import_graph.parse_import(root_node)

I got the error import_graph = ImportGraph() TypeError: init() missing 1 required positional argument: 'entry_point'

Hi, Sorry for the late reply. We haven't updated the tutorial code and will do it soon. For quick fix, use the following code as the tutorial of ImportGraph.

from scalpel.import_graph.import_graph import Tree,ImportGraph

root_node = Tree("import_graph_example_pkg")
import_graph = ImportGraph()
import_graph.build_dir_tree()
module_dict = import_graph.parse_import(root_node)
leaf_nodes = import_graph.get_leaf_nodes()
print(len(leaf_nodes))

But @Jarvx do we want to refactor the parse_import method? It doesn't make much sense that parse_import still takes root_node as parameter after build_dir_tree is refactored. And @alaalial may I know if the installation is successful now? Thank you!

billquan commented 2 years ago

That's the output after receiving this error.

Hi @khatchad, I don't think using sudo with python setup.py install is good way to do so. You can try python setup.py install --user, which works in my environment, or python -m pip install . as I mentioned before. Please let me if these two methods work.

Jarvx commented 2 years ago

@alaalial I will close this issue now. Please feel free to reopen if you still have questions.