search

ex0dus-0x / fuzzable

Framework for Automating Fuzzable Target Discovery with Static Analysis.
MIT License
509 stars 53 forks source link

site-packages/third_party/tree-sitter-c/src/parser.c not found - the entire third_party folder doesn't exist #8

Closed myasn1k closed 2 years ago

myasn1k commented 2 years ago

Hi, sorry bothering you, Running fuzzable on any c file/project produces to me this error:

$ fuzzable analyze exiftags
[...]
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/myasnik/.local/lib/python3.10/site-packages/fuzzable/__main__.py:89 in analyze             │
│                                                                                                  │
│    86 │   if target.is_file():                                                                   │
│    87 │   │   run_on_file(target, mode, score_weights, export, list_ignored)                     │
│    88 │   elif target.is_dir():                                                                  │
│ ❱  89 │   │   run_on_workspace(target, mode, score_weights, export, list_ignored)                │
│    90 │   else:                                                                                  │
│    91 │   │   error(f"Target path `{target}` does not exist")                                    │
│    92                                                                                            │
│                                                                                                  │
│ ╭────────────────── locals ───────────────────╮                                                  │
│ │         debug = False                       │                                                  │
│ │        export = None                        │                                                  │
│ │  list_ignored = False                       │                                                  │
│ │          mode = <AnalysisMode.RECOMMEND: 0> │                                                  │
│ │ score_weights = [0.3, 0.3, 0.05, 0.05, 0.3] │                                                  │
│ │        target = PosixPath('exiftags')       │                                                  │
│ ╰─────────────────────────────────────────────╯                                                  │
│                                                                                                  │
│ /home/myasnik/.local/lib/python3.10/site-packages/fuzzable/__main__.py:163 in run_on_workspace   │
│                                                                                                  │
│   160 │   │   │   "No C/C++ source code found in the workspace. fuzzable currently does not s    │
│   161 │   │   )                                                                                  │
│   162 │                                                                                          │
│ ❱ 163 │   analyzer = AstAnalysis(                                                                │
│   164 │   │   source_files, mode, score_weights=score_weights, basedir=target                    │
│   165 │   )                                                                                      │
│   166 │   log.info(f"Running fuzzable analysis with the {str(analyzer)} analyzer")               │
│                                                                                                  │
│ ╭────────────────────────────────────── locals ──────────────────────────────────────╮           │
│ │             _ = []                                                                 │           │
│ │        export = None                                                               │           │
│ │          file = 'id:000403,src:000253,time:1453128,execs:3687367,op:flip1,pos:187' │           │
│ │         files = []                                                                 │           │
│ │  list_ignored = False                                                              │           │
│ │          mode = <AnalysisMode.RECOMMEND: 0>                                        │           │
│ │ score_weights = [0.3, 0.3, 0.05, 0.05, 0.3]                                        │           │
│ │  source_files = [                                                                  │           │
│ │                 │   PosixPath('exiftags/asahi.c'),                                 │           │
│ │                 │   PosixPath('exiftags/canon.c'),                                 │           │
│ │                 │   PosixPath('exiftags/casio.c'),                                 │           │
│ │                 │   PosixPath('exiftags/exif.c'),                                  │           │
│ │                 │   PosixPath('exiftags/exif.h'),                                  │           │
│ │                 │   PosixPath('exiftags/exifcom.c'),                               │           │
│ │                 │   PosixPath('exiftags/exifgps.c'),                               │           │
│ │                 │   PosixPath('exiftags/exifint.h'),                               │           │
│ │                 │   PosixPath('exiftags/exiftags.c'),                              │           │
│ │                 │   PosixPath('exiftags/exiftime.c'),                              │           │
│ │                 │   ... +18                                                        │           │
│ │                 ]                                                                  │           │
│ │        subdir = 'exiftags/fuzz_out_1/default/hangs'                                │           │
│ │        target = PosixPath('exiftags')                                              │           │
│ ╰────────────────────────────────────────────────────────────────────────────────────╯           │
│                                                                                                  │
│ /home/myasnik/.local/lib/python3.10/site-packages/fuzzable/analysis/ast.py:35 in __init__        │
│                                                                                                  │
│    32 │   │   super().__init__(target, mode, score_weights)                                      │
│    33 │   │                                                                                      │
│    34 │   │   log.debug("Building third-party tree-sitter libraries for C/C++ languages")        │
│ ❱  35 │   │   Language.build_library(                                                            │
│    36 │   │   │   BUILD_PATH,                                                                    │
│    37 │   │   │   [                                                                              │
│    38 │   │   │   │   os.path.join(ROOT_DIR, "third_party/tree-sitter-c"),                       │
│                                                                                                  │
│ ╭─────────────────────────────────── locals ───────────────────────────────────╮                 │
│ │     __class__ = <class 'fuzzable.analysis.ast.AstAnalysis'>                  │                 │
│ │       basedir = PosixPath('exiftags')                                        │                 │
│ │          mode = <AnalysisMode.RECOMMEND: 0>                                  │                 │
│ │ score_weights = [0.3, 0.3, 0.05, 0.05, 0.3]                                  │                 │
│ │          self = <fuzzable.analysis.ast.AstAnalysis object at 0x7f4c4dbfe020> │                 │
│ │        target = [                                                            │                 │
│ │                 │   PosixPath('exiftags/asahi.c'),                           │                 │
│ │                 │   PosixPath('exiftags/canon.c'),                           │                 │
│ │                 │   PosixPath('exiftags/casio.c'),                           │                 │
│ │                 │   PosixPath('exiftags/exif.c'),                            │                 │
│ │                 │   PosixPath('exiftags/exif.h'),                            │                 │
│ │                 │   PosixPath('exiftags/exifcom.c'),                         │                 │
│ │                 │   PosixPath('exiftags/exifgps.c'),                         │                 │
│ │                 │   PosixPath('exiftags/exifint.h'),                         │                 │
│ │                 │   PosixPath('exiftags/exiftags.c'),                        │                 │
│ │                 │   PosixPath('exiftags/exiftime.c'),                        │                 │
│ │                 │   ... +18                                                  │                 │
│ │                 ]                                                            │                 │
│ ╰──────────────────────────────────────────────────────────────────────────────╯                 │
│                                                                                                  │
│ /home/myasnik/.local/lib/python3.10/site-packages/tree_sitter/__init__.py:41 in build_library    │
│                                                                                                  │
│   38 │   │   │   │   source_paths.append(path.join(src_path, "scanner.cc"))                      │
│   39 │   │   │   elif path.exists(path.join(src_path, "scanner.c")):                             │
│   40 │   │   │   │   source_paths.append(path.join(src_path, "scanner.c"))                       │
│ ❱ 41 │   │   source_mtimes = [path.getmtime(__file__)] + [                                       │
│   42 │   │   │   path.getmtime(path_) for path_ in source_paths                                  │
│   43 │   │   ]                                                                                   │
│   44                                                                                             │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │          cpp = False                                                                         │ │
│ │ output_mtime = 0                                                                             │ │
│ │  output_path = '/home/myasnik/.local/lib/python3.10/site-packages/build/lang.so'             │ │
│ │    repo_path = '/home/myasnik/.local/lib/python3.10/site-packages/third_party/tree-sitter-c… │ │
│ │   repo_paths = [                                                                             │ │
│ │                │                                                                             │ │
│ │                '/home/myasnik/.local/lib/python3.10/site-packages/third_party/tree-sitter-c… │ │
│ │                │                                                                             │ │
│ │                '/home/myasnik/.local/lib/python3.10/site-packages/third_party/tree-sitter-c… │ │
│ │                ]                                                                             │ │
│ │ source_paths = [                                                                             │ │
│ │                │                                                                             │ │
│ │                '/home/myasnik/.local/lib/python3.10/site-packages/third_party/tree-sitter-c… │ │
│ │                │                                                                             │ │
│ │                '/home/myasnik/.local/lib/python3.10/site-packages/third_party/tree-sitter-c… │ │
│ │                ]                                                                             │ │
│ │     src_path = '/home/myasnik/.local/lib/python3.10/site-packages/third_party/tree-sitter-c… │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /home/myasnik/.local/lib/python3.10/site-packages/tree_sitter/__init__.py:42 in <listcomp>       │
│                                                                                                  │
│   39 │   │   │   elif path.exists(path.join(src_path, "scanner.c")):                             │
│   40 │   │   │   │   source_paths.append(path.join(src_path, "scanner.c"))                       │
│   41 │   │   source_mtimes = [path.getmtime(__file__)] + [                                       │
│ ❱ 42 │   │   │   path.getmtime(path_) for path_ in source_paths                                  │
│   43 │   │   ]                                                                                   │
│   44 │   │                                                                                       │
│   45 │   │   compiler = new_compiler()                                                           │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │    .0 = <list_iterator object at 0x7f4c4dbfcf10>                                             │ │
│ │ path_ = '/home/myasnik/.local/lib/python3.10/site-packages/third_party/tree-sitter-c/src/'+8 │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /usr/lib/python3.10/genericpath.py:55 in getmtime                                                │
│                                                                                                  │
│    52                                                                                            │
│    53 def getmtime(filename):                                                                    │
│    54 │   """Return the last modification time of a file, reported by os.stat()."""              │
│ ❱  55 │   return os.stat(filename).st_mtime                                                      │
│    56                                                                                            │
│    57                                                                                            │
│    58 def getatime(filename):                                                                    │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ filename = '/home/myasnik/.local/lib/python3.10/site-packages/third_party/tree-sitter-c/src… │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
FileNotFoundError: [Errno 2] No such file or directory: '/home/myasnik/.local/lib/python3.10/site-packages/third_party/tree-sitter-c/src/parser.c'

Maybe it's just that I'm noob and I'm messing somethig.. I hope not to waste your time. I installed fuzzable as defined in the README (using pip install . in the cloned project folder); I also tried with a virtualenv but nothing changes except as expected the path of libraries used. I'm running arch linux (fully update) and the latest python version.

Thanks a lot for your help and patience!

ex0dus-0x commented 2 years ago

Hey @myasn1k,

Thank you for reaching out! This is an issue with my end having to deal with dependency and build hell for a project like this.

I've updated the README for the time such that the manual build method is used as a priority rather than pip installing it. Feel free to git clone with all the submodules and do a local installation from there. Will try to push out a better build strategy when I have the time to.

Thanks!

myasn1k commented 2 years ago

Thanks a lot for your help @ex0dus-0x! Unfortunately I was already doing what you suggest. Anyhow I managed to solve the problem! I thought that the folder missing (third_party) was managed by pip or poetry but it actually isn't; the folder is in the project folder instead. Copying it to ~/.local/lib/python3.10/site-packages/ solves the problem. As i said before I'm a noob so i don't know if this step is standard, if it isn't it would be great if you could add this to the README. Thanks again for your help!