search

viper-framework / viper

Binary analysis and management framework
Other
1.54k stars 353 forks source link

fix long and unreadable one-liners #738

Closed JordanSamhi closed 5 years ago

JordanSamhi commented 5 years ago

I removed long and unreadable one-liners as requested by @Rafiot

JordanSamhi commented 5 years ago

I do not know what is going on with travis tests on debug function : 

if self.IS_PE and self.lief.has_debug:
           signature = ""
            debug = self.lief.debug[0]
           TypeError: '_pylief.PE.Debug' object does not support indexing
viper/modules/lief.py:991: `TypeError

Locally on my computer it works pretty well, no error. I verified, I have lief 0.9.0 like Travis. On my computer the "self.lief.debug" variable is a list. It does not seem to be a list on Travis, I don't know why. But now I understand why you changed from for debug in self.lief.debug: to debug = self.lief.debug in commit f4158fd534f2daf89ee644358a069c48a4d4868d.

Any ideas ?

botherder commented 5 years ago

I can't seem to even be able to install lief. I don't even know where is the lief dependency defined in the requirements files or setup script, cause I just can't see it...


Collecting lief
  Using cached https://files.pythonhosted.org/packages/cd/0e/0d6f3357975dd1530aeb4b4a84a99d493775391758378fb5109f47b613f9/lief-0.9.0.zip
Building wheels for collected packages: lief
  Building wheel for lief (setup.py) ... error
  ERROR: Complete output from command /tmp/venv/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-di02cjnl/lief/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-qte5osyu --python-tag cp37:
  ERROR: running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/lief
  copying lief/__init__.py -> build/lib.linux-x86_64-3.7/lief
  running build_ext
  Url: https://github.com/lief-project/LIEF/releases/download/0.9.0/lief-0.9.0-py3.7-linux.egg
  HTTP Error 404: Not Found
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-di02cjnl/lief/setup.py", line 285, in <module>
      'sdist':     lief_sdist
    File "/tmp/venv/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/tmp/venv/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 192, in run
      self.run_command('build')
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/tmp/venv/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 78, in run
      _build_ext.run(self)
    File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/usr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-install-di02cjnl/lief/setup.py", line 215, in build_extension
      self._install_from_source_package()
    File "/tmp/pip-install-di02cjnl/lief/setup.py", line 265, in _install_from_source_package
      raise Exception("Unable to find {}".format(url))
  Exception: Unable to find /home/user/lief-0.9.0-py3.7-linux.egg
  ----------------------------------------
  ERROR: Failed building wheel for lief
  Running setup.py clean for lief
Failed to build lief
Installing collected packages: lief
  Running setup.py install for lief ... error
    ERROR: Complete output from command /tmp/venv/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-di02cjnl/lief/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-w_ymd0rm/install-record.txt --single-version-externally-managed --compile --install-headers /tmp/venv/include/site/python3.7/lief:
    ERROR: running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/lief
    copying lief/__init__.py -> build/lib.linux-x86_64-3.7/lief
    running build_ext
    Url: https://github.com/lief-project/LIEF/releases/download/0.9.0/lief-0.9.0-py3.7-linux.egg
    HTTP Error 404: Not Found
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-di02cjnl/lief/setup.py", line 285, in <module>
        'sdist':     lief_sdist
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.7/distutils/command/install.py", line 589, in run
        self.run_command('build')
      File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 78, in run
        _build_ext.run(self)
      File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
        self.build_extensions()
      File "/usr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "/usr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "/tmp/pip-install-di02cjnl/lief/setup.py", line 215, in build_extension
        self._install_from_source_package()
      File "/tmp/pip-install-di02cjnl/lief/setup.py", line 265, in _install_from_source_package
        raise Exception("Unable to find {}".format(url))
    Exception: Unable to find /home/user/lief-0.9.0-py3.7-linux.egg
    ----------------------------------------
ERROR: Command "/tmp/venv/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-di02cjnl/lief/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-w_ymd0rm/install-record.txt --single-version-externally-managed --compile --install-headers /tmp/venv/include/site/python3.7/lief" failed with error code 1 in /tmp/pip-install-di02cjnl/lief/```
JordanSamhi commented 5 years ago

I've just tried the installation from scratch on a new machine, it works pretty well. I cloned the repo and used this doc : https://viper.li/en/latest/installation/index.html to install viper. I did not have the same problem as you. But I can still add the lief requirement.

botherder commented 5 years ago

Oh, I know why that is. I use Python 3.7 and LIEF project does not make releases for it: https://github.com/lief-project/LIEF/releases

JordanSamhi commented 5 years ago

Yes exactly I had'nt seen it, it's embarassing. Should we put a warning somewhere to let people know that they have to use python < 3.7, or block the lief installation if this is not the case. Or ask LIEF maintainer to make a release for python 3.7 ?

botherder commented 5 years ago

There will be a release in June apparently: https://github.com/lief-project/LIEF/issues/282#issuecomment-473993736

For the moment, building from source should work, although it is awfully slow.

JordanSamhi commented 5 years ago

Ok great then.

botherder commented 5 years ago

I would say we can merge this, but we will need to wait to add it to requirements-modules.txt file, or it will break installations. If you could comment it out, then we can merge.

JordanSamhi commented 5 years ago

Done. But maybe before merging we should wait @Rafiot response for my old comment :

I do not know what is going on with travis tests on debug function :

if self.IS_PE and self.lief.has_debug:
           signature = ""
            debug = self.lief.debug[0]
           TypeError: '_pylief.PE.Debug' object does not support indexing
viper/modules/lief.py:991: `TypeError

Locally on my computer it works pretty well, no error. I verified, I have lief 0.9.0 like Travis. On my computer the "self.lief.debug" variable is a list. It does not seem to be a list on Travis, I don't know why. But now I understand why you changed from for debug in self.lief.debug: to debug = self.lief.debug in commit f4158fd.

Any ideas ?

It seems that self.lief.debug act like a standalone variable on travis and @Rafiot computer. It's a list on mine, don't know why. This is not stable. I've seen it after the PR, I'll try to find out.

Rafiot commented 5 years ago

Yeah, I'm having the same problem everywhere I use lief and it is pretty annoying.

There is a package available for python 3.7 if you install the nightly build. It's working, but that's very suboptimal: https://github.com/lief-project/LIEF#downloads--install

It is a working(-ish) solution for python 3.5+.

And regarding your question, I'm looking at it now, sorry for the late answer.

=> Note on installing lief: https://github.com/lief-project/LIEF/issues/294

Rafiot commented 5 years ago

I did a few cleanups and changes in the install process, bumped the dependencies (mostly removed installations from repositories) => https://github.com/viper-framework/viper/pull/741