quick install: Rekall install with pip fails on --pre on 12.04?

[adricnet]

Seems the pip in Ubuntu 12.04 doesn't have the install --pre option, though others do, such as pip-2.7 on my Mac.

A quick edit to the install script got rekall installed and the quick install moving along. It's line 127 in the install_script_ubuntu.sh I wget'd in a couple hours ago as per the quick install adoc.

header "Installing Rekall"
run_cmd_confirm sudo pip install --upgrade rekall #--pre

If you want a pull req (or patch) for this please tell me where to send it.

Thanks, adric

[grrrrrrrrr]

Hm interesting. We have to add the --pre because Rekall depends on Distorm3 and that project hasn't done a proper release:

pip install distorm3

Downloading/unpacking distorm3 Could not find a version that satisfies the requirement distorm3 (from versions: 3-sdist) Cleaning up... No distributions matching the version for distorm3 Storing debug log for failure in /root/.pip/pip.log

pip install distorm3 --pre

Downloading/unpacking distorm3 Downloading distorm3-3-sdist.zip (143kB): 143kB downloaded Running setup.py (path:/tmp/pip_build_root/distorm3/setup.py) egg_info for package distorm3 [...]

Does a pip install Distorm3 (without the --pre) install Distorm on Ubuntu 12?

[adricnet]

Yes, that seems to work fine now, after a quick compile, on a different 12.04:

tester@so:~$ sudo pip install distorm3
Downloading/unpacking distorm3
Downloading distorm3-3-sdist.zip (143Kb): 143Kb downloaded
Running setup.py egg_info for package distorm3

Installing collected packages: distorm3
Running setup.py install for distorm3
building 'distorm3' library
... gcc snipped ...
Successfully installed distorm3
Cleaning up...

tester@so:~$ pip --version
pip 1.0 from /usr/lib/python2.7/dist-packages (python 2.7)
tester@so:~$ uname -a
Linux so 3.2.0-69-generic #103-Ubuntu SMP Tue Sep 2 05:02:14 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
tester@so:~$ dpkg -l python-pip
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                Version             Description
  +++-===================-===================-===========================
ii  python-pip          1.0-1build1         alternative Python package installer

hth, adric

[scudette]

The --pre flag should not be needed any more since rekall had been properly released. Am 16.10.2014 15:15 schrieb "Adric Net" notifications@github.com:

Yes, that seems to work fine now, after a quick compile, on a different 12.04:

tester@so:~$ sudo pip install distorm3 Downloading/unpacking distorm3 Downloading distorm3-3-sdist.zip (143Kb): 143Kb downloaded Running setup.py egg_info for package distorm3

Installing collected packages: distorm3 Running setup.py install for distorm3 building 'distorm3' library ... gcc snipped ... Successfully installed distorm3 Cleaning up...

tester@so:~$ pip --version pip 1.0 from /usr/lib/python2.7/dist-packages (python 2.7) tester@so:~$ uname -a Linux so 3.2.0-69-generic #103-Ubuntu SMP Tue Sep 2 05:02:14 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux tester@so:~$ dpkg -l python-pip Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-===================-===================-=========================== ii python-pip 1.0-1build1 alternative Python package installer

hth, adric

— Reply to this email directly or view it on GitHub https://github.com/google/grr/issues/36#issuecomment-59358728.

[grrrrrrrrr]

Yes, Rekall is not the problem, Distorm doesn't install without --pre. And if that fails, Rekall will give up too.

[adricnet]

Looks like distorm3 will install okay on pip today, on 12.04 LTS. Maybe this is no longer a problem with the latest patches to 12.04? I have Ubuntu 12.04.5 LTS, from ISO + apt'd up.

Rekall wasn't able to immediately pip in on stock 12.04 LTS as acora has to be built and python-dev was not yet installed in my base image. Once I add that to base (and it's in the install script for GRR already) it builds fine on stock LTS 12.04 after pulling in all the deps:

Successfully installed acora codegen Flask Flask-Sockets gevent gevent-websocket Werkzeug Jinja2 itsdangerous greenlet

[grrrrrrrrr]

So I quickly verified that this works as I thought it would. There are essentially two different versions of pip here, newer ones that understand --pre and older ones that don't.

If you have the older one (as here on 12.04), you can't specify --pre or it will complain (obviously). However, if you don't give the --pre flag, it will install everything just fine because it doesn't make a distinction between stable and prerelease repos (since back then these didn't exist) and happily installs the prerelease one.

However, if you have the newer version, it will refuse to install without --pre since it will only use stable repos but distorm3 isn't available in stable.

So in the end, we need to try both, or maybe just do the right thing depending on the version installed.

Here is what I tried: http://pastebin.com/8CMGt2ve

We should also somehow get the dependency on python-dev in there somewhere so pip doesn't fail.

[adricnet]

Reads pastebin post Oh wow, so now it works on 14.04 but not on 12.04? Gah.

I'm still stepping through quick install (github version) trying to get it all to work on stock 12.04 or 14.04, but let me know how I can help with these pip madness (testing?).

Do you want a separate issue on python-devel for Ubuntu_install ?

hth, adric

[grrrrrrrrr]

I added a workaround to the install script to try both, with --pre and without. This should work now.

[brifordwylie]

Just an unsolicited comment on this, I noticed that the pip that comes with python 2.7.9 does not seem to recognize the --pre argument (btw Travis CI is now using 2.7.9). So there's literally no way to pip install distorm3 (with or without --pre arg). I've sent an email with this info to Gil Dabah.

[brifordwylie]

Okay, after contacting the distorm3 maintainer this is now fixed.

[scudette]

Thanks Brian for taking care of this!

On 4 January 2015 at 04:20, Brian Wylie notifications@github.com wrote:

Okay, after contacting the distorm3 maintainer this is now fixed.

— Reply to this email directly or view it on GitHub https://github.com/google/grr/issues/36#issuecomment-68604081.