biopython->numpy interactive (y/n) while deploying pipeline

[brainstorm]

Even putting numpy >=1.6.1 in setup.py's before biopython, the following message pops up:

Numerical Python (NumPy) is not installed.

This package is required for many Biopython features.  Please install
it before you install Biopython. You can install Biopython anyway, but
anything dependent on NumPy will not work. If you do this, and later
install NumPy, you should then re-install Biopython.

You can find NumPy at http://numpy.scipy.org

Do you want to continue this installation? (y/N):

Apparently install_requires packages are not installed in order, so no dependency order can be defined that way... are you aware of any "pre_install_requires" or similar in setuptools ? Couldn't find it after quickly checking docs :-/

[chapmanb]

Roman; I don't believe there is a way to specify this. Biopython would have to declare NumPy as a dependency for it to be guaranteed to be pulled in beforehand. This shouldn't affect the nextgen pipeline at all since it doesn't require any of the NumPy specific parts in Biopython.

Your best bet is to preinstall libraries with something like CloudBioLinux which takes care of this. Wish I had a bcbio-nextgen specific fix for you.

Brad

[brainstorm]

Well, I will stick with:

/usr/bin/yes | python setup.py develop

In the automatic install recipe then ;)

Thx !

On 2011-08-16 19:15, chapmanb wrote:

Roman; I don't believe there is a way to specify this. Biopython would have to declare NumPy as a dependency for it to be guaranteed to be pulled in beforehand. This shouldn't affect the nextgen pipeline at all since it doesn't require any of the NumPy specific parts in Biopython.

Your best bet is to preinstall libraries with something like CloudBioLinux which takes care of this. Wish I had a bcbio-nextgen specific fix for you.

Brad

[peterjc]

A drastic solution would be to split Biopython in two, a non-numpy core and a numpy-dependent option (which would include the PDB code, Clustering, etc). I'm not ready to go down that route right now ;)

[brainstorm]

@peterjc what do you think about eliminating all interactive prompts during biopython installation ? (i.e assuming "yes" by default)... that's the approach I would definitely go for, imho.

That's what most package managers do unless explicitly stated (i.e --no-numpy).

[peterjc]

@brainstorm - Have you got an example setup.py from a third party project that does this?

[brainstorm]

@peterjc Unfortunately I haven't found any for distutils/python :-/

I was referring to package managers in general:

http://en.wikipedia.org/wiki/RPM_Package_Manager

For a system administrator performing software installation and maintenance, the use of package management rather than manual building has advantages such as simplicity, consistency and the ability for these processes to be automated and ****non-interactive****.

[peterjc]

Right, and for Debian and RedHat packages (for sure) they can control the dependencies to ensure NumPy is present, and they disable this prompt in our setup.py file.

[chapmanb]

@peterjc and @brainstorm; I wonder if it's possible to detect how setup.py was run. If easy_install and pip pass any extra flags we could pick those up and automatically add NumPy to the install_requires if so. I don't know a lot about the magic behind easy_install and pip, but can take a look. It would be great to have this happen seamlessly in automated scripts, the install_requires stuff is a huge help for distributing packages with dependencies.

Brad

[vals]

@chapmanb and @peterjc Is there any particular reason to not just let numpy be required for biopython?

[chapmanb]

Biopython doesn't want to explicitly require numpy since a lot of functionality works without it, and it can be a pretty heavy dependency. That said, I'd like to see something better happen here so Biopython is more useful with install_requires

[peterjc]

Further to Brad's reply, also note NumPy isn't available on Jython or PyPy.

[brainstorm]

Hello @vals @chapmanb

I just wanted to add that trying to workaround it with /usr/bin/yes does not work very well either :-/

I'm setting up a CI server with Jenkins + python virtualenvs to run Brad's BCBB pipeline testsuite automatically for all our repos, but the automatic build fails right there on biopython spot (exit code -1):

/usr/bin/yes | python setup.py install

Numerical Python (NumPy) is not installed.

(...)

You can find NumPy at http://numpy.scipy.org

error: Setup script exited with -1
Build step 'Execute shell' marked build as failure
Recording test results
Finished: FAILURE

I will go about the workaround suggested by @peterjc regarding the rewrite of biopython's setup.py as package managers do, any other suggestions are welcome.

Thanks ! Roman

[chapmanb]

@brainstorm @peterjc @vals What do you think of this patch?

https://github.com/chapmanb/biopython/commit/be53d850d721fc82af81bedcd9fb9034b0a2099b

It skips the question/answer when run from automated programs like easy_install and pip and also adds in install_requires in these cases. It is back compatible for standard setup.py running but should help greatly for these automated cases.

[brainstorm]

Looks perfect Brad, thanks for this :)

6 okt 2011 kl. 16:28 skrev Brad Chapman:

@brainstorm @peterjs @vals What do you think of this patch?

https://github.com/chapmanb/biopython/commit/be53d850d721fc82af81bedcd9fb9034b0a2099b

It skips the question/answer when run from automated programs like easy_install and pip and also adds in install_requires in these cases. It is back compatible for standard setup.py running but should help greatly for these automated cases.

Reply to this email directly or view it on GitHub: https://github.com/chapmanb/bcbb/issues/35#issuecomment-2310417