Configure with python3

[colindaven]

Hi,

I am not too great with configure and wonder how to get python3 configured as default.

I already have python3 and python3-dev installed on my Ubuntu 14.04 machine.

This does not work ./configure --prefix /home/bioinformatics/NAS01/programs/portcullis/portcullis/bin/ PYTHON_VERSION=3.4

Thanks, Colin

` .. checking for a version of Python >= '3.1'... no configure: WARNING: this package requires Python >= '3.1'. If you have it installed, but it isn't the default Python interpreter in your system path, please pass the PYTHON_VERSION variable to configure. See``configure --help'' for reference.

checking for the distutils Python package... yes checking for Python include path... -I/usr/include/python2.7 checking for Python library path... -L/usr/lib -lpython2.7 checking for Python site-packages path... /usr/lib/python2.7/dist-packages checking python extra libraries... -lpthread -ldl -lutil -lm checking python extra linking flags... -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions checking consistency of all components of python development environment... yes configure: error: Python3 not detected. If running on a debian system try "sudo apt-get install python3-dev". `

[maplesond]

Hi Colin,

So there's a few points here. First of all to get up to speed with configure you can use the ./configure --help like any other decent command line tool/script. It will show you all the available options and what they all do and mean. Second, the error seems strange, if you do have python3 installed I'm guessing it's not the default python installation. If you type which python, what comes up? It could be that there's some bug with using the PYTHON_VERSION option to indicate that portcullis configure should use that version of python. This was some third party code that I've used for this and haven't personally tested out this kind of configuration. Before I start investigating further can you first try breaking this into two commands: export PYTHON_VERSION=3.4 && ./configure. Finally, I suspect that your --prefix value may not lead to where you expect. When typing make install portcullis will put a bunch of directories into the prefix directory, including it's bin lib share, etc directories.

Best, Dan

[colindaven]

Hi,

yeah, we're still on python2.7.6 as default, which is likely the issue here.

export PYTHON_VERSION=3.4 && ./configure did no help, led to exactly the same problem.

The variable is being correctly set though.

export PYTHON_VERSION=3.4 echo $PYTHON_VERSION 3.4

PYTHON was not set though, but I just did this: no changes in the result though, see bottom for output.

PYTHON=/usr/bin/python3 echo $PYTHON /usr/bin/python3

configure: WARNING: this package requires Python >= '3.1'. If you have it installed, but it isn't the default Python interpreter in your system path, please pass the PYTHON_VERSION variable to configure. See ``configure --help'' for reference.

checking for the distutils Python package... yes checking for Python include path... -I/usr/include/python2.7 checking for Python library path... -L/usr/lib -lpython2.7 checking for Python site-packages path... /usr/lib/python2.7/dist-packages checking python extra libraries... -lpthread -ldl -lutil -lm checking python extra linking flags... -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions checking consistency of all components of python development environment... yes configure: error: Python3 not detected. If running on a debian system try "sudo apt-get install python3-dev".

Setting the alias python -> python3 is not going to help here either I would guess ( as in http://askubuntu.com/questions/590027/how-to-set-python-3-as-default-interpreter-in-ubuntu-14-4 )

Interestingly, python 3.4 does not seem to be listed in the file "configure"

    # Find any Python interpreter.
if test -z "$PYTHON"; then
  for ac_prog in python python2 python3 python3.3 python3.2 python3.1 python3.0 python2.7 

Adding it manually do not help either though ...

Thanks Colin

[maplesond]

Hi Colin, interesting observation regarding the python version numbers. Yes, it looks as though some of the python detection code was written before 3.4 and 3.5 were available so these options are missing. I'll try to update this for completeness. Thanks for bringing it to my attention. However, it should still work if python links/aliases to python3. The main test is to see what happens when you type which python if that still points to python2 then that's what portcullis will see as well.

In case you want to investigate the code further, the actual configure script is automatically generated, so the human readable code for python detection is in ./m4/ax_python_devel.m4 in case you want to look at this in more detail and the human readable version of the configuration file is called configure.ac. You can regenerate the configure script by typing ./autogen.sh.

[maplesond]

Hi Colin,

I have a quick test for you to try. Can you comment out line 98 of the configure.ac file, which says AM_PATH_PYTHON([], [], AC_MSG_ERROR([Python interpreter not found.])). Then type: ./autogen.sh to regenerate the configuration script. ./configure PYTHON_VERSION=3.4 you can add in the prefix if you like.

I think what may have been happening is that the autoconf python detection code was overriding the PYTHON_VERSION variable.

Best, Dan

[colindaven]

Hi Dan,

thanks for the help, had to attend some urgent matters on Friday.

So which python refers to python2 as you/we suspected.

which python /usr/bin/python

/usr/bin/python Python 2.7.6 (default, Jun 22 2015, 17:58:13) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information.

The trick is to change from python2 to python3 robustly for this program without destroying all the python2 dependencies in the system.

Part 2

I then tried commenting out the line in the configure.ac

The configure command then got past the python3 part happily - thanks !

Next problem was sphinx, perhaps adding this to the docs would help (if now already there)?

sudo apt-get install python-sphinx

[colindaven]

And sphinx in the ubuntu 14.04 is too old, pip installs a more modern version apparently which isn't detected well either.

Is there any chance the sphinx dependency can be downgraded from error level to warning level status ?

[maplesond]

Hi Colin,

Thanks for the feedback, this is really useful. Even using travis it's difficult to test every system configuration. So for my setup I just installed python3 anaconda, which includes sphinx but I think it's worth adding this comment to the documentation as you suggest. I'll also work on making the doc building optional.

Best, Dan

[colindaven]

Hi Dan,

thanks, I might try anaconda as well but haven't been too happy with it in the past.

I think Ubuntu 14.04 is a pretty common server configuration so might be useful to test the installation on if you have a VM handy. Thanks for working on changing the doc build process.

Colin

[maplesond]

Will do.

I've updated the develop branch with the changes discussed here. Feel free to have a go if you have time.

Best, Dan

[colindaven]

Hi Dan,

thanks for the prompt support.

The develop branch configured without any edits on my side, python3.4 and sphinx went through fine.

The contents of my "build/bin" directory are

programs/portcullis/portcullis_dev2/portcullis/build/bin

analyse_alignments.py analyse_junctions.py bed12.py bed12_set.py bed_merge.py bed_v_ref.py create_ref.py cufflinks2spankicov.py example_pp_ath.yaml filter_junctions.py fs2gff.py hpc.json ml.py pca.py performance.py portcullis portcullis_convert.py predict.snakefile read_gen.snakefile snakey.py spanki_filter.py split_error_classes.py tab_bed12_set.py tab_combine.py tab_in_bedref.py tab.py venn.py

If that looks reasonable I will give the binary a go when I have a moment.

cheers Colin

[maplesond]

Yes. That looks about right. Im in the process of tidying up all the scripts so hopefully future versions will have fewer but better named and useful scripts. The documentation should give you an idea for what some of them do.

[maplesond]

Hello, I've made a new release of portcullis, which should solve your original issue. I've also cleaned up the python scripts. These are now condensed into a single executable called junctools. If there are any issues feel free to reopen the case.

[colindaven]

Hi, thanks for letting me know. I haven't had a chance to try the system again and am moving to a new institute. There will be different priorities there initially but I'll come back to transcripts and annotation in the long run.