phyluce installation: dependencies missing?

[jbernot]

I'm trying to install phyluce with miniconda3 on an HPC (running "Rocky Linux" v8.8). The installation appears to complete successfully, but as I work through the Tutorial IV some tools appear to be missing including:

• faToTwoBit
• art_illumina
• stampy.py

Many tools are present in /envs/phyluce-1.7.3/bin and work fine (samtools, bedtools, spades.py etc) but others do not. I manually installed faToTwoBit and art using:

conda install -c bioconda ucsc-fatotwobit
conda install -c bioconda art

but ran into the issue again with stampy, which I can't find in bioconda.

I tried reinstalling phyluce-1.7.3 and when that didn't fix the issue I installed 1.7.2 and 1.7.1 in separate envs but had the same issue each time. I also installed 1.7.3 on my Mac and had the same issue.

Any suggestions for installing the missing tools? I have manually installed all except for stampy, which I am having trouble installing. Is there another aligner I can use instead of stampy?

[brantfaircloth]

Yes, these are missing... I will work to add them back in the next release.

Stampy is a bit of a pain... It appears that now you have to register for the download (here) and you'll receive a link to the source code by email. You then need to build the source as described in the stampy README.

You can build the source within your conda environment (by navigating to the correct directories), or you can build the source external to your phyluce installation, and then make sure stampy is in your $PATH (probably easiest).

As for other programs you could use, you can potentially modify parameters of bwa (or bowtie) to map reads at less stringency, but I do not know the exact changes you would need to make to emulate stampy.

[jbernot]

Thanks, I appreciate it!

Yeah, I'm struggling with the stampy installation. I tried to install it in a separate conda env from phyluce since it requires python 2.7, but keep getting linking errors like the following when I run make:

cp -r build/python2.7/* .
/usr/bin/ld: cannot find -lpython2.7
collect2: error: ld returned 1 exit status

I know that is a stampy installation problem and not a phyluce problem so I don't mean to bog you down with it here.

Side note, for anyone interested in installing stampy: you can skip the registration and email requirement to download if you search for "stampy" at https://www.well.ox.ac.uk and click "Attachments", or just follow this link.

It downloads a tarball but its a file named download so extract with

tar -xzf \@\@download

[brantfaircloth]

I also have the source if someone cannot access (and you're reading this post) - just request. The license terms are unclear on redistribution.

Let me check on compilation real quickly... ok, here's what I did that seems to have worked (on linux, x86_64):

conda create -n stampy python=2.7
conda activate stampy
# get path to this particular conda env
which python
# returns ~/conda/envs/stampy/bin/python
# now change to that env
cd ~/conda/envs/stampy/
mkdir src && cd src
# cp/mv stampy src to this location
mv ~/tmp/stampy-latest.tgz ./
tar -xzvf stampy-latest.tgz
cd stampy-1.0.32
make

[jbernot]

Thanks! I was having an issue with the stampy installation apparently because g++ linker was not finding the python2.7l library in my conda env, but after fixing that your compilation instructions worked for me. Thank you!

[brantfaircloth]

Excellent.

[forianfinest]

Thanks! I was having an issue with the stampy installation apparently because g++ linker was not finding the python2.7l library in my conda env, but after fixing that your compilation instructions worked for me. Thank you!

Hi Jimmy, sorry for the bother.

I am having the "cannot find -lpython2.7" too, even after following Brant's instruction. How did you fix it?

Best wishes, Mattia

EDIT: maybe I was able to fix it by importing libraries:

`export CPATH =~/mambaforge-pypy3/envs/stampy/include

export LIBRARY_PATH=~/mambaforge-pypy3/envs/stampy/bin

export LD_LIBRARY_PATH=~/mambaforge-pypy3/envs/stampy/bin`

Let me know if you did the same.

[jbernot]

Hi Mattia,

It appears you may have to provide the library path for g++ to build. I struggled with it for several days and eventually reached out to the HPC team at my university and they were able to fix it.

This is what they told me the problem was:

The building issue of stampy should have been fixed. g++ linker could not find the python2.7l library (which is from your "stampy" conda environment). I have to provide the library path for g++ to build.

I hope you can figure it out. Sorry I cannot be more helpful.

Best, Jimmy

James P. Bernot, PhD NSF Postdoctoral Fellow Department of Invertebrate Zoology Smithsonian National Museum of Natural History https://jimmybernot.com Twitter: @JimmyBernot https://twitter.com/JimmyBernot Google Scholar https://scholar.google.com/citations?hl=en&user=9sa6KNwAAAAJ He/Him

On Tue, Feb 20, 2024 at 6:59 AM Mattia De Vivo @.***> wrote:

Thanks! I was having an issue with the stampy installation apparently because g++ linker was not finding the python2.7l library in my conda env, but after fixing that your compilation instructions worked for me. Thank you!

Hi Jimmy, sorry for the bother.

I am having the "cannot find -lpython2.7" too, even after following Brian's instruction. How did you fix it?

Best wishes, Mattia

— Reply to this email directly, view it on GitHub https://github.com/faircloth-lab/phyluce/issues/311#issuecomment-1954065329, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEIPGY745PS77HPPDWMICE3YUSF2TAVCNFSM6AAAAAA4IAH7BOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGA3DKMZSHE . You are receiving this because you authored the thread.Message ID: @.***>

[brantfaircloth]

You may have to make a separate environment (solely) for stampy and Python2.7.

Basically, I think the issue is that building stampy requires Python2.7, while phyluce requires Python3.6. So, you can't build Stampy in the phyluce environment (the python version is wrong, which is why you see the link error). You should be able to create a separate Python2.7 environment, then build stampy within that.

[forianfinest]

Hi Mattia, It appears you may have to provide the library path for g++ to build. I struggled with it for several days and eventually reached out to the HPC team at my university and they were able to fix it. This is what they told me the problem was: The building issue of stampy should have been fixed. g++ linker could not find the python2.7l library (which is from your "stampy" conda environment). I have to provide the library path for g++ to build. I hope you can figure it out. Sorry I cannot be more helpful.

Hy Jimmy, thank you for your answer! It seems that by exporting the right paths it worked fine!

You may have to make a separate environment (solely) for stampy and Python2.7.

Basically, I think the issue is that building stampy requires Python2.7, while phyluce requires Python3.6. So, you can't build Stampy in the phyluce environment (the python version is wrong, which is why you see the link error). You should be able to create a separate Python2.7 environment, then build stampy within that.

Hi Brant, I basically did that and it worked well, albeit I also edited the bash script for the alignment by telling bash where python2 and stampy were while staying in the phyluce conda enviroment.

Something like:

/mambaforge-pypy3/envs/stampy/bin/python2 /mambaforge-pypy3/envs/stampy/src/stampy-1.0.32/stampy.py etc. and (for now) is working.

Thanks to both of you for your answers!

Best wishes, Mattia