etetoolkit / ete

Python package for building, comparing, annotating, manipulating and visualising trees. It provides a comprehensive API and a collection of command line tools, including utilities to work with the NCBI taxonomy tree.
http://etetoolkit.org
GNU General Public License v3.0
782 stars 212 forks source link

ete3 build fails on 64bit Debian/Ubuntu because of t-coffee #658

Open marco-mariotti opened 1 year ago

marco-mariotti commented 1 year ago

Hi, I've been struggling to run ete3 build in my Ubuntu 20.04.3 64bit machine. I finally solved the problem and I share the solution here.

The problem is that, after installing ete3 with conda (conda install -c etetoolkit ete3 ete_toolchain), when running even just: ete3 build check The command hangs, and a process (t_coffee) raises in memory usage until it is all eaten up.

This is due to a known issue of t_coffee related to MAX_N_PID. There is a workaround, however, which is to wrap t_coffee to set an env variable (MAX_N_PID_4_TCOFFEE) to prevent the problem: see implementation here (manual fix) and here (included in bioconda recipe for t-coffee).

Manual fix for those with the same issue: 1) download precompiled t_coffee (or install it from source) and set the workaround, following instructions in this comment 2) replace the ete_toolchain t_coffee with a link to the now functional t_coffee. For example, if you're in miniconda and your environment called ete3, and your functional t_coffee is ~/.t_coffee/bin/linux/t_coffee, this will do: mv ~/miniconda3/envs/ete3/bin/ete3_apps/bin/t_coffee ~/miniconda3/envs/ete3/bin/ete3_apps/bin/t_coffee.bug && ln -s ~/.t_coffee/bin/linux/t_coffee ~/miniconda3/envs/ete3/bin/ete3_apps/bin/t_coffee

In all cases, ete3 should implement a long term solution. In that regard, some useful information below.

There are two (bio)conda recipes for t_coffee that I know of, called "t_coffee", currently linking to version 11, and "t-coffee", linking to version 13. Only the second one has the workaround for the issue above. Ete_toolchain currently installs version 11 (perhaps it uses bioconda "t_coffee" recipe?). Unfortunately, the fix described above can't work at all for version 11, since t_coffee did not read MAX_N_PID_4_TCOFFEE back then.

So ete3 should install t_coffee version 13. While an apparently straightforward solution would seem to use the bioconda "t-coffee" recipe, this is a dependencies-heavy recipe (it's for the whole t_coffee "universe"), and has conflicts with other ete3 apps. So perhaps it is easier to adapt the current procedure used in ete_toolchain to install a newer (but light) t_coffee, then implement the workaround in the recipe like bioconda does.