Open LeoWelter opened 1 year ago
I have also encountered this problem.
Same issue for me: https://github.com/bcbio/bcbio-nextgen/issues/3686
@naumenko-sa any clue what's going on?
It would be great to bring back support for creating Docker images, given all these recurrent Conda issues for new installs.
Hi @amizeranschi ! Sorry for the delay: I am refactoring the whole installation script to address the install issues. SN
I was very fond of the strategy from bcbio_docker
: using a focused, compact tool.yaml
config file that installs only a small subset of the full swiss-army knife collection. This avoids the typical dependency hell of conda
, given that many tools in the standard installation seem to be unused or outdated.
https://github.com/bcbio/bcbio_docker/blob/19369dde53fe40e624b1086d19570b3024929567/Dockerfile.tools#L13
Will you be going in that direction, @naumenko-sa?
Hi @gabeng ! I am not sure about the whole CWL direction, however definitely I am going for small images for the typical analyses: variant calling and bulkRNA-seq. First of all I am just inspecting the installation process, removing unnecessary steps, unused packages, making sure PATH/PYTHONPATH is not an issue (the most frequent obstacle). SN
Hi @LeoWelter, @dnbuckley, @amizeranschi !
It looks like the perl related issue was due to https://github.com/chapmanb/cloudbiolinux/blob/master/contrib/flavor/ngs_pipeline_minimal/packages-conda.yaml#L181
My install command: .bcbio_devel_profie:
BCBIO_HOME=/path-tobcbio_install_test2
PATH=${BCBIO_HOME}/anaconda/bin:${BCBIO_HOME}/tools/bin
PATH=${PATH}:/home/user.aspera/connect/bin:/usr/local/bin:/usr/bin:/opt/puppetlabs/bin:/usr/local/sbin:/usr/sbin:/opt/ibutils/bin:/opt/dell/srvadmin/bin
export PATH
export CONDA_EXE=
export CONDA_PYTHON_EXE=
export CONDA_PREFIX=
export PYTHONPATH=
# don't add ~/.local to PYTHONPATH
export PYTHONNOUSERSITE=True
. .bcbio
$ which python3
/usr/bin/python3
$ python3 bcbio_nextgen_install.py /path-to/bcbio_install_test2 --tooldir=/path-to/bcbio_install_test2/tools --mamba --nodata
(please note --nodata installation, install data separately)
Please give it a try!
It passes the perl issue for me and installs all environments down to python3.6:
# Installing into conda environment python3.6: bowtie2, crossmap, cufflinks, htslib=1.7, optitype, pandas=0.25.3, pyloh, pysam=0.15.4, razers3, samtools=1.7, vt=2015.11.10=he941832_3
Encountered problems while solving:
- package libsqlite-3.40.0-h753d276_0 requires libzlib >=1.2.13,<1.3.0a0, but none of the providers can be installed
In my case, mamba solved that conflict and the installation finished successfully.
I'd also call the community to be more active in solving such bugs: the perl one wasn't too complicated. The way to debug them:
In this case, I also removed battenberg, since we are also using the frozen version of it from some time ago, and it pushes perl down. The battenberg (for subclones and CNV in cancer data) is a little subproject - one would need to have proper data to test the new battenberg, incorporate it into bcbio, update installation script.
SN
problem, installing from scratch on our HPC fails while trying to install a perl packages:
I have perl 5.26.3 as default on the system or 5.30.2 or 5.34.1 loadable as module. thanks for any advice in advance
Version info
bcbio_nextgen.py --version
): 1.2.9lsb_release -ds
):Red Hat Enterprise Linux 8.4To Reproduce Exact bcbio command you have used:
Your yaml configuration file:
Log files (could be found in work/log) Please attach (10MB max):
bcbio-nextgen-commands.log
, andbcbio-nextgen-debug.log
.