search

Hoohm / dropSeqPipe

A SingleCell RNASeq pre-processing snakemake workflow
Creative Commons Attribution Share Alike 4.0 International
147 stars 47 forks source link

Create_star_index failing #66

Closed Hofphi closed 5 years ago

Hofphi commented 5 years ago

Hey, I tried to run snakemake --cores 12 qc filter map for MixedSpecies experiment but get an error while creating the star index. Unfortunately the error message is not telling me much, maybe there is one of you that has a little more experience with this kind of output!

Here the snakemake.log


Building DAG of jobs...
Using shell: /cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Job counts:
        count   jobs
        1       MergeBamAlignment
        1       STAR_align
        1       TagReadWithGeneExon
        1       bam_hist
        1       bead_errors_metrics
        1       create_dict
        1       create_refFlat
        1       create_star_index
        1       extract_reads_expression
        1       extract_umi_expression
        1       map
        1       merge_counts
        1       merge_umi
        1       multiqc_star
        1       plot_knee_plot
        1       plot_yield
        1       sort_sam
        1       violine_plots
        18

[Fri Dec 14 03:21:46 2018]
rule create_star_index:
    input: ~/scratch/dropSeqPipe/ref_genome/hg19_mm10/hg19_mm10_transgenes.fasta,~/scratch/dropSeqPipe/ref_genome/hg19_mm10/hg19_mm10_transgenes.gtf
    output: ~/scratch/dropSeqPipe/ref_genome/hg19_mm10/STAR_INDEX/SA_83/SA
    jobid: 35
    wildcards: star_index_prefix=~/scratch/dropSeqPipe/ref_genome/hg19_mm10/STAR_INDEX/SA, read_length=83
    threads: 8

Activating conda environment: ~/dropSeqPipe/.snakemake/conda/fe458c9e
[Fri Dec 14 04:05:08 2018]
Error in rule create_star_index:
    jobid: 35
    output: ~/dropSeqPipe/ref_genome/hg19_mm10/STAR_INDEX/SA_83/SA
    conda-env: ~/dropSeqPipe/.snakemake/conda/fe458c9e

RuleException:
CalledProcessError in line 107 of ~/dropSeqPipe/rules/generate_meta.smk:
Command 'source activate ~/dropSeqPipe/.snakemake/conda/fe458c9e; set -euo pipefail;  mkdir -p ~/scratch/dropSeqPipe/ref_genome/hg19_mm10/STAR_INDEX/SA_83; STAR           --runThreadN 8          --runMode genomeGenerate                --genomeDir ~/scratch/dropSeqPipe/ref_genome/hg19_mm10/STAR_INDEX/SA_83            --genomeFastaFiles ~/scratch/dropSeqPipe/ref_genome/hg19_mm10/hg19_mm10_transgenes.fasta           --sjdbGTFfile ~/scratch/dropSeqPipe/ref_genome/hg19_mm10/hg19_mm10_transgenes.gtf          --limitGenomeGenerateRAM 30000000000            --sjdbOverhang 82               --genomeChrBinNbits 18 ' returned non-zero exit status 137.
  File "~/dropSeqPipe/rules/generate_meta.smk", line 107, in __rule_create_star_index
  File "~/miniconda3/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Hoohm commented 5 years ago

Hey @Hofphi I found out that exit status 137 is linked to memory issues.

My guess is that your combined ref is too big for the limit that is imposed in STAR or the amount of memory the computer your using has.

How much ram do you have? Under 32 I would not try a mixed ref.

If you have 32, I would advise removing the limitation --limitGenomeGenerateRAM 30000000000\ and run it again.

Hofphi commented 5 years ago

I can not try what you suggest @Hoohm since my dropSeqPipe-folder on the sever got erased...I had to set up everything again and now I get this error:

Building DAG of jobs...
Creating conda environment https:/bitbucket.org/snakemake/snakemake-wrappers/raw/0.27.1/bio/fastqc/environment.yaml...
Downloading remote packages.
CreateCondaEnvironmentException:
Could not create conda environment from /tmp/tmpa4we5re8.yaml:
Solving environment: ...working... done

Followed by that: 

Downloading and Extracting Packages
Preparing transaction: ...working... done
Verifying transaction: ...working... failed

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/exceptions.py", line 819, in __call__
        return func(*args, **kwargs)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda_env/cli/main.py", line 74, in do_call
        exit_code = getattr(module, func_name)(args, parser)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda_env/cli/main_create.py", line 108, in execute
        installer.install(prefix, pkg_specs, args, env)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda_env/installers/conda.py", line 39, in install
        unlink_link_transaction.execute()
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 223, in execute
        self.verify()
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/common/io.py", line 46, in decorated
        return f(*args, **kwds)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 210, in verify
        exceptions = self._verify(self.prefix_setups, self.prefix_action_groups)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 486, in _verify
        cls._verify_transaction_level(prefix_setups),
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 481, in <genexpr>
        exceptions = tuple(exc for exc in concatv(
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 334, in _verify_individual_level
        error_result = axn.verify()
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/path_actions.py", line 428, in verify
        create_link(self.source_full_path, self.intermediate_path, LinkType.copy)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/gateways/disk/create.py", line 334, in create_link
        copy(src, dst)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/gateways/disk/create.py", line 272, in copy
        _do_copy(src, dst)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/gateways/disk/create.py", line 286, in _do_copy
        copyfileobj(fsrc, fdst, buffer_size)
    ProcessLookupError: [Errno 3] No such process

`$ /home/hofphi00/miniconda3/bin/conda-env create --file /scratch/hofphi00/dropSeqPipe/.snakemake/conda/94f1b5d7.yaml --prefix /scratch/hofphi00/dropSeqPipe/.snakemake/conda/94f1b5d7`

  environment variables:
             ACLOCAL_PATH=/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/share/aclocal
                 CIO_TEST=<not set>
 CMAKE_SYSTEM_PREFIX_PATH=/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09
  CONDA_AUTO_UPDATE_CONDA=false
               CONDA_ROOT=/home/hofphi00/miniconda3
                    CPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/int
                          el2016.4/openmpi/2.1.1/include:/cvmfs/soft.computecanada.ca/easybuild/
                          software/2017/Core/imkl/11.3.4.258/mkl/include/fftw:/cvmfs/soft.comput
                          ecanada.ca/easybuild/software/2017/Core/imkl/11.3.4.258/mkl/include:/c
                          vmfs/soft.computecanada.ca/nix/var/nix/profiles/gcc-5.4.0/include
       CPLUS_INCLUDE_PATH=/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/include
           C_INCLUDE_PATH=/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/include
      EASYBUILD_BUILDPATH=/dev/shm/hofphi00
LD_LIBRARY_PATH=/opt/software/lib:/opt/software/slurm/lib
             LIBRARY_PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/int
                          el2016.4/openmpi/2.1.1/lib:/cvmfs/soft.computecanada.ca/easybuild/soft
                          ware/2017/Core/imkl/11.3.4.258/mkl/lib/intel64:/cvmfs/soft.computecana
                          da.ca/easybuild/software/2017/Core/imkl/11.3.4.258/lib/intel64:/cvmfs/
                          soft.computecanada.ca/easybuild/software/2017/Core/ifort/2016.4.258/co
                          mpilers_and_libraries_2016.4.258/linux/compiler/lib/intel64:/cvmfs/sof
                          t.computecanada.ca/nix/var/nix/profiles/gcc-5.4.0/lib64:/cvmfs/soft.co
                          mputecanada.ca/nix/var/nix/profiles/gcc-5.4.0/lib:/cvmfs/soft.computec
                          anada.ca/easybuild/software/2017/Core/icc/2016.4.258/compilers_and_lib
                          raries_2016.4.258/linux/compiler/lib/intel64:/cvmfs/soft.computecanada
                          .ca/nix/var/nix/profiles/16.09/lib
        LMOD_PACKAGE_PATH=/cvmfs/soft.computecanada.ca/config/lmod/
                  MANPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/int
                          el2016.4/openmpi/2.1.1/share/man:/cvmfs/soft.computecanada.ca/easybuil
                          d/software/2017/Core/ifort/2016.4.258/compilers_and_libraries_2016.4.2
                          58/linux/man/common:/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/
                          gcc-5.4.0/share/man:/cvmfs/soft.computecanada.ca/easybuild/software/20
                          17/Core/icc/2016.4.258/compilers_and_libraries_2016.4.258/linux/man/co
                          mmon:/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/share/man
                          :/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/lmod/lmod/sha
                          re/man::
      MIC_LD_LIBRARY_PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/imkl/11.3.4.
                          258/mkl/lib/mic:/cvmfs/soft.computecanada.ca/easybuild/software/2017/C
                          ore/imkl/11.3.4.258/lib/intel64_lin_mic
               MODULEPATH=/opt/software/modulefiles:/cvmfs/soft.computecanada.ca/easybuild/modul
                          es/2017/avx2/MPI/intel2016.4/openmpi2.1:/cvmfs/soft.computecanada.ca/e
                          asybuild/modules/2017/avx2/Compiler/intel2016.4:/home/hofphi00/.local/
                          easybuild/modules/2017/Core:/cvmfs/soft.computecanada.ca/easybuild/mod
                          ules/2017/Core:/cvmfs/soft.computecanada.ca/custom/modules
                  NLSPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/imkl/11.3.4.
                          258/idb/intel64/locale/%l_%t/%N:/cvmfs/soft.computecanada.ca/easybuild
                          /software/2017/Core/ifort/2016.4.258/idb/intel64/locale/%l_%t/%N:/cvmf
                          s/soft.computecanada.ca/easybuild/software/2017/Core/icc/2016.4.258/id
                          b/intel64/locale/%l_%t/%N
                     PATH=/home/hofphi00/miniconda3/bin:/home/hofphi00/miniconda3/bin:/cvmfs/sof
                          t.computecanada.ca/easybuild/software/2017/avx2/Compiler/intel2016.4/o
                          penmpi/2.1.1/bin:/cvmfs/soft.computecanada.ca/easybuild/software/2017/
                          Core/imkl/11.3.4.258/mkl/bin:/cvmfs/soft.computecanada.ca/easybuild/so
                          ftware/2017/Core/imkl/11.3.4.258/bin:/cvmfs/soft.computecanada.ca/easy
                          build/software/2017/Core/ifort/2016.4.258/compilers_and_libraries_2016
                          .4.258/linux/bin/intel64:/cvmfs/soft.computecanada.ca/nix/var/nix/prof
                          iles/gcc-5.4.0/bin:/cvmfs/soft.computecanada.ca/easybuild/software/201
                          7/Core/icc/2016.4.258/compilers_and_libraries_2016.4.258/linux/bin/int
                          el64:/cvmfs/soft.computecanada.ca/easybuild/bin:/cvmfs/soft.computecan
                          ada.ca/custom/bin:/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16
                          .09/bin:/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/sbin:/
                          opt/software/bin:/opt/software/slurm/bin:/usr/local/bin:/usr/bin:/usr/
                          local/sbin:/usr/sbin:/opt/puppetlabs/bin:/opt/dell/srvadmin/bin:/home/
                          hofphi00/.local/bin:/home/hofphi00/bin
          PKG_CONFIG_PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/int
                          el2016.4/openmpi/2.1.1/lib/pkgconfig:/cvmfs/soft.computecanada.ca/nix/
                          var/nix/profiles/16.09/lib/pkgconfig
       PYTHONPATH=/cvmfs/soft.computecanada.ca/custom/python/site-packages
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt
__LMOD_Priority_MODULEPATH=/opt/software/modulefiles:-10
__LMOD_REF_COUNT_MODULEPATH=/opt/software/modulefiles:1;/cvmfs/soft.computecanada.ca/easybuild/mod
                          ules/2017/avx2/MPI/intel2016.4/openmpi2.1:1;/cvmfs/soft.computecanada.
                          ca/easybuild/modules/2017/avx2/Compiler/intel2016.4:1;/home/hofphi00/.
                          local/easybuild/modules/2017/Core:1;/cvmfs/soft.computecanada.ca/easyb
                          uild/modules/2017/Core:1;/cvmfs/soft.computecanada.ca/custom/modules:1

     active environment : None
       user config file : /home/hofphi00/.condarc
 populated config files : /home/hofphi00/.condarc
          conda version : 4.5.12
    conda-build version : not installed
         python version : 3.6.7.final.0
       base environment : /home/hofphi00/miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /home/hofphi00/miniconda3/pkgs
                          /home/hofphi00/.conda/pkgs
       envs directories : /home/hofphi00/miniconda3/envs
                          /home/hofphi00/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.12 requests/2.21.0 CPython/3.6.7 Linux/3.10.0-862.14.4.el7.x86_64 centos/7 glibc/2.17
                UID:GID : 3066188:3066188
             netrc file : None
           offline mode : False

An unexpected error has occurred. Conda has prepared the above report.

I tried to remove miniconda and updating

conda remove --all
conda update conda
conda update --all --yes

I also installed a fresh snakemake. I am really not sure what causes the issue

Hoohm commented 5 years ago

This is really strange. Why was your folder deleted?

It seems like conda can't write the files of the envs. Is there enough space to write up all that you need?

what happens if you run this: /home/hofphi00/miniconda3/bin/conda-env create --file /scratch/hofphi00/dropSeqPipe/.snakemake/conda/94f1b5d7.yaml --prefix /scratch/hofphi00/dropSeqPipe/.snakemake/conda/94f1b5d7

Hofphi commented 5 years ago

Hey @Hoohm

I installed the dropSeqPipe-pipeline locally and resolved the create star_index issue that I had originally. Your proposed solution by deleting the --limitGenomeGenerateRAM 30000000000\ worked great!

Regarding the pipeline on the cluster I got the following error message after running the command above (I certainly have enough space in this directory): 

/home/hofphi00/miniconda3/bin/conda-env create --file /scratch/hofphi00/dropSeqPipe/.snakemake/conda/94f1b5d7.yaml --prefix /scratch/hofphi00/dropSeqPipe/.snakemake/conda/94f1b5d7

Solving environment: done
Preparing transaction: done
Verifying transaction: failed

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/exceptions.py", line 819, in __call__
        return func(*args, **kwargs)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda_env/cli/main.py", line 74, in do_call
        exit_code = getattr(module, func_name)(args, parser)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda_env/cli/main_create.py", line 108, in execute
        installer.install(prefix, pkg_specs, args, env)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda_env/installers/conda.py", line 39, in install
        unlink_link_transaction.execute()
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 223, in execute
        self.verify()
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/common/io.py", line 46, in decorated
        return f(*args, **kwds)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 210, in verify
        exceptions = self._verify(self.prefix_setups, self.prefix_action_groups)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 486, in _verify
        cls._verify_transaction_level(prefix_setups),
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 481, in <genexpr>
        exceptions = tuple(exc for exc in concatv(
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/link.py", line 334, in _verify_individual_level
        error_result = axn.verify()
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/core/path_actions.py", line 428, in verify
        create_link(self.source_full_path, self.intermediate_path, LinkType.copy)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/gateways/disk/create.py", line 334, in create_link
        copy(src, dst)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/gateways/disk/create.py", line 272, in copy
        _do_copy(src, dst)
      File "/home/hofphi00/miniconda3/lib/python3.6/site-packages/conda/gateways/disk/create.py", line 286, in _do_copy
        copyfileobj(fsrc, fdst, buffer_size)
    ProcessLookupError: [Errno 3] No such process

`$ /home/hofphi00/miniconda3/bin/conda-env create --file /scratch/hofphi00/dropSeqPipe/.snakemake/conda/94f1b5d7.yaml --prefix /scratch/hofphi00/dropSeqPipe/.snakemake/conda/94f1b5d7`

  environment variables:
             ACLOCAL_PATH=/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/share/aclocal
                 CIO_TEST=<not set>
 CMAKE_SYSTEM_PREFIX_PATH=/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09
  CONDA_AUTO_UPDATE_CONDA=false
               CONDA_ROOT=/home/hofphi00/miniconda3
                    CPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/int
                          el2016.4/openmpi/2.1.1/include:/cvmfs/soft.computecanada.ca/easybuild/
                          software/2017/Core/imkl/11.3.4.258/mkl/include/fftw:/cvmfs/soft.comput
                          ecanada.ca/easybuild/software/2017/Core/imkl/11.3.4.258/mkl/include:/c
                          vmfs/soft.computecanada.ca/nix/var/nix/profiles/gcc-5.4.0/include
       CPLUS_INCLUDE_PATH=/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/include
           C_INCLUDE_PATH=/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/include
      EASYBUILD_BUILDPATH=/dev/shm/hofphi00
          LD_LIBRARY_PATH=/opt/software/lib:/opt/software/slurm/lib
             LIBRARY_PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/int
                          el2016.4/openmpi/2.1.1/lib:/cvmfs/soft.computecanada.ca/easybuild/soft
                          ware/2017/Core/imkl/11.3.4.258/mkl/lib/intel64:/cvmfs/soft.computecana
                          da.ca/easybuild/software/2017/Core/imkl/11.3.4.258/lib/intel64:/cvmfs/
                          soft.computecanada.ca/easybuild/software/2017/Core/ifort/2016.4.258/co
                          mpilers_and_libraries_2016.4.258/linux/compiler/lib/intel64:/cvmfs/sof
                          t.computecanada.ca/nix/var/nix/profiles/gcc-5.4.0/lib64:/cvmfs/soft.co
                          mputecanada.ca/nix/var/nix/profiles/gcc-5.4.0/lib:/cvmfs/soft.computec
                          anada.ca/easybuild/software/2017/Core/icc/2016.4.258/compilers_and_lib
                          raries_2016.4.258/linux/compiler/lib/intel64:/cvmfs/soft.computecanada
                          .ca/nix/var/nix/profiles/16.09/lib
        LMOD_PACKAGE_PATH=/cvmfs/soft.computecanada.ca/config/lmod/
                  MANPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/int
                          el2016.4/openmpi/2.1.1/share/man:/cvmfs/soft.computecanada.ca/easybuil
                          d/software/2017/Core/ifort/2016.4.258/compilers_and_libraries_2016.4.2
                          58/linux/man/common:/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/
                          gcc-5.4.0/share/man:/cvmfs/soft.computecanada.ca/easybuild/software/20
                          17/Core/icc/2016.4.258/compilers_and_libraries_2016.4.258/linux/man/co
                          mmon:/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/share/man
                          :/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/lmod/lmod/sha
                          re/man::
      MIC_LD_LIBRARY_PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/imkl/11.3.4.
                          258/mkl/lib/mic:/cvmfs/soft.computecanada.ca/easybuild/software/2017/C
                          ore/imkl/11.3.4.258/lib/intel64_lin_mic
               MODULEPATH=/opt/software/modulefiles:/cvmfs/soft.computecanada.ca/easybuild/modul
                          es/2017/avx2/MPI/intel2016.4/openmpi2.1:/cvmfs/soft.computecanada.ca/e
                          asybuild/modules/2017/avx2/Compiler/intel2016.4:/home/hofphi00/.local/
                          easybuild/modules/2017/Core:/cvmfs/soft.computecanada.ca/easybuild/mod
                          ules/2017/Core:/cvmfs/soft.computecanada.ca/custom/modules
                  NLSPATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/imkl/11.3.4.
                          258/idb/intel64/locale/%l_%t/%N:/cvmfs/soft.computecanada.ca/easybuild
                          /software/2017/Core/ifort/2016.4.258/idb/intel64/locale/%l_%t/%N:/cvmf
                          s/soft.computecanada.ca/easybuild/software/2017/Core/icc/2016.4.258/id
                          b/intel64/locale/%l_%t/%N
                     PATH=/home/hofphi00/miniconda3/bin:/home/hofphi00/miniconda3/bin:/cvmfs/sof
                          t.computecanada.ca/easybuild/software/2017/avx2/Compiler/intel2016.4/o
                          penmpi/2.1.1/bin:/cvmfs/soft.computecanada.ca/easybuild/software/2017/
                          Core/imkl/11.3.4.258/mkl/bin:/cvmfs/soft.computecanada.ca/easybuild/so
                          ftware/2017/Core/imkl/11.3.4.258/bin:/cvmfs/soft.computecanada.ca/easy
                          build/software/2017/Core/ifort/2016.4.258/compilers_and_libraries_2016
                          .4.258/linux/bin/intel64:/cvmfs/soft.computecanada.ca/nix/var/nix/prof
                          iles/gcc-5.4.0/bin:/cvmfs/soft.computecanada.ca/easybuild/software/201
                          7/Core/icc/2016.4.258/compilers_and_libraries_2016.4.258/linux/bin/int
                          el64:/cvmfs/soft.computecanada.ca/easybuild/bin:/cvmfs/soft.computecan
                          ada.ca/custom/bin:/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16
                          .09/bin:/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09/sbin:/
                          opt/software/bin:/opt/software/slurm/bin:/usr/local/bin:/usr/bin:/usr/
                          local/sbin:/usr/sbin:/opt/puppetlabs/bin:/opt/dell/srvadmin/bin:/home/
                          hofphi00/.local/bin:/home/hofphi00/bin
          PKG_CONFIG_PATH=/cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/Compiler/int
                          el2016.4/openmpi/2.1.1/lib/pkgconfig:/cvmfs/soft.computecanada.ca/nix/
                          var/nix/profiles/16.09/lib/pkgconfig
               PYTHONPATH=/cvmfs/soft.computecanada.ca/custom/python/site-packages
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt
__LMOD_Priority_MODULEPATH=/opt/software/modulefiles:-10
__LMOD_REF_COUNT_MODULEPATH=/opt/software/modulefiles:1;/cvmfs/soft.computecanada.ca/easybuild/mod
                          ules/2017/avx2/MPI/intel2016.4/openmpi2.1:1;/cvmfs/soft.computecanada.
                          ca/easybuild/modules/2017/avx2/Compiler/intel2016.4:1;/home/hofphi00/.
                          local/easybuild/modules/2017/Core:1;/cvmfs/soft.computecanada.ca/easyb
                          uild/modules/2017/Core:1;/cvmfs/soft.computecanada.ca/custom/modules:1

     active environment : None
       user config file : /home/hofphi00/.condarc
 populated config files : 
          conda version : 4.5.12
    conda-build version : not installed
         python version : 3.6.7.final.0
       base environment : /home/hofphi00/miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /home/hofphi00/miniconda3/pkgs
                          /home/hofphi00/.conda/pkgs
       envs directories : /home/hofphi00/miniconda3/envs
                          /home/hofphi00/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.12 requests/2.21.0 CPython/3.6.7 Linux/3.10.0-862.14.4.el7.x86_64 centos/7 glibc/2.17
                UID:GID : 3066188:3066188
             netrc file : None
           offline mode : False
Hoohm commented 5 years ago

Is it possible that by removing conda you just left the call to conda and it can't find it? Normally when I remove/reinstall conda I just delete the miniconda3 folder and then run the bash installation again.

Hofphi commented 5 years ago

I also tried that but with no success. I already opened an issue-thread on the conda github because I could not find any entry regarding this error here:

Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: failed
ERROR conda.core.link:_execute(507): An error occurred while installing package 'conda-forge::libgcc-ng-7.2.0-hdf63c60_3'.
ProcessLookupError(3, 'No such process')
Attempting to roll back.

Rolling back transaction: done

ProcessLookupError(3, 'No such process')

I hope for help from them, so you can close this issue