subprocess.CalledProcessError

Hi there,

A subprocess.CalledProcessError occurred when starting to call variants. Python version 3.8 and 3.7 did not make any difference. The jobs were submitted with LSF to a cluster (see blow script). I also forced the CPU number to 1, still the same error.

Any solutions fort that?

Thank you in advance, Shawn

By the way, "conda install -c bioconda mutmap" doesn't work, as 'samtools' and/or 'bcftools' keep giving the following error samtools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory

So I installed mutmap and dependencies manually. Version info as below:

(python3.7) [qtclab1@loginview03 MutMap-master]$ bcftools --version bcftools 1.3.1 Using htslib 1.3.1 Copyright (C) 2016 Genome Research Ltd. License Expat: The MIT/Expat license This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

(python3.7) [qtclab1@loginview03 MutMap-master]$ samtools --version samtools 1.3.1 Using htslib 1.3.1 Copyright (C) 2016 Genome Research Ltd.

The stout was like:

[MutMap:2020-09-21 09:45:08] start to run MutMap. [MutMap:2020-09-21 09:45:08] maximum number of threads which you can use is up to 16. [MutMap:2020-09-21 09:45:08] start to index reference fasta. [MutMap:2020-09-21 09:47:07] indexing of reference successfully finished. [MutMap:2020-09-21 09:47:07] start to filter reads. [MutMap:2020-09-21 10:06:36] filtering process successfully finished. [MutMap:2020-09-21 10:06:36] start to merge BAMs. [MutMap:2020-09-21 10:18:37] merging process successfully finished. [MutMap:2020-09-21 10:18:37] start to call variants. [MutMap:2020-09-21 10:18:38] !!ERROR!! bcftools mpileup -a AD,ADF,ADR -B -q 40 -Q 18 -C 50 -O u -r 4 -f / Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA11C3G337_new_python3.7/10_ref/Arabidop sis_thaliana.TAIR10.dna.toplevel.fa /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA1 1C3G337_new_python3.7/20_bam/cultivar.filt.bam /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutma p/CNUsongA11C3G337_new_python3.7/20_bam/bulk.filt.bam | bcftools call -vm -f GQ,GP -O u | bcftools filter -i "INFO/MQ>=40" -O z -o /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA11C3G337_ne w_python3.7/30_vcf/mutmap.4.vcf.gz >> /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsong A11C3G337_new_python3.7/log/bcftools.4.log 2>&1

The sterr was like:

multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/Share2/home/qtclab1/apps/MutMap-master/mutmap/mpileup.py", line 143, in mpileup check=True) File "/Share2/home/qtclab1/anaconda3/envs/python3.7/lib/python3.7/subprocess.py", line 512, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command 'bcftools mpileup -a AD,ADF,ADR -B -q 40 -Q 18 -C 50 -O u -r 1 -f /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA11C3G337_new_python3.7/10_ref/Arabidopsis_thaliana.TAIR10.dna.toplevel.fa /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA11C3G337_new_python3.7/20_bam/cultivar.filt.bam /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA11C3G337_new_python3.7/20_bam/bulk.filt.bam | bcftools call -vm -f GQ,GP -O u | bcftools filter -i "INFO/MQ>=40" -O z -o /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA11C3G337_new_python3.7/30_vcf/mutmap.1.vcf.gz >> /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA11C3G337_new_python3.7/log/bcftools.1.log 2>&1' returned non-zero exit status 255.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Share2/home/qtclab1/anaconda3/envs/python3.7/lib/python3.7/multiprocessing/pool.py", line 121, in worker result = (True, func(*args, *kwds)) File "/Share2/home/qtclab1/anaconda3/envs/python3.7/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar return list(map(args)) File "/Share2/home/qtclab1/apps/MutMap-master/mutmap/mpileup.py", line 145, in mpileup call_log(self.out, 'bcftools', cmd1) File "/Share2/home/qtclab1/apps/MutMap-master/mutmap/utils.py", line 19, in call_log with open('{}/log/{}.log'.format(out_dir, name)) as log: FileNotFoundError: [Errno 2] No such file or directory: '/Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA11C3G337_new_python3.7/log/bcftools.log' """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Share2/home/qtclab1/anaconda3/envs/python3.7/bin/mutmap", line 33, in sys.exit(load_entry_point('mutmap', 'console_scripts', 'mutmap')()) File "/Share2/home/qtclab1/apps/MutMap-master/mutmap/mutmap.py", line 152, in main MutMap(args).run() File "/Share2/home/qtclab1/apps/MutMap-master/mutmap/mutmap.py", line 147, in run self.mpileup() File "/Share2/home/qtclab1/apps/MutMap-master/mutmap/mutmap.py", line 107, in mpileup mp.run() File "/Share2/home/qtclab1/apps/MutMap-master/mutmap/mpileup.py", line 228, in run p.map(self.mpileup, chr_names) File "/Share2/home/qtclab1/anaconda3/envs/python3.7/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/Share2/home/qtclab1/anaconda3/envs/python3.7/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value FileNotFoundError: [Errno 2] No such file or directory: '/Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/CNUsongA11C3G337_new_python3.7/log/bcftools.log'

The job submission script:

 15 #BSUB -J Mutmap
 16 #BSUB -q TEST-A
 17 #BSUB -e /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/00.log/%J.err
 18 #BSUB -o /Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap/00.log/%J.out
 19 #BSUB -n 4
 20 #BSUB -R "span[hosts=1]"
 21
 22 source /Share2/home/qtclab1/.bashrc
 23 conda deactivate
 24 conda activate python3.7
 25 conda env list
 26
 27
 28 REF="/Share2/home/qtclab1/00.users/shang_gao/00.reference"
 29 INPUT="/Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap"
 30 OUTPUT="/Share2/home/qtclab1/00.users/shang_gao/01.SongSS/01.mutmap"
 31
 32 mutmap \
 33 -t 4 \
 34 -r $REF/Arabidopsis_thaliana.TAIR10.dna.toplevel.fa \
 35 -c $INPUT/CNUsongA11C3G337/20_bam/cultivar.filt.bam \
 36 -b $INPUT/CNUsongA11C3G337/20_bam/bulk.filt.bam \
 37 -n 20 \
 38 -o $OUTPUT/CNUsongA11C3G337_new_python3.7

YuSugihara / MutMap

subprocess.CalledProcessError #5