KChen-lab / Monopogen

SNV calling from single cell sequencing
GNU General Public License v3.0
80 stars 17 forks source link

Issues for germline variant calling part from scRNA-seq #36

Closed lijinw closed 4 months ago

lijinw commented 8 months ago

Hi, Thank you for developing such a wonderful tool. I have faced a issue when I running the germline variant calling part of your tutorial. Actually I have succeeded running chromosome 1 and 20 from my data with the guidance of your tutorial. So I try to run 5 chromosomes (chr12-16) at the same time. But there are some errors occurred.

Could you please provide some guidance on how to resolve this issue? Thank you in advance for your assistance.

My code

source activate gva path="/data/ouyangjfc/home/gmslijiw/Monopogen" export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${path}/apps python ${path}/src/Monopogen.py germline \ -a ${path}/apps -t 5 -r /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+_past/region_12_16.lst \ -p /data/ouyangjfc/home/gmslijiw/Monopogen/reference/1KG3/ \ -g /data/ouyangjfc/home/gmslijiw/Monopogen/reference/hg38.analysisSet.fa -m 3 -s all -o /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+_past

Error Message

[2024-01-25 17:04:08,041] INFO Monopogen.py Performing germline variant calling... [2024-01-25 17:04:08,041] INFO germline.py Parameters in effect: [2024-01-25 17:04:08,041] INFO germline.py --subcommand = [germline] [2024-01-25 17:04:08,042] INFO germline.py --region = [/data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/region_12_16.lst] [2024-01-25 17:04:08,042] INFO germline.py --step = [all] [2024-01-25 17:04:08,042] INFO germline.py --out = [/data/ouyangjfc/home/gmslijiw/Monopogen/CD34+] [2024-01-25 17:04:08,042] INFO germline.py --reference = [/data/ouyangjfc/home/gmslijiw/Monopogen/reference/hg38.analysisSet.fa] [2024-01-25 17:04:08,042] INFO germline.py --imputation_panel = [/data/ouyangjfc/home/gmslijiw/Monopogen/reference/1KG3/] [2024-01-25 17:04:08,042] INFO germline.py --max_softClipped = [3] [2024-01-25 17:04:08,042] INFO germline.py --app_path = [/data/ouyangjfc/home/gmslijiw/Monopogen/apps] [2024-01-25 17:04:08,042] INFO germline.py --nthreads = [8] [2024-01-25 17:04:08,042] INFO germline.py --norun = [FALSE] [2024-01-25 17:04:08,042] INFO Monopogen.py Checking existence of essenstial resource files... [2024-01-25 17:04:08,068] INFO Monopogen.py Checking dependencies... /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/.filter.bam.lst: No such file or directory [mpileup] 1 samples in 1 input files [mpileup] 1 samples in 1 input files [mpileup] 1 samples in 1 input files [mpileup] 1 samples in 1 input files [mpileup] 1 samples in 1 input files Failed to open -: unknown file type Failed to open -: unknown file type Exception in thread "main" java.lang.IllegalArgumentException: missing value in key-value pair: chrom= at blbutil.Validate.argsToMap(Validate.java:75) at main.Par.(Par.java:100) at main.Main.parameters(Main.java:388) at main.Main.main(Main.java:104) gzip: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/germline/.gp.vcf.gz: No such file or directory /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/germline/.gp.vcf.gz: No such file or directory Exception in thread "main" java.lang.IllegalArgumentException: missing value in key-value pair: chrom= at blbutil.Validate.argsToMap(Validate.java:75) at main.Par.(Par.java:100) at main.Main.parameters(Main.java:388) at main.Main.main(Main.java:104) (mpileup) Max depth is above 1M. Potential memory hog! (mpileup) Max depth is above 1M. Potential memory hog! (mpileup) Max depth is above 1M. Potential memory hog! (mpileup) Max depth is above 1M. Potential memory hog! (mpileup) Max depth is above 1M. Potential memory hog! Lines total/split/realigned/skipped: 40866257/38219/3819/0 Lines total/split/realigned/skipped: 52806980/61472/4782/0 Lines total/split/realigned/skipped: 52742649/56641/5151/0 Lines total/split/realigned/skipped: 53210593/63155/5514/0 Lines total/split/realigned/skipped: 77572219/99785/7434/0 [2024-01-25 18:09:36,625] INFO Monopogen.py Success! See instructions above.

jinzhuangdou commented 8 months ago

It shows /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/.filter.bam.lst: No such file or directory. Could you let me know which files included in the Bam folder?

lijinw commented 8 months ago

Hi @jinzhuangdou,

Thanks for your reply. Actually, I already found the issue with the germline function, which is caused by the empty lines in the region.lst file. So I have finished germline calling for all chromosome except chr6. When I submit the job, it will never end. My code

!/bin/bash

PBS -q super

PBS -l walltime=12:00:00

PBS -l select=1:ncpus=20:mem=100gb

PBS -N runGermline_6

PBS -S /bin/bash

source activate gva path="/data/ouyangjfc/home/gmslijiw/Monopogen" export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${path}/apps python ${path}/src/Monopogen.py germline \ -a ${path}/apps -t 8 -r /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/region_6.lst \ -p /data/ouyangjfc/home/gmslijiw/Monopogen/reference/1KG3/ \ -g /data/ouyangjfc/home/gmslijiw/Monopogen/reference/hg38.analysisSet.fa -m 3 -s all -o /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+

Error Message

[2024-02-01 16:33:11,386] INFO Monopogen.py Performing germline variant calling... [2024-02-01 16:33:11,386] INFO germline.py Parameters in effect: [2024-02-01 16:33:11,386] INFO germline.py --subcommand = [germline] [2024-02-01 16:33:11,387] INFO germline.py --region = [/data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/region_6.lst] [2024-02-01 16:33:11,387] INFO germline.py --step = [all] [2024-02-01 16:33:11,387] INFO germline.py --out = [/data/ouyangjfc/home/gmslijiw/Monopogen/CD34+] [2024-02-01 16:33:11,387] INFO germline.py --reference = [/data/ouyangjfc/home/gmslijiw/Monopogen/reference/hg38.analysisSet.fa] [2024-02-01 16:33:11,387] INFO germline.py --imputation_panel = [/data/ouyangjfc/home/gmslijiw/Monopogen/reference/1KG3/] [2024-02-01 16:33:11,387] INFO germline.py --max_softClipped = [3] [2024-02-01 16:33:11,387] INFO germline.py --app_path = [/data/ouyangjfc/home/gmslijiw/Monopogen/apps] [2024-02-01 16:33:11,387] INFO germline.py --nthreads = [8] [2024-02-01 16:33:11,387] INFO germline.py --norun = [FALSE] [2024-02-01 16:33:11,387] INFO Monopogen.py Checking existence of essenstial resource files... [2024-02-01 16:33:11,408] INFO Monopogen.py Checking dependencies... [mpileup] 1 samples in 1 input files (mpileup) Max depth is above 1M. Potential memory hog! Lines total/split/realigned/skipped: 96453302/110330/9130/0 Exception in thread "Thread-2" java.lang.RuntimeException: java.util.zip.DataFormatException: invalid distance too far back at net.sf.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:112) at net.sf.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:383) at net.sf.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:365) at net.sf.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:109) at net.sf.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:238) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.readLine(BufferedReader.java:324) at java.io.BufferedReader.readLine(BufferedReader.java:389) at blbutil.InputIt.next(InputIt.java:120) at blbutil.InputIt.next(InputIt.java:48) at vcf.RefIt.readLine(RefIt.java:288) at vcf.RefIt.lambda$fileReadingThread$15(RefIt.java:168) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.zip.DataFormatException: invalid distance too far back at java.util.zip.Inflater.inflateBytes(Native Method) at java.util.zip.Inflater.inflate(Inflater.java:259) at net.sf.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96) ... 16 more =>> PBS: job killed: walltime 43229 exceeded limit 43200

Beside this issue, I also face a problem when I run the cellScan functiom. My featureInfo output looks the same as your tutorial. But there are some errors when I run the cellScan function. Your support would be greatly appreciated. Thank you in advance for your time and consideration.

My code for cellScan

!/bin/bash

PBS -q super

PBS -l walltime=12:00:00

PBS -l select=1:ncpus=20:mem=100gb

PBS -N 02_runcellScan_12_16

PBS -S /bin/bash

source activate gva path="/data/ouyangjfc/home/gmslijiw/Monopogen" export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${path}/apps python ${path}/src/Monopogen.py somatic \ -a ${path}/apps -r /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/region_12_16.lst -t 22 -w 10MB \ -i /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+ \ -l /data/ouyangjfc/home/gmslijiw/Monopogen/reference/BM157barcodes_revised.csv -s cellScan \ -g /data/ouyangjfc/home/gmslijiw/Monopogen/reference/hg38.analysisSet.fa

My error message for cellScan

[2024-02-02 10:27:32,129] INFO Monopogen.py Get single cell level information from sequencing data... [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success task = job = result = func = args = kwds = None Could you please give me some guidance to solve this problem? Thank you in advance for your time and consideration. OSError: [Errno 0] Success [W::hts_idx_load3] [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filte$

[E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.bai [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] Inflate operation failed: progress temporarily not possible, or in() / out() returned an error [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] Inflate operation failed: progress temporarily not possible, or in() / out() returned an error [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] Inflate operation failed: progress temporarily not possible, or in() / out() returned an error [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] Inflate operation failed: progress temporarily not possible, or in() / out() returned an error [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] CRC32 checksum mismatch [E::bgzf_read] Read block operation failed with error 33 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.bam.$ [E::bgzf_uncompress] Inflate operation failed: invalid distance too far back [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.b$ [E::bgzf_uncompress] Inflate operation failed: invalid distance too far back [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.b$ [E::bgzf_uncompress] Inflate operation failed: invalid distance too far back [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.b$ [E::bgzf_uncompress] Inflate operation failed: invalid distance too far back [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.b$ [E::bgzf_uncompress] Inflate operation failed: invalid distance too far back [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success [W::hts_idx_load3] The index file is older than the data file: /data/ouyangjfc/home/gmslijiw/Monopogen/CD34+/Bam/merge.filter.targeted.b$ [E::bgzf_uncompress] Inflate operation failed: invalid distance too far back [E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes OSError: [Errno 0] Success Exception ignored in: 'pysam.libcalignmentfile.AlignmentFile.dealloc' Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 138, in worker task = job = result = func = args = kwds = None OSError: [Errno 0] Success

multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, *kwds)) File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar return list(map(args)) File "/data/ouyangjfc/home/gmslijiw/Monopogen/src/somatic.py", line 179, in bamSplit for s in infile: File "pysam/libcalignmentfile.pyx", line 1877, in pysam.libcalignmentfile.AlignmentFile.next OSError: truncated file """ The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/data/ouyangjfc/home/gmslijiw/Monopogen/src/Monopogen.py", line 436, in main() File "/data/ouyangjfc/home/gmslijiw/Monopogen/src/Monopogen.py", line 429, in main args.func(args) File "/data/ouyangjfc/home/gmslijiw/Monopogen/src/Monopogen.py", line 195, in somatic result = pool.map(bamSplit, joblst) File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 364, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/data/ghosh/home/gmslijiw/miniconda3/envs/gva/lib/python3.9/multiprocessing/pool.py", line 771, in get raise self._value OSError: truncated file

jinzhuangdou commented 7 months ago

We have updated a new version on cellScan step which is computationally efficient. Please try it.