MGI-tech-bioinformatics / DNBelab_C_Series_HT_scRNA-analysis-software

An open source and flexible pipeline to analysis high-throughput DNBelab C Series single-cell RNA datasets
MIT License
72 stars 24 forks source link

dnbc4tools rna run 的时候报错:Signals.SIGABRT: 6 #83

Closed YaoQiong1127 closed 5 months ago

YaoQiong1127 commented 5 months ago

您好,我在使用 dnbc4tools rna run 的时候,出现报错,信息如下:

代码: export LD_LIBRARY_PATH=/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib:/HXsz1/YQ/Software/Miniconda3/lib:/usr/lib64:/usr/lib binscr=/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/bin/dnbc4tools db_dir=/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/database/Mouse-GRCm38 Sample=KO_5 cDNAfq1=/HXsz1/YQ/Project/scRNA-Seq.C4/testing/Raw_Data/E250013997_L01_KO_5_1.fq.gz cDNAfq2=/HXsz1/YQ/Project/scRNA-Seq.C4/testing/Raw_Data/E250013997_L01_KO_5_2.fq.gz Oligofq1=/HXsz1/YQ/Project/scRNA-Seq.C4/testing/Raw_Data/E250015259_L01_KO_5_oligo_1.fq.gz Oligofq2=/HXsz1/YQ/Project/scRNA-Seq.C4/testing/Raw_Data/E250015259_L01_KO_5_oligo_2.fq.gz $binscr rna run --name $Sample --cDNAfastq1 $cDNAfq1 --cDNAfastq2 $cDNAfq2 --oligofastq1 $Oligofq1 --oligofastq2 $Oligofq2 --genomeDir $db_dir --threads 8 --calling_method "emptydrops" --expectcells 5000

报错信息: Chemistry(darkreaction) determined in oligoR1 : v2v3nodarkreaction Chemistry(darkreaction) determined in oligoR2 : nodarkreaction Chemistry(darkreaction) determined in cDNAR1 : scRNAv2v3HT

2024-06-25 11:19:35 Conduct quality control for cDNA library barcoding, perform alignment.

2024-06-25 11:19:35 Perform quality control for oligo library barcodes.

Traceback (most recent call last): File "/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/bin/dnbc4tools", line 8, in sys.exit(main()) File "/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/dnbc4tools.py", line 110, in main args.func(args) File "/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/rna/run.py", line 144, in run Runpipe(args).runpipe() File "/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/rna/run.py", line 131, in runpipe start_print_cmd(pipecmd,os.path.join(self.outdir,self.name)) File "/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/tools/utils.py", line 138, in start_print_cmd subprocess.check_call(arg, shell=True) File "/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/subprocess.py", line 364, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/bin/dnbc4tools rna data --cDNAfastq1 /HXsz1/YQ/Project/scRNA-Seq.C4/testing/Raw_Data/E250013997_L01_KO_5_1.fq.gz --cDNAfastq2 /HXsz1/YQ/Project/scRNA-Seq.C4/testing/Raw_Data/E250013997_L01_KO_5_2.fq.gz --oligofastq1 /HXsz1/YQ/Project/scRNA-Seq.C4/testing/Raw_Data/E250015259_L01_KO_5_oligo_1.fq.gz --oligofastq2 /HXsz1/YQ/Project/scRNA-Seq.C4/testing/Raw_Data/E250015259_L01_KO_5_oligo_2.fq.gz --threads 8 --name KO_5 --chemistry auto --darkreaction auto --outdir /HXsz1/YQ/Project/scRNA-Seq.C4/testing/ztmp_KO_5 --genomeDir /HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/database/Mouse-GRCm38 --gtf /HXsz1/YQ/Software/Miniconda3/envs/dnbc4tools-2.1.2/database/Mouse-GRCm38/genes.filter.gtf --chrMT chrM' died with <Signals.SIGABRT: 6>.

已输出数据: image

数据库目录: image 数据库生成过程无异常信息;

能麻烦您帮忙看一下报错的原因吗?

lishuangshuang0616 commented 5 months ago

After looking at your directory, your alignment process has generated 6.2G of Aligned.out.bam, which means that there is no problem at the beginning of the alignment process. However, during the operation, the node memory was occupied at a certain time period, resulting in insufficient running memory. You can confirm whether this problem exists. Re-analyze to ensure that the memory is stable.

lishuangshuang0616 commented 5 months ago

Sorry, I just checked carefully and your comparison has finished running and generated Log.final.out.

YaoQiong1127 commented 5 months ago

是的,比对看上去已经跑完了;Log.out 的倒数第2、3两行,显示 “finished building index!”: image 但 01.data 里面,并没有 bam的index文件;

lishuangshuang0616 commented 5 months ago
export PATH=/haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/bin:$PATH
export LD_LIBRARY_PATH=/haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib:$LD_LIBRARY_PATH
/haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/software/Anno -I /haplox/users/yaoqiong/Project/scRNA-Seq.C4/testing/ztmp_KO_5/01.data/Aligned.out.bam -a /haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/database/Mouse-GRCm38/genes.filter.gtf -L /haplox/users/yaoqiong/Project/scRNA-Seq.C4/testing/ztmp_KO_5/01.data/cDNA.barcode.counts.txt -o /haplox/users/yaoqiong/Project/scRNA-Seq.C4/testing/ztmp_KO_5/01.data -c 10 -m chrM -B /haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/config/cellbarcode/scRNAv2v3HT/scRNA_beads_v2_3.json --anno 1 --intron

Please test this command manually. I want to see what went wrong with Anno.

lishuangshuang0616 commented 5 months ago

libgcc_s.so.1 must be installed for pthread_cancel to work

I have not encountered this error. Try adding

import ctypes
libgcc_s = ctypes.CDLL('libgcc_s.so.1')

to the top line of /haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/rna/data.py

YaoQiong1127 commented 5 months ago
export PATH=/haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/bin:$PATH
export LD_LIBRARY_PATH=/haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib:$LD_LIBRARY_PATH
/haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/software/Anno -I /haplox/users/yaoqiong/Project/scRNA-Seq.C4/testing/ztmp_KO_5/01.data/Aligned.out.bam -a /haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/database/Mouse-GRCm38/genes.filter.gtf -L /haplox/users/yaoqiong/Project/scRNA-Seq.C4/testing/ztmp_KO_5/01.data/cDNA.barcode.counts.txt -o /haplox/users/yaoqiong/Project/scRNA-Seq.C4/testing/ztmp_KO_5/01.data -c 10 -m chrM -B /haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/config/cellbarcode/scRNAv2v3HT/scRNA_beads_v2_3.json --anno 1 --intron

Please test this command manually. I want to see what went wrong with Anno.

image

image

单独跑 Anno,无报错,final.bam, anno_report.csv, alignment_report.csv, beads_stat.txt 这4个文件可以正常生成;

image

YaoQiong1127 commented 5 months ago

libgcc_s.so.1 must be installed for pthread_cancel to work

I have not encountered this error. Try adding

import ctypes
libgcc_s = ctypes.CDLL('libgcc_s.so.1')

to the top line of /haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/rna/data.py

好的,我试试

YaoQiong1127 commented 5 months ago

libgcc_s.so.1 must be installed for pthread_cancel to work

I have not encountered this error. Try adding

import ctypes
libgcc_s = ctypes.CDLL('libgcc_s.so.1')

to the top line of /haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/rna/data.py

这样处理之后,之前报错的步骤已正常运行了,但出现了新的报错:

ztmp_p1

ztmp_p2

ztmp_p3

ztmp_p4

lishuangshuang0616 commented 5 months ago

实在很抱歉给你带来这么不好的体验。 从报错结果来看处理oligo数据时报错。我重新注释了前面的步骤然后直接运行该步骤进行测试。 使用我提供的这个文件data.py替换下 /haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/rna/data.py 这个文件,原data.py文件记得备份。 测试时如果还是出现这个报错,将 _oligo_combine_pl 改成 oligo_combine_pd_ 再次进行尝试。 image 最终看测试结果再决定如何调整代码。 麻烦了。

YaoQiong1127 commented 5 months ago

实在很抱歉给你带来这么不好的体验。 从报错结果来看处理oligo数据时报错。我重新注释了前面的步骤然后直接运行该步骤进行测试。 使用我提供的这个文件data.py替换下 /haplox/users/yaoqiong/Software/Miniconda3/envs/dnbc4tools-2.1.2/lib/python3.8/site-packages/dnbc4tools/rna/data.py 这个文件,原data.py文件记得备份。 测试时如果还是出现这个报错,将 _oligo_combine_pl 改成 oligo_combine_pd_ 再次进行尝试。 image 最终看测试结果再决定如何调整代码。 麻烦了。

替换 data.py 之后,没有运行,直接报错了:

image

lishuangshuang0616 commented 5 months ago

image 先注释了试试.

YaoQiong1127 commented 5 months ago

image 先注释了试试.

不运行,直接报错了:

image

lishuangshuang0616 commented 5 months ago

你是重新开了个新的分析吗,在原分析目录运行即可,我看你的oligo.reads.fq.gz找不到了

YaoQiong1127 commented 5 months ago

你是重新开了个新的分析吗,在原分析目录运行即可,我看你的oligo.reads.fq.gz找不到了

啊,是的;我再试试

YaoQiong1127 commented 5 months ago

image 先注释了试试.

我用了您新发给我的 data.py , oligo_combine_pl 和 oligo_combine_pd 都试过,但都报错了:

zzz_p2

zzz_p1

lishuangshuang0616 commented 5 months ago

分析的时候这个报错一般都是内存问题。能注意到内存占用吗?很奇怪,pd的话应该不会导致大内存。

YaoQiong1127 commented 5 months ago

分析的时候这个报错一般都是内存问题。能注意到内存占用吗?很奇怪,pd的话应该不会导致大内存。

内存占用不是太高:

image

image

lishuangshuang0616 commented 5 months ago

Check if _export LD_LIBRARYPATH exists in your bashrc. If it does, comment it out and open a new terminal before re-running the work. The software will use Python multi threading in several places, so addressing the environment is necessary.