ncbi / egapx

Eukaryotic Genome Annotation Pipeline-External caller scripts and documentation
Other
80 stars 8 forks source link

rnaseq_collapse:Perhaps the program crashes due to excessive thread usage #40

Open bismarck1008 opened 1 month ago

bismarck1008 commented 1 month ago

python3 data/egapx/ui/egapx.py data/input_D_farinae_small.local4.yaml -e docker -w ./temp -o egapx_out

[- ] process > export - [ae/0765b8] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (6) terminated with an error exit status (3) -- Execution is retried (3) [41/720134] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (5) terminated with an error exit status (3) -- Execution is retried (3) [6d/b77983] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (7) terminated with an error exit status (3) -- Execution is retried (3) [66/59923f] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (4) terminated with an error exit status (3) -- Execution is retried (2) [10/c33777] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (1) terminated with an error exit status (3) -- Execution is retried (3) [54/5d8c25] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (9) terminated with an error exit status (3) -- Execution is retried (3) ERROR ~ Error executing process > 'egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (10)'

Caused by: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (10) terminated with an error exit status (3)

Command executed:

njobs=wc -l <job.009 if [ $njobs -lt 16 ]; then threads=$njobs else threads=16 fi

boukn commented 1 month ago

its a lot of threads but it that shouldnt crash anything by itself... Can you provide the contents of the ./temp/41/720134 workdir?

bismarck1008 commented 1 month ago

its a lot of threads but it that shouldnt crash anything by itself... Can you provide the contents of the ./temp/41/720134 workdir?

The file has been deleted. After reducing the input file to a pair of sequences, I still encountered similar errors. [- ] process > export [03/b488e2] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (9) terminated with an error exit status (3) -- Execution is retried (3) [ee/72b6e2] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (3) terminated with an error exit status (3) -- Execution is retried (2) [f5/917462] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (1) terminated with an error exit status (3) -- Execution is retried (3) [a0/c9f280] NOTE: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (7) terminated with an error exit status (3) -- Execution is retried (3) ERROR ~ Error executing process > 'egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (10)'

Caused by: Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (10) terminated with an error exit status (3)

Command executed:

njobs=wc -l <job.009 if [ $njobs -lt 16 ]; then threads=$njobs else threads=16 fi

ll 03/b488e2 GCA_040085125.1_ASM4008512v1_genomic-bin1.align.asnb.gz -> /data/dell/CNI.2024.10.5/anotation/temp/c1/b07c7e05b63c1bb7014234c2d71813/GCA_040085125.1_ASM4008512v1_genomic-bin1.align.asnb.gz 130 10:22 GCA_040085125.1_ASM4008512v1_genomic-bin17.align.asnb.gz -> /data/dell/CNI.2024.10.5/anotation/temp/53/cfb8104531b57a2768b405aa103d19/GCA_040085125.1_ASM4008512v1_genomic-bin17.align.asnb.gz 4096 10:22 genome/ 81 10:22 job.008 -> /data/dell/CNI.2024.10.5/anotation/temp/af/7d3e2c5828ef6dd2d549b59abc9be1/job.008 0 10:22 .command.out 0 10:22 .command.trace 1071 10:22 scaffold_list.mft 34 10:22 metadata.mft 1017 10:22 align.mft 884736 10:22 genome_lds 4096 10:22 interim/ 140778 10:23 .command.log 140778 10:23 .command.err 1 10:23 .exitcode 4096 10:23 ./

ll ee/72b6e2

1017  10:21 align.mft
 0   10:21 .command.begin

149960 10:23 .command.err 149960 10:23 .command.log 0 10:21 .command.out 19973 10:21 .command.run 3475 10:21 .command.sh 0 10:21 .command.trace 92 10:21 egapx_reads_metadata_ns63yaxv.tsv -> /data/dell/CNI.2024.10.5/anotation/egapx_GCA_040085125_out/egapx_reads_metadata_ns63yaxv.tsv 1 10月 8 10:23 .exitcode GCA_040085125.1_ASM4008512v1_genomic-bin10.align.asnb.gz -> /data/dell/CNI.2024.10.5/anotation/temp/c2/df2abb714d4f40524588daa8d2efdf/GCA_040085125.1_ASM4008512v1_genomic-bin10.align.asnb.gz 4096 10:21 genome/ 884736 10:22 genome_lds 4096 10:22 interim/ 81 10:21 job.002 -> /data/dell/CNI.2024.10.5/anotation/temp/af/7d3e2c5828ef6dd2d549b59abc9be1/job.002 34 10:21 metadata.mft 1071 10:21 scaffold_list.mft