[BUG] Modify cpus parameter: very low value (8), no parameter. Running PGAP still not working

[wanjinhu]

Hi @azat-badretdin ,

The problem I mentioned in https://github.com/ncbi/pgap/issues/205 before, according to your suggestion, I modify --cpus parameter: very low value (8), or no parameter, but PGAP still did not run successfully.

Here are new log file: cwltool.log

Can you help me figure out what the problem is?

Thanks very much

wanjinhu

[azat-badretdin]

The problem is different now:

choosing min/max for populated genome for species: 562, ngenomes = 154837
min = 4000000
max = 6300000
genome_size = 580076
verify-genome-size: fail
verify-only-ns: pass
verify-seqids: pass
[2022-06-02 01:54:16] DEBUG Could not collect memory usage, job ended before monitoring began.
[2022-06-02 01:54:16] WARNING [job Prepare_Unannotated_Sequences_pgapx_input_check] exited with status: 1
[2022-06-02 01:54:16] WARNING [job Prepare_Unannotated_Sequences_pgapx_input_check] completed permanentFail

For specified taxonomy the size of the genome is very small. You can circumvene this by specifiying --ignore-all-errors parameter.

[wanjinhu]

Hi @azat-badretdin

I'm sorry to bother you again, but as you suggested: specifiying --ignore-all-errors parameter. And it still fails.

Here are new log file: cwltool.log

wanjinhu

[azat-badretdin]

Thanks.

OK. Now the problem that prompted this ticket is reproduced.

The echo of the command in cwltool.log suggests that you used somehow modified MG37 test. The genome size diagnostics you posted earlier also indicates that you did not use actual MG37 sequence as input. Can you share the data with us?

If yes, could you please email it to prokaryote-tools@ncbi.nlm.nih.gov? Please email as input.yaml, submol.yaml, FASTA file and command line you used to envoke PGAPx.

Thanks

[wanjinhu]

@azat-badretdin Hi,

As you suggested, I have email input files to prokaryote-tools@ncbi.nlm.nih.gov two days ago. Maybe you didn't notice the email, I'd love to hear from you if you see it.

Thanks a lot

wanjinhu

[thibaudnis]

Hi @wanjinhu : sorry for the slow response. Your email ended up in my junk folder. We'll have a look as soon as we can. Thanks!

[azat-badretdin]

For reference, copy pasting relevant portion of cwltool.log

20220606_015102: Start rpsblast for /tmp/36j8pmn_/stg3292d79a-0db2-4b40-a709-45396cc5c8c0/proteins.fa
+ echo '"/panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-04-14.build6021/arch/x86_64/bin/rpsblast" "-parse_deflines' -outfmt '11" "-evalue' 0.01 -num_threads 0 -db '/tmp/36j8pmn_/stgb1e02bda-2d9a-40f8-83c9-f2db504a9750/CDD2/cdd_split8" -query "/tmp/36j8pmn_/stg3292d79a-0db2-4b40-a709-45396cc5c8c0/proteins.fa" -out "/tmp/9k9h34ew/sparclbl_mp.sh_20220606_015102/sparclbl_work/proteins.asn" '
"/panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-04-14.build6021/arch/x86_64/bin/rpsblast" "-parse_deflines -outfmt 11" "-evalue 0.01 -num_threads 0 -db /tmp/36j8pmn_/stgb1e02bda-2d9a-40f8-83c9-f2db504a9750/CDD2/cdd_split8" -query "/tmp/36j8pmn_/stg3292d79a-0db2-4b40-a709-45396cc5c8c0/proteins.fa" -out "/tmp/9k9h34ew/sparclbl_mp.sh_20220606_015102/sparclbl_work/proteins.asn" 
+ /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-04-14.build6021/arch/x86_64/bin/rpsblast -parse_deflines -outfmt 11 -evalue 0.01 -num_threads 0 -db /tmp/36j8pmn_/stgb1e02bda-2d9a-40f8-83c9-f2db504a9750/CDD2/cdd_split8 -query /tmp/36j8pmn_/stg3292d79a-0db2-4b40-a709-45396cc5c8c0/proteins.fa -out /tmp/9k9h34ew/sparclbl_mp.sh_20220606_015102/sparclbl_work/proteins.asn
/panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-04-14.build6021/arch/x86_64/bin/sparclbl_mp.sh: line 384: 412408 Segmentation fault      (core dumped) "${rpsblast_bin}" "${BLASTCMD[@]-}" "${blastargs[@]-}" -query "${single_seqfile}" -out "${sglasn}"

[wanjinhu]

Hi @azat-badretdin

I don't understand the purpose of your reply to this comment. Could it be a problem with rpsblast? I checked the rpsblast in my system environment and there is no problem.

How should the tmp file in PGAP operation be saved? Maybe I can see if it's a problem with rpsblast running.

Thanks again

wanjinhu

[azat-badretdin]

Could it be a problem with rpsblast? I checked the rpsblast in my system environment and there is no problem.

No, it's not a problem with rpsblast on your system.

Since we have received the data you sent us we opened an internal ticket to investigate.

Nothing needs to be done on your side at this moment.

[azat-badretdin]

We failed to reproduce this problem with your data on our local testing machine.

But we noticed that for some computationally nodes you used a lot more memory than us, for example:

{code}

-[YYYY-MM-DD hh:mm:ss] INFO [job cluster_blastp_wnode_3] Max memory used: 1316MiB
+[YYYY-MM-DD hh:mm:ss] INFO [job cluster_blastp_wnode_3] Max memory used: 32MiB

{code}

I would recommend to explore more memory options, like increasing memory or reducing number of cores used using --cpus parameter. Have you tried using just 4 cores?

[wanjinhu]

Hi @azat-badretdin

I have tried used 4 cores as --cpus parameter, but it still not working. Maybe the way of online analysis is more suitable for me

Anyway, thanks again for your patience

wanjin.hu