Closed flandrs closed 5 years ago
azat-badretdin
commented
5 years ago You might have a log file named cluster_blastp_wnode.*.log for the application that fail. You might have it depending on how you ran pgap.py: with or without --debug flag....
Could you please check if you have it: find . -name cluster_blastp_wnode.\*.log and if not rerun your input with `./pgap.py --debug' flag added?
Thanks!
flandrs
commented
5 years ago Thanks Azat ! I don’t have the cluster_blastp_wnode.*.log file anywhere I re-run with —debug And the surprise is the initial warning: WARNING: memory (GiB) is less than the recommended value of 8 WARNING: memory per CPU core (GiB) is less than the recommended value of 2 This is unexpected (I have 10 cores, 64 Go memory) -> this is a docker problem (and this is due to the default values I do not check) I ask for less cores and a bigger amount of ram to fulfil the warning indications.
And I run again. Waiting...
flandrs
commented
5 years ago It works, [2019-07-15 15:10:32] [workflow ] completed success docker exited with rc = 0 I have the files 588482 May 14 04:28 ASM2732v1.annotation.nucleotide.1.fasta 3033376 Jul 15 17:10 annot-gb.ent 222985 Jul 15 17:10 annot.faa 588444 Jul 15 17:10 annot.fna 1357673 Jul 15 17:10 annot.gbk 246392 Jul 15 17:10 annot.gff 3359893 Jul 15 17:10 annot.sqn 901450 Jul 15 17:36 cwltool.log 224 Jul 15 16:22 debug 1529 May 14 04:28 submol.yaml
All is OK So, the docker's parameters have to be set correctly (this is not mentioned in the documentation and the memory limitations are not pointed out) Thanks for your help
azat-badretdin
commented
5 years ago You are welcome!
flandrs
commented
5 years ago Please note that for Mac OS and python 3.7 we need to activate the SSL certificate with
open /Applications/Python\ 3.6/Install\ Certificates.command
This Mac-only requirement is mandatory to upload with PGAP.py May be helpful :)
JP Flandrois
On 15 Jul 2019, at 19:20, Azat Badretdin notifications@github.com<mailto:notifications@github.com> wrote:
Closed #27https://github.com/ncbi/pgap/issues/27.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ncbi/pgap/issues/27?email_source=notifications&email_token=AL3E5TMAQMBQVH2UJ2K57OLP7SWUJA5CNFSM4IDW6RM2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOSQEW2RI#event-2483645765, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL3E5TNBG5IHG6Y3HI6ZXKTP7SWUJANCNFSM4IDW6RMQ.
azat-badretdin
commented
5 years ago Thank you for your tip!
I just made an error and close the issue :) ...
I just have tried the test genome yesterday with a just-downloaded last version of PGAP on Mac (Os version 10.14.5 ; 2 different types, 32/64Go memory; 8/20 threads,200/300Go free on the internal SSD) But I got an stop in the process with the following errors: [2019-07-15 05:16:37] [step cluster_blastp_wnode] start [2019-07-15 05:17:42] [step cluster_blastp_wnode] completed permanentFail [2019-07-15 05:17:42] [workflow cluster_and_qdump] completed permanentFail [2019-07-15 05:17:42] [step cluster_and_qdump] completed permanentFail [2019-07-15 05:17:42] [workflow Find_Naming_Protein_Hits_I] completed permanentFail [2019-07-15 05:17:42] [step Find_Naming_Protein_Hits_I] completed permanentFail [2019-07-15 05:17:42] [workflow bacterial_annot_2] completed permanentFail [2019-07-15 05:17:42] [step bacterial_annot_2] completed permanentFail [2019-07-15 05:17:42] [workflow standard_pgap] completed permanentFail [2019-07-15 05:17:42] [step standard_pgap] completed permanentFail [2019-07-15 05:17:42] [workflow ] completed permanentFail docker exited with rc = 1
What did I do to obtain that ?
Thanks for the help.
Here is the tail of the log file: internal_thread=monitor&cpu_count=10&normalized_load=119&total_ram=2094985216&pct_ram_used=90&mem_total=1904762880&mem_peak=1904762880&mem_self=1904762880&ncbi_app_version=0.0.0&ncbi_app_sc_version=22&ncbi_app_vcs_revision=586246 68548/001/0047/RE BB760BC4D2C0C361 0144/0038 2019-07-15T05:17:33.869736 f0a199b4f08d UNK_CLIENT UNK_SESSION cluster_blastp_wnode request-stop 200 0.015216112 0 0 68548/001/0048/RB BB760BC4D2C0C361 0145/0039 2019-07-15T05:17:37.067589 f0a199b4f08d UNK_CLIENT UNK_SESSION cluster_blastp_wnode request-start internal_thread=monitor&cpu_count=10&normalized_load=118&total_ram=2094985216&pct_ram_used=92&mem_total=1935921152&mem_peak=1935921152&mem_self=1935921152&ncbi_app_version=0.0.0&ncbi_app_sc_version=22&ncbi_app_vcs_revision=586246 68548/001/0048/RE BB760BC4D2C0C361 0146/0040 2019-07-15T05:17:37.071790 f0a199b4f08d UNK_CLIENT UNK_SESSION cluster_blastp_wnode request-stop 200 0.008281946 0 0 [2019-07-15 05:17:42] [job cluster_blastp_wnode] Max memory used: 174MiB [2019-07-15 05:17:42] [job cluster_blastp_wnode] completed permanentFail [2019-07-15 05:17:42] [step cluster_blastp_wnode] completed permanentFail [2019-07-15 05:17:42] [workflow cluster_and_qdump] completed permanentFail [2019-07-15 05:17:42] [step cluster_and_qdump] completed permanentFail [2019-07-15 05:17:42] [workflow Find_Naming_Protein_Hits_I] completed permanentFail [2019-07-15 05:17:42] [step Find_Naming_Protein_Hits_I] completed permanentFail [2019-07-15 05:17:42] [workflow bacterial_annot_2] completed permanentFail [2019-07-15 05:17:42] [step bacterial_annot_2] completed permanentFail [2019-07-15 05:17:42] [workflow standard_pgap] completed permanentFail [2019-07-15 05:17:42] [step standard_pgap] completed permanentFail [2019-07-15 05:17:42] [workflow ] completed permanentFail [2019-07-15 05:17:42] Final process status is permanentFail { "gbent": null, "gbk": null, "gff": null, "input_fasta": { "class": "File", "location": "file:///pgap/output/ASM2732v1.annotation.nucleotide.1.fasta", "size": 588482, "basename": "ASM2732v1.annotation.nucleotide.1.fasta", "checksum": "sha1$f6129783cc8562db7bca3c87310d57d8dd07ce2c", "path": "/pgap/output/ASM2732v1.annotation.nucleotide.1.fasta" }, "input_submol": { "class": "File", "location": "file:///pgap/output/submol.yaml", "size": 1529, "basename": "submol.yaml", "checksum": "sha1$f9eb1bbbbb1115b94b22b7148647adc3b6107c9f", "path": "/pgap/output/submol.yaml" }, "nucleotide_fasta": null, "protein_fasta": null, "sqn": null }