AuReMe / mpwt

Pathway Tools multiprocessing wrapper (for PathoLogic).
GNU Lesser General Public License v3.0
15 stars 0 forks source link

Issue mpwt with the --hf option #73

Closed Kisekya closed 2 years ago

Kisekya commented 2 years ago

Hi,

I use Pathway Tools 25.5. and mpwt 0.7.0. on 12 bacteria gbk files generated with emapper2gbk and I use this command of mpwt:

mpwt -f ${path_directory_input} -o ${path_directory_output} --patho --hf --md --log ../output_job/mpwt/outpmpwt_logs --taxon-file ${path_directory_input}taxon_id.tsv --cpu 40  -v --flat --clean

There seems to be an error when using mpwt with the --hf option When I run my job on the cluster the .dat files are all created but when generating the log_error.txt and resume_inference.tsv files, there is an error and the files are not complete. In my .log it is written :

....
|Output Check| 12 on 12 builds have passed!
Traceback (most recent call last):
  File "/usr/local/bin/mpwt", line 8, in <module>
    sys.exit(run_mpwt())
  File "/usr/local/lib/python3.6/dist-packages/mpwt/__main__.py", line 376, in run_mpwt
    permission=permission)
  File "/usr/local/lib/python3.6/dist-packages/mpwt/mpwt_workflow.py", line 153, in multiprocess_pwt
    taxon_file, permission)
  File "/usr/local/lib/python3.6/dist-packages/mpwt/mpwt_workflow.py", line 393, in independent_mpwt
    check_mpwt_pathologic_runs(input_folders, patho_log)
  File "/usr/local/lib/python3.6/dist-packages/mpwt/results_check.py", line 199, in check_mpwt_pathologic_runs
    patho_resume_writer.writerow(*species_pathologic_informations[12])
TypeError: writerow() takes exactly one argument (0 given)
srun: error: zonda14: task 0: Exited with exit code 1

The files are not complete and since they are written from the pathologic.log file I have gone through it and there seems to be a recurring error which would explain why the log_error.txt and resume_inference.tsv files are not complete:

Normally to the end of one of my pathologic.log file after hole filler:(this file is resumed in the log_error.txt and resume_inference.tsv files)

Starting Hole Filler for ACUTALIBACTER_MURIS_STRAIN_KB18.
[Saving KB ACUTALIBACTER_MURIS_STRAIN_KB18BASE to V4-FASL file /beegfs/cmuller/pathwaystools/ptools-local/pgdbs/user/acutalibacter_muris_strain_kb18cyc/1.0/kb/acutalibacter_muris_strain_kb18base.ocelot
 17977 frames saved]
02-Mar-2022  03:05:21 Done

Crash to the end of one of my pathologic.log file after hole filler:(this file is not resumed in the log_error.txt and resume_inference.tsv files)

02-Mar-2022  00:38:14 Fatal error: mkdir of /programs/pathway-tools/aic-export/pgdbs/biocyc/ecocyc/25.5/data/priors/ failed: Read-only file system [errno=30].
Evaluation stack:

 ->(TPL::ZOOM-COMMAND :FROM-READ-EVAL-PRINT-LOOP NIL ...)
   (SYS::..RUNTIME-OPERATION "applyn" :UNKNOWN-ARGS)
   (TPL:DO-COMMAND "zoom" :FROM-READ-EVAL-PRINT-LOOP ...)

I think it might be a --hf problem because when I do the mpwt command without this option, it runs correctly. I found in pathwaystools/ptools-local/pgdbs/user/SPECIEScyc/1.0 that some species do not have the files: STRAIN_KB18_filled-holes_2022-03-07.html and the hole data.

And I suspect, from my data, that this might be because these specific gbk files are generated with multi-sequence FASTA files. I'm not sure about this but it seems that mpwt with the --hf option on each strain with a gbk file of this fasta type doesn't work in my data but work with others.

But it's weird because then why would it work without the --hf option?

I thank you in advance for your help

Best wishes,

Muller Coralie

Kisekya commented 2 years ago

Finally, I found the problem. I generated my gff and proteome via Prodigal and the gff had to be modified to be usable for emapper2gbk. For the fsa files with one sequence there was no problem for the modification of my gff but in the multi-sequence fsa the modification did not behave as it should and the sequences after the first one in each gbk had the "translation" part empty. So mpwt with the --hf option was not working. After the corrected modification of my gff, I have a full gbk and mpwt seems to work normally without crashing

Have a nice day

Best wishes, Muller Coralie