Snakemake issue?

[drob2727]

Everything seems to be running fine until I restart the job, the job times out on our cluster. Then I get this issue

Error in rule merge_provirus_call_by_group_by_split: jobid: 627 output: iter-0/ssDNA/all.pdg.prv.bdy, iter-0/ssDNA/all.pdg.prv.ftr conda-env: /xena/scratch/drob2727/db/conda_envs/0e64dbf1 shell:

        printf "%s

" iter-0/ssDNA/all.pdg.gff.splitdir/all.pdg.gff.0.split.prv.bdy iter-0/ssDNA/all.pdg.gff.splitdir/all.pdg.gff.1.split.prv.bdy iter-0/ssDNA/all.pdg.gff.splitdir/all.pdg.gff.2.split.prv.bdy | xargs cat | awk '!/^seqname / || !f++' > iter-0/ssDNA/all.pdg.prv.bdy printf "%s " iter-0/ssDNA/all.pdg.gff.splitdir/all.pdg.gff.0.split.prv.ftr iter-0/ssDNA/all.pdg.gff.splitdir/all.pdg.gff.1.split.prv.ftr iter-0/ssDNA/all.pdg.gff.splitdir/all.pdg.gff.2.split.prv.ftr | xargs cat | awk '!/^seqname / || !f++' > iter-0/ssDNA/all.pdg.prv.ftr

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

cat: iter-0/dsDNAphage/all.pdg.gff.splitdir/all.pdg.gff.0.split.prv.bdy: No such file or directory cat: iter-0/dsDNAphage/all.pdg.gff.splitdir/all.pdg.gff.1.split.prv.bdy: No such file or directory [Sun Jun 19 14:49:35 2022] Error in rule merge_provirus_call_by_group_by_split: jobid: 623 output: iter-0/dsDNAphage/all.pdg.prv.bdy, iter-0/dsDNAphage/all.pdg.prv.ftr conda-env: /xena/scratch/drob2727/db/conda_envs/0e64dbf1 shell:

        printf "%s

" iter-0/dsDNAphage/all.pdg.gff.splitdir/all.pdg.gff.0.split.prv.bdy iter-0/dsDNAphage/all.pdg.gff.splitdir/all.pdg.gff.1.split.prv.bdy iter-0/dsDNAphage/all.pdg.gff.splitdir/all.pdg.gff.2.split.prv.bdy | xargs cat | awk '!/^seqname / || !f++' > iter-0/dsDNAphage/all.pdg.prv.bdy printf "%s " iter-0/dsDNAphage/all.pdg.gff.splitdir/all.pdg.gff.0.split.prv.ftr iter-0/dsDNAphage/all.pdg.gff.splitdir/all.pdg.gff.1.split.prv.ftr iter-0/dsDNAphage/all.pdg.gff.splitdir/all.pdg.gff.2.split.prv.ftr | xargs cat | awk '!/^seqname / || !f++' > iter-0/dsDNAphage/all.pdg.prv.ftr

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

cat: iter-0/NCLDV/all.pdg.gff.splitdir/all.pdg.gff.0.split.prv.bdy: No such file or directory cat: iter-0/NCLDV/all.pdg.gff.splitdir/all.pdg.gff.1.split.prv.bdy: No such file or directory [Sun Jun 19 14:49:35 2022] Error in rule merge_provirus_call_by_group_by_split: jobid: 631 output: iter-0/NCLDV/all.pdg.prv.bdy, iter-0/NCLDV/all.pdg.prv.ftr conda-env: /xena/scratch/drob2727/db/conda_envs/0e64dbf1 shell:

        printf "%s

" iter-0/NCLDV/all.pdg.gff.splitdir/all.pdg.gff.0.split.prv.bdy iter-0/NCLDV/all.pdg.gff.splitdir/all.pdg.gff.1.split.prv.bdy iter-0/NCLDV/all.pdg.gff.splitdir/all.pdg.gff.2.split.prv.bdy | xargs cat | awk '!/^seqname / || !f++' > iter-0/NCLDV/all.pdg.prv.bdy printf "%s " iter-0/NCLDV/all.pdg.gff.splitdir/all.pdg.gff.0.split.prv.ftr iter-0/NCLDV/all.pdg.gff.splitdir/all.pdg.gff.1.split.prv.ftr iter-0/NCLDV/all.pdg.gff.splitdir/all.pdg.gff.2.split.prv.ftr | xargs cat | awk '!/^seqname / || !f++' > iter-0/NCLDV/all.pdg.prv.ftr

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Exiting because a job execution failed. Look above for error message

*** An error occurred. Detailed errors may not be printed for certain rules. Refer to the log file of the failed command for troubleshooting Issues can be raised at: https://github.com/jiarong/VirSorter2/issues

[jiarong]

Hi, thanks for the feedback. Looks like the issue is VS2 could not identify where previous run failed due to a forced kill of process by the system. You can either delete the whole output directory and rerun from scratch, or rerun only the classify step since that's where the job timeout happens by replacing virsorter run [options] all with virsorter run [options] classify to save some computation. In your case, [options] should be the same.

[martinshugoc]

Hi I’m having a error that resembles this one and it happens only in servers not in personal machines. I tried to run it again and just the classification step, but it didn’t work anyway and the error was the same.

Log=/storage1/dados/es59724/visorter2/analises_dezembro_2022/3/VS2_GCF_seq_genomic.fna/log/iter-0/step3-classify/all-score-ssDNA.log python /data/apps/miniconda3/4.9.2/envs/virsorter2-2.2.3/lib/python3.10/site-packages/virsorter/./scripts/classify.py iter-0/ssDNA/all.pdg.ftr /data/apps/virsorter2/2.2.3/db/group/ssDNA/model ssDNA iter-0/ssDNA/all.pdg.clf 2> $Log || { echo "See error details in $Log" | python /data/apps/miniconda3/4.9.2/envs/virsorter2-2.2.3/lib/python3.10/site-packages/virsorter/./scripts/echo.py --level error; exit 1; }

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Apparently the error is related to the line 665 of the file numpy_pickle_2.py, but I don’t know why it only cause an error when using a server.

[jiarong]

Hi, you are having a different issue. It's very likely your server have different version of packages installed and somehow not picking up the version installed in VirSorter2. I recommend installation option 3, which should get around this issue.