assembly.homopolymer-compressed.gfa file?

[chunlinxiao]

From verkko run output, I noticed that the file "assembly.homopolymer-compressed.noseq.gfa" was there, but not the file "assembly.homopolymer-compressed.gfa". The run finished to the end without error. Is this intentional or anything we missed?

thanks

[skoren]

I suspect something did fail for that file to be missing, it's copied in the last step of the snakemake file. However, it is identical to the 5-untip/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.gfa file so you could just copy that instead.

[chunlinxiao]

okay that's good to know. I run it on SGE, and the job was successful without any error.

I believe same happened to files assembly.hifi-coverage.csv and assembly.ont-coverage.csv ( corresponding to 5-untip/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.hifi-coverage.csv, and 5-untip/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.ont-coverage.csv) .

These happened to my other verkko runs on sge as well - not sure why those files were not copied over.

I will use 5-untip/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.gfa etc instead - thanks

[skoren]

Interesting, we've run on SGE w/o issues, I remember some older snakemake versions stopped before the pipeline had fully completed. What is the end of the log from the run say? It should have a localrule verkko::

[Sat May 25 20:48:11 2024]
localrule verkko:
    input: 5-untip/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.gfa, 6-rukki/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.noseq.gfa, 6-layoutContigs/unitig-popped.layout, 6-layoutContigs/unitig-popped.layout.scfmap, 7-consensus/unitig-popped.fasta, 7-consensus/unitig-popped.haplotype1.fasta, 7-consensus/unitig-popped.haplotype2.fasta, 7-consensus/unitig-popped.unassigned.fasta, 6-rukki/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.paths.tsv, 6-rukki/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.colors.csv, 2-processGraph/unitig-unrolled-hifi-resolved.hifi-coverage.csv, 2-processGraph/unitig-unrolled-hifi-resolved.ont-coverage.csv, 4-processONT/unitig-unrolled-ont-resolved.hifi-coverage.csv, 4-processONT/unitig-unrolled-ont-resolved.ont-coverage.csv, 5-untip/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.hifi-coverage.csv, 5-untip/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.ont-coverage.csv
    output: assembly.homopolymer-compressed.gfa, assembly.homopolymer-compressed.noseq.gfa, assembly.homopolymer-compressed.layout, assembly.hifi-coverage.csv, assembly.ont-coverage.csv, assembly.fasta, assembly.scfmap
    jobid: 0
    reason: Missing output files: assembly.fasta, assembly.homopolymer-compressed.noseq.gfa, assembly.hifi-coverage.csv, assembly.ont-coverage.csv, assembly.homopolymer-compressed.gfa, assembly.scfmap, assembly.homopolymer-compressed.layout; Input files updated by another job: 5-untip/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.ont-coverage.csv, 6-rukki/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.noseq.gfa, 7-consensus/unitig-popped.unassigned.fasta, 6-layoutContigs/unitig-popped.layout.scfmap, 7-consensus/unitig-popped.fasta, 2-processGraph/unitig-unrolled-hifi-resolved.ont-coverage.csv, 6-rukki/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.paths.tsv, 7-consensus/unitig-popped.haplotype1.fasta, 7-consensus/unitig-popped.haplotype2.fasta, 4-processONT/unitig-unrolled-ont-resolved.hifi-coverage.csv, 2-processGraph/unitig-unrolled-hifi-resolved.hifi-coverage.csv, 4-processONT/unitig-unrolled-ont-resolved.ont-coverage.csv, 5-untip/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.hifi-coverage.csv, 6-rukki/unitig-unrolled-unitig-unrolled-popped-unitig-normal-connected-tip.colors.csv, 6-layoutContigs/unitig-popped.layout
    resources: mem_mb=31479, mem_mib=30021, disk_mb=31479, disk_mib=30021, tmpdir=/tmp

[Sat May 25 20:48:59 2024]
Finished job 0.
36 of 36 steps (100%) done
Complete log: .snakemake/log/2024-05-25T153507.663813.snakemake.log

which does the actual copying.

[chunlinxiao]

The end of the log below (from verkko2.1 finished run on SGE).

Also snakemake version:

snakemake --version 7.32.4

[Tue Jun 4 22:27:24 2024] localrule cnspath: input: 6-layoutContigs/unitig-popped.layout, 6-layoutContigs/unitig-popped.layout.scfmap, 7-consensus/unitig-popped.fasta, 7-consensus/unitig-popped.haplotype1.fasta, 7-consensus/unitig-popped.haplotype2.fasta, 7-consensus/unitig-popped.unassigned.fasta output: assembly.homopolymer-compressed.layout, assembly.fasta, assembly.scfmap jobid: 0 reason: Missing output files: assembly.scfmap, assembly.homopolymer-compressed.layout, assembly.fasta; Input files updated by another job: 6-layoutContigs/unitig-popped.layout, 7-consensus/unitig-popped.fasta, 7-consensus/unitig-popped.unassigned.fasta, 7-consensus/unitig-popped.haplotype1.fasta, 6-layoutContigs/unitig-popped.layout.scfmap, 7-consensus/unitig-popped.haplotype2.fasta resources: tmpdir=/export/home/TMP/1207729406.1.high

[Tue Jun 4 22:27:53 2024] Finished job 0. 33 of 33 steps (100%) done Complete log: .snakemake/log/2024-06-03T190137.610129.snakemake.log

Again, there was no any error messages.

Thanks

[chunlinxiao]

Just checked the latest snakemake from https://github.com/snakemake/snakemake/releases/tag/v8.14.0, released on 2024-06-11.

The latest is 8.14.0, mine 7.32.4 - not sure if this was the reason, but I will update my snakemake to see what happens.

[skoren]

Don't update to snakemake8, that has breaking changes and won't be compatible with verkko.

I didn't realize you were using cnspaths, that is a known issue (#241) so not a surprise. Should be easy to fix but I'll close this issue and track the status of fixes in #241

[chunlinxiao]

okay good - thanks