The specified vcf file () does not exist

[nicokrez]

Hi,

I could not find the answer to my problem in the other Issues. My analysis is running against hg38, chromosomes are named like "1","2","3", ... and not "chr1", "chr2", "chr3"... The naming is the same for all the reference files. I get the following error:

2019-07-25T10:16:11+02:00,ERROR: Job index.partitioned failed with exit code 134. Job logs: /media/data/analysis/ERR233225/results/bwa/index_canvas/Logging/index.partitioned.stdout /media/data/analysis/ERR233225/results/bwa/index_canvas/Logging/index.partitioned.stderr Job error message: Unhandled Exception: System.IO.FileNotFoundException: The specified vcf file () does not exist. at Isas.SequencingFiles.Vcf.VcfReader.GetGzipReader(String vcfPath) at CanvasCommon.PloidyInfo.LoadPloidyFromVcfFileNoSampleId(String vcfPath) in D:\TeamCity\buildAgent\work\a29a190a11771d97\Src\Canvas\CanvasCommon\PloidyInfo.cs:line 115 at CanvasPartition.CanvasPartition.Main(String[] args) in D:\TeamCity\buildAgent\work\a29a190a11771d97\Src\Canvas\CanvasPartition\CanvasPartition.cs:line 114 2019-07-25T10:16:11+02:00,ERROR: Canvas workflow error: Isas.Framework.WorkManagement.JobFailedException: Job index.partitioned failed with exit code 134 at Isas.Framework.WorkManagement.JobLaunching.BasicJobLauncher.RunSystemProcessCheckExitAsync(JobInfo job, Action1 stdoutCallback, Action1 stderrCallback) at Isas.Framework.WorkManagement.JobLaunching.RuntimeTrackingJobLauncher.LaunchJobAsync(JobInfo jobInfo, Action1 stdoutCallback, Action1 stderrCallback) at Isas.Framework.WorkManagement.IJobLauncherExtensions.LaunchJob(IJobLauncher jobLauncher, JobInfo jobInfo, Action1 stdoutCallback, Action1 stderrCallback) at Isas.Framework.WorkManagement.IWorkDoerJobExtensions.<>c__DisplayClass0_01.b0(WorkResources sr, IJobLauncher jobLauncher) at Isas.Framework.WorkManagement.ResourceManagement.ThreadedWorkResourceManager.<>cDisplayClass11_01.<RunWithResources>b__3(WorkResources resources) --- End of stack trace from previous location where exception was thrown --- at Isas.Framework.WorkManagement.ResourceManagement.ThreadedWorkResourceManager.<RunWithResources>b__11_1[T](Task1 resourceRequestTask, Task1 functionTask) at Isas.Framework.WorkManagement.AsTaskExtensions.Await[T](Task1 task) at Canvas.CanvasRunner.InvokeCanvasPartition(CanvasCallset callset, IFileLocation cleanedPath, String canvasBedPath, IFileLocation canvasSnvPath, String ploidyVcfPath) at Isas.Framework.Checkpointing.Internals.SerializingCheckpointRunner.<>c__DisplayClass8_01.<RunCheckpoint>b__0(Checkpoint checkpoint) at Isas.Framework.Checkpointing.Internals.CoreCheckpointRunner.RunCheckpoint[TResult](String name, Func2 run) at Canvas.CanvasRunner.CallSampleInternal(CanvasCallset callset) at Canvas.CanvasRunner.CallSample(CanvasCallset callset) at Canvas.ModeLauncher.Launch() `

While searching for a reason, I found that some files are emtpy, only a header is present. Here is one example "TempCNV_index/3-index.SNV.txt.gz", all of the file types have only a header.

I ran the command to create these files:

`canvas/Canvas-1.40.0.1613+master_x64/CanvasSNV/CanvasSNV.dll -c 3 -v /media/data/analysis/ERR233225/results/gatk/index.vcf -b /media/data/analysis/ERR233225/results/bwa/index_dedup.bam -o /media/data/analysis/ERR233225/results/bwa/index_canvas/TempCNV_index/3-index.SNV.txt.gz

Command-line arguments: -c 3 -v /media/data/analysis/ERR233225/results/gatk/index.vcf -b /media/data/analysis/ERR233225/results/bwa/index_dedup.bam -o /media/data/analysis/ERR233225/results/bwa/index_canvas/TempCNV_index/3-index.SNV.txt.gz 7/25/19 10:38:22 AM Loading variants of interest from /media/data/analysis/ERR233225/results/gatk/index.vcf Retained 0 variants, out of 243654 records for 3 7/25/19 10:38:29 AM Looping over bam records from /media/data/analysis/ERR233225/results/bwa/index_dedup.bam Jump to refid 2 3 Looped over 2 bam records in all 7/25/19 10:38:29 AM Results written to /media/data/analysis/ERR233225/results/bwa/index_canvas/TempCNV_index/3-index.SNV.txt.gz 7/25/19 10:38:29 AM Results written to /media/data/analysis/ERR233225/results/bwa/index_canvas/TempCNV_index/3-index.SNV.txt.gz.baf`

I think the problems might be related to the messages: Retained 0 variants, out of 243654 records for 3 and Looped over 2 bam records in all

Can anyone help me to overcome the problem?

Kind regards, Nicolas

[eroller]

what is the partitioned command line from the CanvasLog.txt file? That is the process that is failing. The dotnet executable should be named CanvasPartition.dll.

[nicokrez]

I think this is the command:

2019-07-26T08:49:21+02:00,Running checkpoint 05: CanvasPartition 2019-07-26T08:49:21+02:00,Checkpoint 04 CanvasClean complete. Elapsed time (hh/mm/ss): 00:00:04.2 2019-07-26T08:49:21+02:00,Launching process for job index.partitioned: /usr/bin/dotnet /home/krezdorn/tools/canvas/Canvas-1.40.0.1613+master_x64/CanvasPartition/CanvasPartition.dll -v /media/data/analysis/ERR233225/results/bwa/index_canvas/TempCNV_index/VFResultsindex.txt.gz -i "/media/data/analysis/ERR233225/results/bwa/index_canvas/TempCNV_index/index.cleaned" -b "/home/krezdorn/tools/canvas/Canvas-1.40.0.1613+master_x64/hg38/filter_hg38.bed" -o "/media/data/analysis/ERR233225/results/bwa/index_canvas/TempCNV_index/index.partitioned" -r "/home/krezdorn/tools/canvas/Canvas-1.40.0.1613+master_x64/hg38" -p "" -g

Might it be a problem that the file "VFResultsindex.txt.gz" consists only of the headline: gunzip -c /media/data/analysis/ERR233225/results/bwa/index_canvas/TempCNV_index/VFResultsindex.txt.gz

Chromosome Position Ref Alt CountRef CountAlt

Thank you for your help and kind regards.

[eroller]

If you are confident the reference names are matching in all files including the b-allele vcf then the problem could be with the format of the b-allele vcf. I believe we use GT and GQ fields to filter events. It is recommended that you use strelka to generate the b-allele vcf from the matched normal bam file.

[nicokrez]

Thank you for indicating strelka. Now the VFResults... file seems to have a normal content.

Still I get an error: 2019-08-20T10:57:45+02:00,Launching process for job tabix-index-tmp-ballele.bedgraph.gz.1: bash -o pipefail -c " /home/krezdorn/tools/canvas/Canvas-1.40.0.1613+master_x64/tabix -p bed -f /media/data/analysis/CNV/CNV_Canvas/WES_val18_1_canvas/TempCNV_WES_val18_1/ballele.bedgraph.gz " 2019-08-20T10:57:45+02:00,Job tabix-index-tmp-ballele.bedgraph.gz.1 duration: 00:00:00.5 2019-08-20T10:57:45+02:00,Finished converting '/media/data/analysis/CNV/CNV_Canvas/WES_val18_1_canvas/TempCNV_WES_val18_1/VFResultsWES_val18_1.txt.gz.baf' to '/media/data/analysis/CNV/CNV_Canvas/WES_val18_1_canvas/TempCNV_WES_val18_1/ballele.bedgraph.gz'. Elapsed time: 00:00:02.9 2019-08-20T10:57:45+02:00, 2019-08-20T10:57:45+02:00,Running checkpoint 05: CanvasPartition 2019-08-20T10:57:45+02:00,Checkpoint 02 CanvasSNV complete. Elapsed time (hh/mm/ss): 00:08:59.7 2019-08-20T10:57:45+02:00,Launching process for job WES_val18_1.partitioned: /usr/bin/dotnet /home/krezdorn/tools/canvas/Canvas-1.40.0.1613+master_x64/CanvasPartition/CanvasPartition.dll -v /media/data/analysis/CNV/CNV_Canvas/WES_val18_1_canvas/TempCNV_WES_val18_1/VFResultsWES_val18_1.txt.gz -i "/media/data/analysis/CNV/CNV_Canvas/WES_val18_1_canvas/TempCNV_WES_val18_1/WES_val18_1.cleaned" -b "/home/krezdorn/tools/canvas/Canvas-1.40.0.1613+master_x64/hg38/filter_hg38.bed" -o "/media/data/analysis/CNV/CNV_Canvas/WES_val18_1_canvas/TempCNV_WES_val18_1/WES_val18_1.partitioned" -r "/home/krezdorn/tools/canvas/Canvas-1.40.0.1613+master_x64/hg38" -p "" -g 2019-08-20T10:57:56+02:00,Job WES_val18_1.partitioned duration: 00:00:10.9 2019-08-20T10:57:56+02:00,ERROR: Job WES_val18_1.partitioned failed with exit code 134. Job logs: /media/data/analysis/CNV/CNV_Canvas/WES_val18_1_canvas/Logging/WES_val18_1.partitioned.stdout /media/data/analysis/CNV/CNV_Canvas/WES_val18_1_canvas/Logging/WES_val18_1.partitioned.stderr Job error message: Unhandled Exception: System.IO.FileNotFoundException: The specified vcf file () does not exist. at Isas.SequencingFiles.Vcf.VcfReader.GetGzipReader(String vcfPath) at CanvasCommon.PloidyInfo.LoadPloidyFromVcfFileNoSampleId(String vcfPath) in D:\TeamCity\buildAgent\work\a29a190a11771d97\Src\Canvas\CanvasCommon\PloidyInfo.cs:line 115 at CanvasPartition.CanvasPartition.Main(String[] args) in D:\TeamCity\buildAgent\work\a29a190a11771d97\Src\Canvas\CanvasPartition\CanvasPartition.cs:line 114 2019-08-20T10:57:56+02:00,ERROR: Canvas workflow error: Isas.Framework.WorkManagement.JobFailedException: Job WES_val18_1.partitioned failed with exit code 134 at Isas.Framework.WorkManagement.JobLaunching.BasicJobLauncher.RunSystemProcessCheckExitAsync(JobInfo job, Action1 stdoutCallback, Action1 stderrCallback) at Isas.Framework.WorkManagement.JobLaunching.RuntimeTrackingJobLauncher.LaunchJobAsync(JobInfo jobInfo, Action1 stdoutCallback, Action1 stderrCallback) at Isas.Framework.WorkManagement.IJobLauncherExtensions.LaunchJob(IJobLauncher jobLauncher, JobInfo jobInfo, Action1 stdoutCallback, Action1 stderrCallback) at Isas.Framework.WorkManagement.IWorkDoerJobExtensions.<>c__DisplayClass0_01.b0(WorkResources sr, IJobLauncher jobLauncher) at Isas.Framework.WorkManagement.ResourceManagement.ThreadedWorkResourceManager.<>cDisplayClass11_01.<RunWithResources>b__3(WorkResources resources) --- End of stack trace from previous location where exception was thrown --- at Isas.Framework.WorkManagement.ResourceManagement.ThreadedWorkResourceManager.<RunWithResources>b__11_1[T](Task1 resourceRequestTask, Task1 functionTask) at Isas.Framework.WorkManagement.AsTaskExtensions.Await[T](Task1 task) at Canvas.CanvasRunner.InvokeCanvasPartition(CanvasCallset callset, IFileLocation cleanedPath, String canvasBedPath, IFileLocation canvasSnvPath, String ploidyVcfPath) at Isas.Framework.Checkpointing.Internals.SerializingCheckpointRunner.<>c__DisplayClass8_01.<RunCheckpoint>b__0(Checkpoint checkpoint) at Isas.Framework.Checkpointing.Internals.CoreCheckpointRunner.RunCheckpoint[TResult](String name, Func2 run) at Canvas.CanvasRunner.CallSampleInternal(CanvasCallset callset) at Canvas.CanvasRunner.CallSample(CanvasCallset callset) at Canvas.ModeLauncher.Launch() ` One thing that strikes me, if I look at the failing command, is the -p "" .

I hope the problem is not due to my initial command: dotnet /home/user/tools/canvas/Canvas-1.40.0.1613+master_x64/Canvas.dll Germline-WGS -n WES_val18_1 --sample-b-allele-vcf /media/data/analysis/CNV/CNV_Canvas/strelka_v7_Wes_val18/Wes_val18_exome/results/variants/genome.S1.vcf.gz -b ../CNV_GATK/hdf5_samples/WES_val18_1.bam -o WES_val18_1_canvas/ -r /home/user/tools/canvas/Canvas-1.40.0.1613+master_x64/GRCh38.primary_assembly.genome.canvas.Kmers -g /home/user/tools/canvas/Canvas-1.40.0.1613+master_x64/hg38/ -f ~/tools/canvas/Canvas-1.40.0.1613+master_x64/hg38/filter_hg38.bed

[eroller]

Is this an exome dataset? For exome you should use the Enrichment workflow with version 1.11. For WGS the latest version will work, but you should use the SmallPedigree-WGS workflow, even for a single sample analysis as Germline-WGS has been deprecated.

[nicokrez]

Thank you for your help.