The gs://ot_orchestration/releases/24.10_freeze2/ produced VariantIndex dataset with ~ 2mln variants, which was aorund half less then expected - see variant to vcf batch job.
The cause of the credible_set failing the variant_to_vcf step was due to the Out Of Memory Exception thrown by spark session - see OOM Error
fix on VM heap size causing incomplete variant_index - variants_to_vcf was failing due to heap space.
After increasing the driver memory, the run was successful - variant_to_vcf
The
gs://ot_orchestration/releases/24.10_freeze2/
producedVariantIndex
dataset with ~ 2mln variants, which was aorund half less then expected - see variant to vcf batch job.The cause of the
credible_set
failing thevariant_to_vcf
step was due to the Out Of Memory Exception thrown by spark session - see OOM Errorfix on VM heap size causing incomplete variant_index - variants_to_vcf was failing due to heap space.
After increasing the driver memory, the run was successful - variant_to_vcf
Things implemented: