mroosmalen
commented
1 year ago I know SMuRF is not optimized for a lot of samples. We have tested it only on 5-10 samples , and it works the fastest if you give it also a control. What SMuRF does is it checks every mutation for each sample the raw bam file, to check if its finds any evidence for this mutation in the specified sample. To make it faster it will make some assumes, it will skip for example every mutation with a FILTER tag in the vcf file (keeps only the PASS variants) but also mutations that are found in the control (e.g. germline mutations) will be skipped.
The To give you some examples from our side:
17 hours for 8 cord blood samples and 1 control (15x) with 8 threads 8 hours for a AML patient with 4 samples and 1 control (15x) with 8 threads
I never tested it, but you could give PTATO also your own somatic vcf files per sample and skip SMuRF. The you should mimic this kind of folder:
/path/to/somatic_vcfs/DONOR_ID
./sample1.vcf.gz
./sample2.vcf.gz
....and then point to this folder in your run config file
short_variants {
somatic_vcfs_dir = '/path/to/somatic_vcfs' # without the DONOR_ID
}I hope this give will help you
um-pdavila
Good afternoon,
The SMuRF section of our PTATO job has been running since 2023-04-26. We want to confirm this is not because of a bug. That is, that SMuRF's performance is the result of our dataset and parallelization limitations of SMuRF, as used in PTATO. We have spoken with several collaborators who are experiencing similar performance issues using PTATO (because of SMuRF).
Our dataset has 170 single cell, with coverage of 13x. This is much larger than the demo dataset, which does complete.
The SLURM job is running on 1 node, for two weeks, using 23 cores (64 allocated).
[fxd317@n533 ~]$ uptime load average: 22.35, 22.21, 22.19
[root@n521 ~]# du -hs $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp 59G /sccc/scratch/fxd317/DATA/bioskryb_3-6-2023/PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp
[root@n521 ~]# ls -lhtc $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/* | wc -l 3366
$ ls -lhtc /sccc/scratch/fxd317/DATA/bioskryb_3-6-2023/PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/* | head -n 25 -rw-r--r-- 1 fxd317 ccsuser 2.3G May 8 13:53 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/4.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.5G May 8 13:51 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/6.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 3.3G May 8 13:49 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/16.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.2G May 8 13:49 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/X.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 3.4G May 8 13:49 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/19.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.5G May 8 13:48 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/18.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.9G May 8 13:45 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/13.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.4G May 8 13:44 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/9.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.3G May 8 13:43 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/11.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 3.2G May 8 13:42 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/20.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.2G May 8 13:40 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/3.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.6G May 8 13:39 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/12.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.5G May 8 13:36 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/8.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 3.2G May 8 13:33 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/17.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.8G May 8 13:32 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/1.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.2G May 8 13:27 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/14.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.5G May 8 13:26 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/15.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.3G May 8 13:26 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/5.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.2G May 8 13:24 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/2.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 3.3G May 8 13:22 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/10.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.2G May 8 13:09 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/7.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 2.6G May 7 07:41 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/21.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 1.9G Apr 30 22:38 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/22.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 210K Apr 26 00:05 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/Un_KI270394v1.SMuRF.vcf -rw-r--r-- 1 fxd317 ccsuser 210K Apr 26 00:05 $PTATO/work/d6/3892059004580b0cfbf223d5421269/SMuRF_tmp/Un_KI270392v1.SMuRF.vcf
... 3340 additional files created and last accessed on “Apr 26 00:05”.
Best, Pedro