Open denisovg opened 5 years ago
Hi Gennady, thanks for taking the time to try Clinker, let's get it working for you.
Your column parameter looks like it should work (plus you would generally receive an error if it wasn't).
Which stage are you getting up to in particular? By the sounds of it you're completing the first stage and the alignment hasn't been performed via STAR yet? Do you have a fst_reference.fasta that has appeared in your SKBR3\reference\
directory?
Thanks! Breon.
Hi Breon,
Thanks for getting back to me promptly.
Yes, the file SKBR3/reference/fst_reference.fasta was produced by my run
Regards, Gennady
No problem at all!
Breaking this problem down:
genome/Genome
file has been generated with contents, then likely this stage has performed correctly as well.The next stage is the STAR alignment. I just noticed your FASTQ file input naming convention, could you please try and renaming and gzipping them so they reflect SKBR3_R1.fastq.gz
and SKBR3_R2.fastq.gz
, respectively?
Within the workflow\clinker.pipe
this file naming structure is hard coded in (another user has brought this up in a previous issue https://github.com/Oshlack/Clinker/issues/9). I will endeavour to resolve this in the next version release to a catch all for everyone's naming conventions.
Of course, if this is the problem and you have a standardised format for your FASTQ naming, we can make this change to your clinker.pipe
as the other user did. If not, and this is a key requirement for you, then I will escalate the issue and rustle something up.
If you could make this change then rerun the pipeline after either deleting the contents of the previous Clinker run (the SKBR3a folder) or simply changing your out
parameter to something like SKBR3a, you will get fresh output.
Let me know how you go! Breon.
Hi Breon,
Thanks for your additional input.
Following your suggestion, I renamed the input FASTQ files to be SKBR3.Left.fastq.gz and SKBR3.Right.fastq.gz, respectively.
Furthermore, I have re-ran once again STAR-Fusion and then Clinker to produce a fresh output. Both the runs were completed smoothly, without any error messages.
The fst_reference.fasta file, as well as a number of files in the "genome" subfolder were produced:
SKBR3/reference: fst_reference.fasta
SKBR3/genome: chrLength.txt chrName.txt Genome SA chrNameLength.txt chrStart.txt genomeParameters.txt SAindex
However, the SKBR3/alignment folder is still empty.
One possibility I have been thinking about is whether the Clinker option -p col=6,8 is correct/sufficient for parsing the "caller" file produced by STAR-Fusion.
In the test example provided in Clinker Github repository, the "caller" CSV file, bcr_abl1.csv, is
"chrom1","base1","chrom2","base2" "chr22",23524426,"chr9",133729451
and the option -p col=1,2,3,4 specifies each the chromosome id and the coordinate inside the chromosome as a separate column.
However, the format of the "caller" TSV file produced by STAR-Fusion is different:
TATDN1--GSDMB 235 315 ONLY_REF_SPLICE TATDN1^ENSG00000147687.18 chr8:124539025:- GSDMB^ENSG00000073605.18 chr17:39909924:- ..........
In each the column specified by the the option -p col=6,8 , chromosome id is separated from coordinate by a colon (:). I am not sure how Clinker handles this and whether it is "aware" that colon (:) is used here as a separator. (since we don't tell Clinker explicitly that STAR-Fusion results are used as input.)
Regards, Gennady
Hi Gennaby,
Apologies, I should have made it clear, would you mind renaming the files to the below and trying again:
SKBR3_R1.fastq.gz
and SKBR3_R2.fastq.gz
The _R1 and _R2 are important in this case :).
In terms of the Star Fusion input, I think that stage has been completed successfully and the col="6,8" is correct. Clinker has been developed to recognise that when two columns have been specified, the input format must be something like chrA:123456
or chrB;123456
.
If you look at the first two lines of the fst_reference.fasta
file we can verify this. If the first line is a fusion gene (such as GENEA:GENEB) and the second line is basically a string of bases, then the STAR fusion input has been parsed successfully. If your first line is a single gene (such as GENEA), then there's been a problem during this stage
When you do rerun, could you please post your command line output of the generate_fst
stage? There will be a count for how many fusions successfully were parsed from the Star Fusion input.
Thanks! Let me know how you go :).
EDIT Maybe just send me the whole bpipe output, not just the generate_fst stage.
Thank you Breon!
Your suggestion works: the alignment folder is no longer empty.
However, I noticed that STAR-Fusion and Clinker actually have contradictory requirements for naming the input FASTQ files:
1) STAR-Fusion will produce an "empty" output file star-fusion.fusion_predictions.tsv if the input files named SKBR3_R1.fastq.gz and SKBR3_R2.fastq.gz (acceptable by Clinker) are used
2) on the other hand, Clinker's output subfolder "alignment" will be empty if the input files named SKBR3.Left.fq.gz and SKBR3.Right.fq.gz (acceptable by STAR-Fusion) are used.
Thus, one has to rename the files between the STAR-Fusion and Clinker runs!
Therefore, I believe it would be helpful in the future to relax the Clinker's FASTQ file naming requirements. Why don't you just use the order of files passed to bpipe to distinguish between the "left" and "right" FASTQs?
Regards, Gennady
Hi Gennady,
Great to hear that you got it working! If you run into any other problems, please don't hesitate to ask. Happy to help.
That's a really interesting discovery! I will certainly be updating how FASTQ's get read in based on this thread and another, I will let you know when it's complete, but it will certainly be in the next version.
Cheers, Breon.
Clinker works fine in our hands on the test data provided together with the source code in the GitHub repository. However, we experience difficulty with producing meaningful output when STAR-Fusion results are used as input for Clinker. Although Clinker runs through completion, without error messages, but no data is found in the "alignment" subfolder after the run.
More specifically, we run the following command:
bpipe -p out=SKBR3 -p caller=data/SKBR3_star-fusion.fusion_predictions.tsv -p del="t" -p col="6,8" -p print="true" -p competitive=true -p header=true -p align_mem=32000000000 -p genome_mem=32000000000 -p fusions="TATDN1:GSDMB" $CLINKERDIR/workflow/clinker.pipe data/SKBR3.Left.fq.gz data/SKBR3.Right.fq.gz
where the first two lines in the "caller" file, data/SKBR3_star-fusion.fusion_predictions.tsv, are:
FusionName JunctionReadCount SpanningFragCount SpliceType LeftGene LeftBreakpoint RightGene RightBreakpoint JunctionReads SpanningFrags LargeAnchorSupport FFPM LeftBreakDinuc LeftBreakEntropy RightBreakDinuc RightBreakEntropy annots
TATDN1--GSDMB 229 324 ONLY_REF_SPLICE TATDN1^ENSG00000147687.18 chr8:124539025:- GSDMB^ENSG00000073605.18 chr17:39909924:- &GRPundef@HWI-EAS418:7:39:1563:750,&GRPundef@HWI-EAS418:7:50:1361:162,&GRPundef@HWI-EAS418:7:1:1026:266,&GRPundef@HWI-EAS418:8:42:1649:890,&GRPundef@HWI-EAS418:7:12:1541:540,&GRPundef@HWI-EAS418:7:84:992:1927,&GRPundef@HWI-EAS418:7:26:1419:1336,&GRPundef@HWI-EAS418:7:6:474:982,&GRPundef@HWI-EAS418:8:18:1298:1792,&GRPundef@HWI-EAS418:7:39:1061:1882,&GRPundef@HWI-EAS418:8:74:964:1246,&GRPundef@HWI-EAS418:7:8:1109:1374,&GRPundef@HWI-EAS418:8:31:1613:1324,&GRPundef@HWI-EAS418:7:70:175:886,&GRPundef@HWI-EAS418:8:31:1072:604,&GRPundef@HWI-EAS418:7:12:1635:1745,&GRPundef@HWI-EAS418:7:63:885:309,&GRPundef@HWI-EAS418:8:100:343:1523,&GRPundef@HWI-EAS418:7:88:363:1275,&GRPundef@HWI-EAS418:7:18:820:1514,&GRPundef@HWI-EAS418:7:5:1593:1989,&GRPundef@HWI-EAS418:7:34:1171:461................&GRPundef@HWI-EAS418:7:12:355:1524,&GRPundef@HWI-EAS418:8:63:602:1590,&GRPu ndef@HWI-EAS418:8:68:1193:532,&GRPundef@HWI-EAS418:7:47:999:1488,&GRPundef@HWI-EAS418:8:81:848:1401,&GRPundef@HWI-EAS418 :7:57:276:1261,&GRPundef@HWI-EAS418:8:35:947:752,&GRPundef@HWI-EAS418:7:86:1155:630 YES_LDAS 30.4759 GT
1.9219 AG 1.5628 ["CCLE","Klijn_CellLines","FA_CancerSupp","ChimerPub","INTERCHROMOSOMAL[chr8--chr17]"]
and the first few lines in the FASTAQ file SKBR3.Left.fq are (SKBR3.Right.fq is similar):
@HWI-EAS418:8:1:3:1091/1 GGGAGCGGCTTCGGGTGCCCTCGNTNGNNTNNNNNNNNNNNNNNNNNNNN +HWI-EAS418:8:1:3:1091 @AABBA9?B?>@1;;,#######!#!#!!#!!!!!!!!!!!!!!!!!!!! @HWI-EAS418:8:1:3:674/1 CACGTGGTCCACAAATGGTTGGCTTNTNNGNNNNNNNNNNNNNNNNNNNN +HWI-EAS418:8:1:3:674 BABA@@<BB=@797@?<?#####!#!!#!!!!!!!!!!!!!!!!!!!! @HWI-EAS418:8:1:3:1772/1 CGAGCCGTGAGCTCGCGCGACCACTNTNNCNNNANNNNNNNNGNNNNNNN +HWI-EAS418:8:1:3:1772 B?BABBA?>?>A;B>B=>:>#####!#!!#!!!#!!!!!!!!#!!!!!!! @HWI-EAS418:8:1:3:1421/1 GGGGTCCACGGGCAACAGCAACCTGNGNNGNNNGNNNNNNNNGNNNNNNN +HWI-EAS418:8:1:3:1421
In particular, I am wondering if the bpipe option -p col="6,8" is correct for the "caller" file I have been using. (The format of this file, produced by STAR-Fusion, is very different from the format of your test example).
Please, let me know if you need more info.
Regards, Gennady