Generate bigWig files where easy/possible (in place or in addition to TDFs)

[GoogleCodeExporter]

We are using wigs/bigWigs and TDFs kind of interchangeably with the same type 
of wiggle data. Many users want wig/bigWig.  I think we should add it as a 
primary visualization file.  Right now we are using TDF, which i'd love to also 
keep.  But I think TDF is a luxury and .bw is a necessity.

Original issue reported on code.google.com by benb...@gmail.com on 3 Oct 2013 at 9:28

[GoogleCodeExporter]

would we be losing anything by dropping TDF? 

afaik TDF is only used in IGV, but if IGV can use bigwig is there any reason to 
have tdfs

Original comment by zack...@gmail.com on 3 Oct 2013 at 9:31

[GoogleCodeExporter]

Original comment by zack...@gmail.com on 3 Oct 2013 at 9:31

[GoogleCodeExporter]

I agree, I did a lot of my analysis on .bw file. bigWig could be converted
 back to .wig file for checking the data value. IGV could definitely accept
.bw file for the visualization. Some groups (e.g. Peggy's lab) are using
IGB or some other browsers, which don't accept .tdf but accept .bw.
I totally agree to just provide .bw rather than .tdf. If we need .tdf in
the future (find some hidden bug in IGV to show .bw file), we could easily
to convert .bw to .tdf file.

-- 
Yaping Liu

PhD candidate
USC Epigenome Center
Genetics Molecular and Cellular Biology Program
University of Southern California
USA

Phone: +1 213 400 2164
Email:lyping1986@gmail.com

Original comment by lyping1...@gmail.com on 3 Oct 2013 at 9:40

[GoogleCodeExporter]

Also notably , UCSC takes bigwig.    So it's a good and popular format.  I 
think I originally wanted to keep it , but at some point we decided it was too 
big.  TDF is IGV specific.  While IGV is also popular, it's not as broad user 
base as bigwig.

Original comment by benb...@gmail.com on 3 Oct 2013 at 9:46

[GoogleCodeExporter]

but does bigwig work well with IGV?

if so, I dont see any use for tdf.

Original comment by zack...@gmail.com on 3 Oct 2013 at 9:48

[GoogleCodeExporter]

bigwigs work great in igv and I use them frequently for diagnostics.  I
never got much from tdf's but mostly was viewing the chipseq tdf output.
It did seem like generation of tdf's was computationally intensive

Original comment by cmnico...@gmail.com on 3 Oct 2013 at 9:56

[GoogleCodeExporter]

I just double check. bigwig works well with IGV. No matter chip-seq, NOMe-seq 
or bisulfite-seq data. 

But indeed, .bw file is larger than .tdf. For example,  HCT116's whole genome 
GCH track:
-rw-r--r-- 1 yapingli bisc-ar 1.3G Aug 31 05:02 
HCT116_merge_C118BACXX_C18FYACXX_D1W2BACXX.fastq-mcf.hg19.mdups.header.clean.mdu
ps.properPaired.nodups.mapped.uniq.realign.recal.cytosine.filtered.sort.GCH.bw

-rw-r--r-- 1 yapingli bisc-ar 850M Aug 31 08:05 
HCT116_merge_C118BACXX_C18FYACXX_D1W2BACXX.fastq-mcf.hg19.mdups.header.clean.mdu
ps.properPaired.nodups.mapped.uniq.realign.recal.cytosine.filtered.sort.GCH.tdf

---
Yaping Liu

PhD candidate 
    in
USC Epigenome Center

University of Southern California

lyping1986@gmail.com

Original comment by lyping1...@gmail.com on 3 Oct 2013 at 10:00

[GoogleCodeExporter]

sounds good to me. 
The only tool that supports TDF is IGV, and since IGV also supports bigwig, 
seems that TDF is pretty much pointless. 

will transition all tdfs to bigwig since there seems to be no downside to 
having tdf disappear.

Original comment by zack...@gmail.com on 3 Oct 2013 at 10:06

[GoogleCodeExporter]

I will add one line in the pipeline to output bigwig file. 

yaping

Original comment by lyping1...@gmail.com on 3 Oct 2013 at 10:12

[GoogleCodeExporter]

I've updated all pipelines to use bigwig. bissnp output can be either bigwig or 
tdf, pipeline will look for both and stage-out whatever is available.

Original comment by zack...@gmail.com on 8 Oct 2013 at 9:23

[GoogleCodeExporter]

bissnp can be updated to produce bw instead of tdf whenever it is 
safe/convenient to do so.

the expected filenames are replace s/tdf/bw

Original comment by zack...@gmail.com on 10 Oct 2013 at 8:03

[GoogleCodeExporter]

Ok, I just make the changes in our pipeline script. I will test it this 
afternoon.
---
Yaping Liu

PhD candidate 
    in
USC Epigenome Center

University of Southern California

lyping1986@gmail.com

Original comment by lyping1...@gmail.com on 10 Oct 2013 at 8:17

[GoogleCodeExporter]

Test finished. It should work well to produce .bw file now~
---
Yaping Liu

PhD candidate 
    in
USC Epigenome Center

University of Southern California

lyping1986@gmail.com

Original comment by lyping1...@gmail.com on 10 Oct 2013 at 8:42