DSL2 implementation of nf-core-hic pipeline

[nservant]

Here is a place where we could exchange about the different modules pour DLS2 implementations. I'm putting here the list of modules I have in mind

Quality Controls

• [x] fastqc

Hi-C data processing

From fastq to a list of interactions = sub-workflows

• [ ] hicpro
  • [x] bowtie2 step1 mapping
  • [x] reads trimming (hicpro C++ implementation)
  • [x] bowtie2 step2
  • [x] hicpro bam pairing
  • [x] hicpro valid pairs detection
• [ ] hicup
  • [ ] hicup trimming
  • [x] bowtie2 mapping
  • [ ] hicup valid pairs detection
• [ ] bwa-mem
  • [ ] bwa-mem with Hi-C option (-SP5M)
  • [ ] valid pairs detection (could use the hicpro module there)

Format convertion

• [x] pairs file (4D)
• [x] (m)cool file
• [ ] hic file

Downstream analysis

All from cool files if possible

• [ ] counts~distance decay with hicExplorer
• [ ] Compartment calling (cooltools)
• [ ] insulation score (cooltools)
• [ ] TADs calling (hicExplorer)
• [ ] loops calling (cooltools, fithic)

[ewels]

A couple of these tools are ready to go already 🎉

• FastQC: https://github.com/nf-core/modules/tree/master/software/fastqc
• BWA: https://github.com/nf-core/modules/tree/master/software/bwa

Once you have your empty(ish) DSL2 pipeline in place, you should be able to install them with nf-core modules install fastqc for example.

Two nf-core/bytesize talks coming up which will be relevant:

• Bytesize 5: DSL2 module development (March 2nd)
• Bytesize 6: DSL2 - Using modules in a pipeline (March 9th)

Phil

[nservant]

Thanks @ewels The other thing I was thinking about is that the Hi-C data processing per se (i.e hicpro, hicup, bwa + pair filtering) should rely more on sub-workflows than on modules

[koustav-pal]

Hi @nservant, I can start working on the bam pairing part of the DSL2?

[nservant]

Hi @koustav-pal Yes, if you want you can start looking at all process related to HiC-Pro, ie.

bowtie2_end_to_end trim_reads bowtie2_on_trimmed_reads bowtie2_merge_mapping_steps dnase_mapping_stats combine_mates get_valid_interaction get_valid_interaction_dnase remove_duplicates merge_stats convert_to_pairs

I did not have time to really look at DSL2 so far, but as we discuss before I think the idea would be to have all these process in a sub-workflow. And also, I guess that the bowtie2 modules already available should also work for HiC-pro.

I guess we can work on a new DLS2 branch in the meantime. And please, try to start from the version which is not yet merged, or from my own fork if it's easier because I change some options to ease the usage of the pipeline. Many thanks to you !

[koustav-pal]

Thanks @nservant,

I will start looking into these function. I think I will use the RNA-seq DSL2 as a reference. Do you prefer that I work on a separate fork or can I be added to the repo? In either case, I will create a new branch containing an empty DSL2 template, that I can start populating.

[nservant]

I think the best practice would be that you work on your own fork, and then ask for PR in this one. Thanks

[koustav-pal]

Hi @nservant,

Apologies for the long silence on my end regarding this. I had to deal with an unexpected cancer related family obligation, over the past two months. Things have settled down for now and I think I can now start looking into this with better clarity.

Thanks.

[nservant]

Hi @koustav-pal Sorry about this ! don't worry at all, there is absolutely no emergency ! I'll to release a new version of the pipeline in DSL1 in a couple of days. Then, we can move on DSL2. Best

[nservant]

Hi @koustav-pal, I was wondering if you already moved on the DSL2 implementation ? otherwise, I will try to have a look in the coming months Thanks N

[nservant]

Hi @koustav-pal, Last call :) any news from your side ? otherwise, I'll try to move on it. Best