split files in hicpro

[skyingg]

Hi, I use macOS. I run hichipper after hicpro. In hicpro I split my raw data to chunks, so the hicpro output is a little different.

1. In bwt2, there is '.bwt2pairs.RSstat' for each chunk, but not for the whole data. The '.mpairstat' record the same thing though.
2. In data, there are '.DEPairs','DumpPairs','.SCPairs','.SinglePairs','.validPairs','.RSstat' for each chunk, but not for the whole data.

hichipper runs fine with test data. When I run it for my own data, I encounter the following error. I do not know what that means. Is it because of the chunk thing?

Tue Mar 07 11:02:52 CST 2017: ['Rscript', u'output1/qcReport.R', '/Users/lab2116/Library/Python/2.7/lib/python/site-packages/hichipper', 'output1', '/Users/lab2116/Desktop/analysis/hichipper-master', '0.4.20', 'split'] /Users/lab2116/Desktop/analysis/hichipper-masterError: pandoc version 1.12.3 or higher is required and was not found (see the help page ?rmarkdown::pandoc_available). Execution halted Processing: split Error in { : task 1 failed - "missing value where TRUE/FALSE needed" Calls: %do% -> Execution halted

PS: another question is about pandoc. I download the rstudio binary and try to install it, but it says can not install windows binary. I tried unzip the file and install the files in /mac, it does not work eaither.

[caleblareau]

Hi, thanks for the note.

First, please note that hichipper will be under pretty extensively modified over the next ~week as I'm trying to finish incorporating all necessary features. There will hopefully be a stable release of version 0.5.0 before the middle of this month. To your specific points--

1/2) could you possibly provide a tarball of the HiC-Pro / a full ls -R output so that I can see the file substructure? This would be of great interest for me.

Your error, insofar as I can tell, only has to do with pandoc. Note that the files we provided on the README are the binaries and not files designed for installation by say double-clicking on the file itself. You basically want to put the file somewhere safe on your computer and then make sure that the pandoc binary is accessible in your bin when running hichipper. Alternatives for installing via say a .pkg are discussed here: http://pandoc.org/installing.html. I'll try to make this a bit more explicit sometime soon.

[skyingg]

This is for my hicpro output folder bowtie_results hic_results config_test_latest.txt hicpro.log

./bowtie_results: bwt2

./bowtie_results/bwt2: split

./bowtie_results/bwt2/split: 00_trimmedS6clean_R1_hg19.bwt2merged.bam 00_trimmedS6clean_R1_hg19.mapstat 00_trimmedS6clean_R2_hg19.bwt2merged.bam 00_trimmedS6clean_R2_hg19.mapstat 00_trimmedS6clean_hg19.bwt2pairs.bam 00_trimmedS6clean_hg19.bwt2pairs.pairstat 01_trimmedS6clean_R1_hg19.bwt2merged.bam 01_trimmedS6clean_R1_hg19.mapstat 01_trimmedS6clean_R2_hg19.bwt2merged.bam 01_trimmedS6clean_R2_hg19.mapstat 01_trimmedS6clean_hg19.bwt2pairs.bam 01_trimmedS6clean_hg19.bwt2pairs.pairstat 02_trimmedS6clean_R1_hg19.bwt2merged.bam 02_trimmedS6clean_R1_hg19.mapstat 02_trimmedS6clean_R2_hg19.bwt2merged.bam 02_trimmedS6clean_R2_hg19.mapstat 02_trimmedS6clean_hg19.bwt2pairs.bam 02_trimmedS6clean_hg19.bwt2pairs.pairstat 03_trimmedS6clean_R1_hg19.bwt2merged.bam 03_trimmedS6clean_R1_hg19.mapstat 03_trimmedS6clean_R2_hg19.bwt2merged.bam 03_trimmedS6clean_R2_hg19.mapstat 03_trimmedS6clean_hg19.bwt2pairs.bam 03_trimmedS6clean_hg19.bwt2pairs.pairstat 04_trimmedS6clean_R1_hg19.bwt2merged.bam 04_trimmedS6clean_R1_hg19.mapstat 04_trimmedS6clean_R2_hg19.bwt2merged.bam 04_trimmedS6clean_R2_hg19.mapstat 04_trimmedS6clean_hg19.bwt2pairs.bam 04_trimmedS6clean_hg19.bwt2pairs.pairstat 05_trimmedS6clean_R1_hg19.bwt2merged.bam 05_trimmedS6clean_R1_hg19.mapstat 05_trimmedS6clean_R2_hg19.bwt2merged.bam 05_trimmedS6clean_R2_hg19.mapstat 05_trimmedS6clean_hg19.bwt2pairs.bam 05_trimmedS6clean_hg19.bwt2pairs.pairstat 06_trimmedS6clean_R1_hg19.bwt2merged.bam 06_trimmedS6clean_R1_hg19.mapstat 06_trimmedS6clean_R2_hg19.bwt2merged.bam 06_trimmedS6clean_R2_hg19.mapstat 06_trimmedS6clean_hg19.bwt2pairs.bam 06_trimmedS6clean_hg19.bwt2pairs.pairstat 07_trimmedS6clean_R1_hg19.bwt2merged.bam 07_trimmedS6clean_R1_hg19.mapstat 07_trimmedS6clean_R2_hg19.bwt2merged.bam 07_trimmedS6clean_R2_hg19.mapstat 07_trimmedS6clean_hg19.bwt2pairs.bam 07_trimmedS6clean_hg19.bwt2pairs.pairstat 08_trimmedS6clean_R1_hg19.bwt2merged.bam 08_trimmedS6clean_R1_hg19.mapstat 08_trimmedS6clean_R2_hg19.bwt2merged.bam 08_trimmedS6clean_R2_hg19.mapstat 08_trimmedS6clean_hg19.bwt2pairs.bam 08_trimmedS6clean_hg19.bwt2pairs.pairstat 09_trimmedS6clean_R1_hg19.bwt2merged.bam 09_trimmedS6clean_R1_hg19.mapstat 09_trimmedS6clean_R2_hg19.bwt2merged.bam 09_trimmedS6clean_R2_hg19.mapstat 09_trimmedS6clean_hg19.bwt2pairs.bam 09_trimmedS6clean_hg19.bwt2pairs.pairstat 10_trimmedS6clean_R1_hg19.bwt2merged.bam 10_trimmedS6clean_R1_hg19.mapstat 10_trimmedS6clean_R2_hg19.bwt2merged.bam 10_trimmedS6clean_R2_hg19.mapstat 10_trimmedS6clean_hg19.bwt2pairs.bam 10_trimmedS6clean_hg19.bwt2pairs.pairstat 11_trimmedS6clean_R1_hg19.bwt2merged.bam 11_trimmedS6clean_R1_hg19.mapstat 11_trimmedS6clean_R2_hg19.bwt2merged.bam 11_trimmedS6clean_R2_hg19.mapstat 11_trimmedS6clean_hg19.bwt2pairs.bam 11_trimmedS6clean_hg19.bwt2pairs.pairstat 12_trimmedS6clean_R1_hg19.bwt2merged.bam 12_trimmedS6clean_R1_hg19.mapstat 12_trimmedS6clean_R2_hg19.bwt2merged.bam 12_trimmedS6clean_R2_hg19.mapstat 12_trimmedS6clean_hg19.bwt2pairs.bam 12_trimmedS6clean_hg19.bwt2pairs.pairstat 13_trimmedS6clean_R1_hg19.bwt2merged.bam 13_trimmedS6clean_R1_hg19.mapstat 13_trimmedS6clean_R2_hg19.bwt2merged.bam 13_trimmedS6clean_R2_hg19.mapstat 13_trimmedS6clean_hg19.bwt2pairs.bam 13_trimmedS6clean_hg19.bwt2pairs.pairstat 14_trimmedS6clean_R1_hg19.bwt2merged.bam 14_trimmedS6clean_R1_hg19.mapstat 14_trimmedS6clean_R2_hg19.bwt2merged.bam 14_trimmedS6clean_R2_hg19.mapstat 14_trimmedS6clean_hg19.bwt2pairs.bam 14_trimmedS6clean_hg19.bwt2pairs.pairstat 15_trimmedS6clean_R1_hg19.bwt2merged.bam 15_trimmedS6clean_R1_hg19.mapstat 15_trimmedS6clean_R2_hg19.bwt2merged.bam 15_trimmedS6clean_R2_hg19.mapstat 15_trimmedS6clean_hg19.bwt2pairs.bam 15_trimmedS6clean_hg19.bwt2pairs.pairstat 16_trimmedS6clean_R1_hg19.bwt2merged.bam 16_trimmedS6clean_R1_hg19.mapstat 16_trimmedS6clean_R2_hg19.bwt2merged.bam 16_trimmedS6clean_R2_hg19.mapstat 16_trimmedS6clean_hg19.bwt2pairs.bam 16_trimmedS6clean_hg19.bwt2pairs.pairstat 17_trimmedS6clean_R1_hg19.bwt2merged.bam 17_trimmedS6clean_R1_hg19.mapstat 17_trimmedS6clean_R2_hg19.bwt2merged.bam 17_trimmedS6clean_R2_hg19.mapstat 17_trimmedS6clean_hg19.bwt2pairs.bam 17_trimmedS6clean_hg19.bwt2pairs.pairstat 18_trimmedS6clean_R1_hg19.bwt2merged.bam 18_trimmedS6clean_R1_hg19.mapstat 18_trimmedS6clean_R2_hg19.bwt2merged.bam 18_trimmedS6clean_R2_hg19.mapstat 18_trimmedS6clean_hg19.bwt2pairs.bam 18_trimmedS6clean_hg19.bwt2pairs.pairstat 19_trimmedS6clean_R1_hg19.bwt2merged.bam 19_trimmedS6clean_R1_hg19.mapstat 19_trimmedS6clean_R2_hg19.bwt2merged.bam 19_trimmedS6clean_R2_hg19.mapstat 19_trimmedS6clean_hg19.bwt2pairs.bam 19_trimmedS6clean_hg19.bwt2pairs.pairstat 20_trimmedS6clean_R1_hg19.bwt2merged.bam 20_trimmedS6clean_R1_hg19.mapstat 20_trimmedS6clean_R2_hg19.bwt2merged.bam 20_trimmedS6clean_R2_hg19.mapstat 20_trimmedS6clean_hg19.bwt2pairs.bam 20_trimmedS6clean_hg19.bwt2pairs.pairstat 21_trimmedS6clean_R1_hg19.bwt2merged.bam 21_trimmedS6clean_R1_hg19.mapstat 21_trimmedS6clean_R2_hg19.bwt2merged.bam 21_trimmedS6clean_R2_hg19.mapstat 21_trimmedS6clean_hg19.bwt2pairs.bam 21_trimmedS6clean_hg19.bwt2pairs.pairstat 22_trimmedS6clean_R1_hg19.bwt2merged.bam 22_trimmedS6clean_R1_hg19.mapstat 22_trimmedS6clean_R2_hg19.bwt2merged.bam 22_trimmedS6clean_R2_hg19.mapstat 22_trimmedS6clean_hg19.bwt2pairs.bam 22_trimmedS6clean_hg19.bwt2pairs.pairstat 23_trimmedS6clean_R1_hg19.bwt2merged.bam 23_trimmedS6clean_R1_hg19.mapstat 23_trimmedS6clean_R2_hg19.bwt2merged.bam 23_trimmedS6clean_R2_hg19.mapstat 23_trimmedS6clean_hg19.bwt2pairs.bam 23_trimmedS6clean_hg19.bwt2pairs.pairstat 24_trimmedS6clean_R1_hg19.bwt2merged.bam 24_trimmedS6clean_R1_hg19.mapstat 24_trimmedS6clean_R2_hg19.bwt2merged.bam 24_trimmedS6clean_R2_hg19.mapstat 24_trimmedS6clean_hg19.bwt2pairs.bam 24_trimmedS6clean_hg19.bwt2pairs.pairstat 25_trimmedS6clean_R1_hg19.bwt2merged.bam 25_trimmedS6clean_R1_hg19.mapstat 25_trimmedS6clean_R2_hg19.bwt2merged.bam 25_trimmedS6clean_R2_hg19.mapstat 25_trimmedS6clean_hg19.bwt2pairs.bam 25_trimmedS6clean_hg19.bwt2pairs.pairstat 26_trimmedS6clean_R1_hg19.bwt2merged.bam 26_trimmedS6clean_R1_hg19.mapstat 26_trimmedS6clean_R2_hg19.bwt2merged.bam 26_trimmedS6clean_R2_hg19.mapstat 26_trimmedS6clean_hg19.bwt2pairs.bam 26_trimmedS6clean_hg19.bwt2pairs.pairstat 27_trimmedS6clean_R1_hg19.bwt2merged.bam 27_trimmedS6clean_R1_hg19.mapstat 27_trimmedS6clean_R2_hg19.bwt2merged.bam 27_trimmedS6clean_R2_hg19.mapstat 27_trimmedS6clean_hg19.bwt2pairs.bam 27_trimmedS6clean_hg19.bwt2pairs.pairstat 28_trimmedS6clean_R1_hg19.bwt2merged.bam 28_trimmedS6clean_R1_hg19.mapstat 28_trimmedS6clean_R2_hg19.bwt2merged.bam 28_trimmedS6clean_R2_hg19.mapstat 28_trimmedS6clean_hg19.bwt2pairs.bam 28_trimmedS6clean_hg19.bwt2pairs.pairstat 29_trimmedS6clean_R1_hg19.bwt2merged.bam 29_trimmedS6clean_R1_hg19.mapstat 29_trimmedS6clean_R2_hg19.bwt2merged.bam 29_trimmedS6clean_R2_hg19.mapstat 29_trimmedS6clean_hg19.bwt2pairs.bam 29_trimmedS6clean_hg19.bwt2pairs.pairstat 30_trimmedS6clean_R1_hg19.bwt2merged.bam 30_trimmedS6clean_R1_hg19.mapstat 30_trimmedS6clean_R2_hg19.bwt2merged.bam 30_trimmedS6clean_R2_hg19.mapstat 30_trimmedS6clean_hg19.bwt2pairs.bam 30_trimmedS6clean_hg19.bwt2pairs.pairstat 31_trimmedS6clean_R1_hg19.bwt2merged.bam 31_trimmedS6clean_R1_hg19.mapstat 31_trimmedS6clean_R2_hg19.bwt2merged.bam 31_trimmedS6clean_R2_hg19.mapstat 31_trimmedS6clean_hg19.bwt2pairs.bam 31_trimmedS6clean_hg19.bwt2pairs.pairstat 32_trimmedS6clean_R1_hg19.bwt2merged.bam 32_trimmedS6clean_R1_hg19.mapstat 32_trimmedS6clean_R2_hg19.bwt2merged.bam 32_trimmedS6clean_R2_hg19.mapstat 32_trimmedS6clean_hg19.bwt2pairs.bam 32_trimmedS6clean_hg19.bwt2pairs.pairstat 33_trimmedS6clean_R1_hg19.bwt2merged.bam 33_trimmedS6clean_R1_hg19.mapstat 33_trimmedS6clean_R2_hg19.bwt2merged.bam 33_trimmedS6clean_R2_hg19.mapstat 33_trimmedS6clean_hg19.bwt2pairs.bam 33_trimmedS6clean_hg19.bwt2pairs.pairstat split.bwt2pairs.pairstat split.mpairstat splitR1.mmapstat splitR2.mmapstat

./hic_results: data matrix pic

./hic_results/data: split

./hic_results/data/split: 00_trimmedS6clean_hg19.bwt2pairs.DEPairs 00_trimmedS6clean_hg19.bwt2pairs.DumpPairs 00_trimmedS6clean_hg19.bwt2pairs.REPairs 00_trimmedS6clean_hg19.bwt2pairs.RSstat 00_trimmedS6clean_hg19.bwt2pairs.SCPairs 00_trimmedS6clean_hg19.bwt2pairs.SinglePairs 00_trimmedS6clean_hg19.bwt2pairs.validPairs 00_trimmedS6clean_hg19.bwt2pairs_interaction.sam 01_trimmedS6clean_hg19.bwt2pairs.DEPairs 01_trimmedS6clean_hg19.bwt2pairs.DumpPairs 01_trimmedS6clean_hg19.bwt2pairs.REPairs 01_trimmedS6clean_hg19.bwt2pairs.RSstat 01_trimmedS6clean_hg19.bwt2pairs.SCPairs 01_trimmedS6clean_hg19.bwt2pairs.SinglePairs 01_trimmedS6clean_hg19.bwt2pairs.validPairs 01_trimmedS6clean_hg19.bwt2pairs_interaction.sam 02_trimmedS6clean_hg19.bwt2pairs.DEPairs 02_trimmedS6clean_hg19.bwt2pairs.DumpPairs 02_trimmedS6clean_hg19.bwt2pairs.REPairs 02_trimmedS6clean_hg19.bwt2pairs.RSstat 02_trimmedS6clean_hg19.bwt2pairs.SCPairs 02_trimmedS6clean_hg19.bwt2pairs.SinglePairs 02_trimmedS6clean_hg19.bwt2pairs.validPairs 02_trimmedS6clean_hg19.bwt2pairs_interaction.sam 03_trimmedS6clean_hg19.bwt2pairs.DEPairs 03_trimmedS6clean_hg19.bwt2pairs.DumpPairs 03_trimmedS6clean_hg19.bwt2pairs.REPairs 03_trimmedS6clean_hg19.bwt2pairs.RSstat 03_trimmedS6clean_hg19.bwt2pairs.SCPairs 03_trimmedS6clean_hg19.bwt2pairs.SinglePairs 03_trimmedS6clean_hg19.bwt2pairs.validPairs 03_trimmedS6clean_hg19.bwt2pairs_interaction.sam 04_trimmedS6clean_hg19.bwt2pairs.DEPairs 04_trimmedS6clean_hg19.bwt2pairs.DumpPairs 04_trimmedS6clean_hg19.bwt2pairs.REPairs 04_trimmedS6clean_hg19.bwt2pairs.RSstat 04_trimmedS6clean_hg19.bwt2pairs.SCPairs 04_trimmedS6clean_hg19.bwt2pairs.SinglePairs 04_trimmedS6clean_hg19.bwt2pairs.validPairs 04_trimmedS6clean_hg19.bwt2pairs_interaction.sam 05_trimmedS6clean_hg19.bwt2pairs.DEPairs 05_trimmedS6clean_hg19.bwt2pairs.DumpPairs 05_trimmedS6clean_hg19.bwt2pairs.REPairs 05_trimmedS6clean_hg19.bwt2pairs.RSstat 05_trimmedS6clean_hg19.bwt2pairs.SCPairs 05_trimmedS6clean_hg19.bwt2pairs.SinglePairs 05_trimmedS6clean_hg19.bwt2pairs.validPairs 05_trimmedS6clean_hg19.bwt2pairs_interaction.sam 06_trimmedS6clean_hg19.bwt2pairs.DEPairs 06_trimmedS6clean_hg19.bwt2pairs.DumpPairs 06_trimmedS6clean_hg19.bwt2pairs.REPairs 06_trimmedS6clean_hg19.bwt2pairs.RSstat 06_trimmedS6clean_hg19.bwt2pairs.SCPairs 06_trimmedS6clean_hg19.bwt2pairs.SinglePairs 06_trimmedS6clean_hg19.bwt2pairs.validPairs 06_trimmedS6clean_hg19.bwt2pairs_interaction.sam 07_trimmedS6clean_hg19.bwt2pairs.DEPairs 07_trimmedS6clean_hg19.bwt2pairs.DumpPairs 07_trimmedS6clean_hg19.bwt2pairs.REPairs 07_trimmedS6clean_hg19.bwt2pairs.RSstat 07_trimmedS6clean_hg19.bwt2pairs.SCPairs 07_trimmedS6clean_hg19.bwt2pairs.SinglePairs 07_trimmedS6clean_hg19.bwt2pairs.validPairs 07_trimmedS6clean_hg19.bwt2pairs_interaction.sam 08_trimmedS6clean_hg19.bwt2pairs.DEPairs 08_trimmedS6clean_hg19.bwt2pairs.DumpPairs 08_trimmedS6clean_hg19.bwt2pairs.REPairs 08_trimmedS6clean_hg19.bwt2pairs.RSstat 08_trimmedS6clean_hg19.bwt2pairs.SCPairs 08_trimmedS6clean_hg19.bwt2pairs.SinglePairs 08_trimmedS6clean_hg19.bwt2pairs.validPairs 08_trimmedS6clean_hg19.bwt2pairs_interaction.sam 09_trimmedS6clean_hg19.bwt2pairs.DEPairs 09_trimmedS6clean_hg19.bwt2pairs.DumpPairs 09_trimmedS6clean_hg19.bwt2pairs.REPairs 09_trimmedS6clean_hg19.bwt2pairs.RSstat 09_trimmedS6clean_hg19.bwt2pairs.SCPairs 09_trimmedS6clean_hg19.bwt2pairs.SinglePairs 09_trimmedS6clean_hg19.bwt2pairs.validPairs 09_trimmedS6clean_hg19.bwt2pairs_interaction.sam 10_trimmedS6clean_hg19.bwt2pairs.DEPairs 10_trimmedS6clean_hg19.bwt2pairs.DumpPairs 10_trimmedS6clean_hg19.bwt2pairs.REPairs 10_trimmedS6clean_hg19.bwt2pairs.RSstat 10_trimmedS6clean_hg19.bwt2pairs.SCPairs 10_trimmedS6clean_hg19.bwt2pairs.SinglePairs 10_trimmedS6clean_hg19.bwt2pairs.validPairs 10_trimmedS6clean_hg19.bwt2pairs_interaction.sam 11_trimmedS6clean_hg19.bwt2pairs.DEPairs 11_trimmedS6clean_hg19.bwt2pairs.DumpPairs 11_trimmedS6clean_hg19.bwt2pairs.REPairs 11_trimmedS6clean_hg19.bwt2pairs.RSstat 11_trimmedS6clean_hg19.bwt2pairs.SCPairs 11_trimmedS6clean_hg19.bwt2pairs.SinglePairs 11_trimmedS6clean_hg19.bwt2pairs.validPairs 11_trimmedS6clean_hg19.bwt2pairs_interaction.sam 12_trimmedS6clean_hg19.bwt2pairs.DEPairs 12_trimmedS6clean_hg19.bwt2pairs.DumpPairs 12_trimmedS6clean_hg19.bwt2pairs.REPairs 12_trimmedS6clean_hg19.bwt2pairs.RSstat 12_trimmedS6clean_hg19.bwt2pairs.SCPairs 12_trimmedS6clean_hg19.bwt2pairs.SinglePairs 12_trimmedS6clean_hg19.bwt2pairs.validPairs 12_trimmedS6clean_hg19.bwt2pairs_interaction.sam 13_trimmedS6clean_hg19.bwt2pairs.DEPairs 13_trimmedS6clean_hg19.bwt2pairs.DumpPairs 13_trimmedS6clean_hg19.bwt2pairs.REPairs 13_trimmedS6clean_hg19.bwt2pairs.RSstat 13_trimmedS6clean_hg19.bwt2pairs.SCPairs 13_trimmedS6clean_hg19.bwt2pairs.SinglePairs 13_trimmedS6clean_hg19.bwt2pairs.validPairs 13_trimmedS6clean_hg19.bwt2pairs_interaction.sam 14_trimmedS6clean_hg19.bwt2pairs.DEPairs 14_trimmedS6clean_hg19.bwt2pairs.DumpPairs 14_trimmedS6clean_hg19.bwt2pairs.REPairs 14_trimmedS6clean_hg19.bwt2pairs.RSstat 14_trimmedS6clean_hg19.bwt2pairs.SCPairs 14_trimmedS6clean_hg19.bwt2pairs.SinglePairs 14_trimmedS6clean_hg19.bwt2pairs.validPairs 14_trimmedS6clean_hg19.bwt2pairs_interaction.sam 15_trimmedS6clean_hg19.bwt2pairs.DEPairs 15_trimmedS6clean_hg19.bwt2pairs.DumpPairs 15_trimmedS6clean_hg19.bwt2pairs.REPairs 15_trimmedS6clean_hg19.bwt2pairs.RSstat 15_trimmedS6clean_hg19.bwt2pairs.SCPairs 15_trimmedS6clean_hg19.bwt2pairs.SinglePairs 15_trimmedS6clean_hg19.bwt2pairs.validPairs 15_trimmedS6clean_hg19.bwt2pairs_interaction.sam 16_trimmedS6clean_hg19.bwt2pairs.DEPairs 16_trimmedS6clean_hg19.bwt2pairs.DumpPairs 16_trimmedS6clean_hg19.bwt2pairs.REPairs 16_trimmedS6clean_hg19.bwt2pairs.RSstat 16_trimmedS6clean_hg19.bwt2pairs.SCPairs 16_trimmedS6clean_hg19.bwt2pairs.SinglePairs 16_trimmedS6clean_hg19.bwt2pairs.validPairs 16_trimmedS6clean_hg19.bwt2pairs_interaction.sam 17_trimmedS6clean_hg19.bwt2pairs.DEPairs 17_trimmedS6clean_hg19.bwt2pairs.DumpPairs 17_trimmedS6clean_hg19.bwt2pairs.REPairs 17_trimmedS6clean_hg19.bwt2pairs.RSstat 17_trimmedS6clean_hg19.bwt2pairs.SCPairs 17_trimmedS6clean_hg19.bwt2pairs.SinglePairs 17_trimmedS6clean_hg19.bwt2pairs.validPairs 17_trimmedS6clean_hg19.bwt2pairs_interaction.sam 18_trimmedS6clean_hg19.bwt2pairs.DEPairs 18_trimmedS6clean_hg19.bwt2pairs.DumpPairs 18_trimmedS6clean_hg19.bwt2pairs.REPairs 18_trimmedS6clean_hg19.bwt2pairs.RSstat 18_trimmedS6clean_hg19.bwt2pairs.SCPairs 18_trimmedS6clean_hg19.bwt2pairs.SinglePairs 18_trimmedS6clean_hg19.bwt2pairs.validPairs 18_trimmedS6clean_hg19.bwt2pairs_interaction.sam 19_trimmedS6clean_hg19.bwt2pairs.DEPairs 19_trimmedS6clean_hg19.bwt2pairs.DumpPairs 19_trimmedS6clean_hg19.bwt2pairs.REPairs 19_trimmedS6clean_hg19.bwt2pairs.RSstat 19_trimmedS6clean_hg19.bwt2pairs.SCPairs 19_trimmedS6clean_hg19.bwt2pairs.SinglePairs 19_trimmedS6clean_hg19.bwt2pairs.validPairs 19_trimmedS6clean_hg19.bwt2pairs_interaction.sam 20_trimmedS6clean_hg19.bwt2pairs.DEPairs 20_trimmedS6clean_hg19.bwt2pairs.DumpPairs 20_trimmedS6clean_hg19.bwt2pairs.REPairs 20_trimmedS6clean_hg19.bwt2pairs.RSstat 20_trimmedS6clean_hg19.bwt2pairs.SCPairs 20_trimmedS6clean_hg19.bwt2pairs.SinglePairs 20_trimmedS6clean_hg19.bwt2pairs.validPairs 20_trimmedS6clean_hg19.bwt2pairs_interaction.sam 21_trimmedS6clean_hg19.bwt2pairs.DEPairs 21_trimmedS6clean_hg19.bwt2pairs.DumpPairs 21_trimmedS6clean_hg19.bwt2pairs.REPairs 21_trimmedS6clean_hg19.bwt2pairs.RSstat 21_trimmedS6clean_hg19.bwt2pairs.SCPairs 21_trimmedS6clean_hg19.bwt2pairs.SinglePairs 21_trimmedS6clean_hg19.bwt2pairs.validPairs 21_trimmedS6clean_hg19.bwt2pairs_interaction.sam 22_trimmedS6clean_hg19.bwt2pairs.DEPairs 22_trimmedS6clean_hg19.bwt2pairs.DumpPairs 22_trimmedS6clean_hg19.bwt2pairs.REPairs 22_trimmedS6clean_hg19.bwt2pairs.RSstat 22_trimmedS6clean_hg19.bwt2pairs.SCPairs 22_trimmedS6clean_hg19.bwt2pairs.SinglePairs 22_trimmedS6clean_hg19.bwt2pairs.validPairs 22_trimmedS6clean_hg19.bwt2pairs_interaction.sam 23_trimmedS6clean_hg19.bwt2pairs.DEPairs 23_trimmedS6clean_hg19.bwt2pairs.DumpPairs 23_trimmedS6clean_hg19.bwt2pairs.REPairs 23_trimmedS6clean_hg19.bwt2pairs.RSstat 23_trimmedS6clean_hg19.bwt2pairs.SCPairs 23_trimmedS6clean_hg19.bwt2pairs.SinglePairs 23_trimmedS6clean_hg19.bwt2pairs.validPairs 23_trimmedS6clean_hg19.bwt2pairs_interaction.sam 24_trimmedS6clean_hg19.bwt2pairs.DEPairs 24_trimmedS6clean_hg19.bwt2pairs.DumpPairs 24_trimmedS6clean_hg19.bwt2pairs.REPairs 24_trimmedS6clean_hg19.bwt2pairs.RSstat 24_trimmedS6clean_hg19.bwt2pairs.SCPairs 24_trimmedS6clean_hg19.bwt2pairs.SinglePairs 24_trimmedS6clean_hg19.bwt2pairs.validPairs 24_trimmedS6clean_hg19.bwt2pairs_interaction.sam 25_trimmedS6clean_hg19.bwt2pairs.DEPairs 25_trimmedS6clean_hg19.bwt2pairs.DumpPairs 25_trimmedS6clean_hg19.bwt2pairs.REPairs 25_trimmedS6clean_hg19.bwt2pairs.RSstat 25_trimmedS6clean_hg19.bwt2pairs.SCPairs 25_trimmedS6clean_hg19.bwt2pairs.SinglePairs 25_trimmedS6clean_hg19.bwt2pairs.validPairs 25_trimmedS6clean_hg19.bwt2pairs_interaction.sam 26_trimmedS6clean_hg19.bwt2pairs.DEPairs 26_trimmedS6clean_hg19.bwt2pairs.DumpPairs 26_trimmedS6clean_hg19.bwt2pairs.REPairs 26_trimmedS6clean_hg19.bwt2pairs.RSstat 26_trimmedS6clean_hg19.bwt2pairs.SCPairs 26_trimmedS6clean_hg19.bwt2pairs.SinglePairs 26_trimmedS6clean_hg19.bwt2pairs.validPairs 26_trimmedS6clean_hg19.bwt2pairs_interaction.sam 27_trimmedS6clean_hg19.bwt2pairs.DEPairs 27_trimmedS6clean_hg19.bwt2pairs.DumpPairs 27_trimmedS6clean_hg19.bwt2pairs.REPairs 27_trimmedS6clean_hg19.bwt2pairs.RSstat 27_trimmedS6clean_hg19.bwt2pairs.SCPairs 27_trimmedS6clean_hg19.bwt2pairs.SinglePairs 27_trimmedS6clean_hg19.bwt2pairs.validPairs 27_trimmedS6clean_hg19.bwt2pairs_interaction.sam 28_trimmedS6clean_hg19.bwt2pairs.DEPairs 28_trimmedS6clean_hg19.bwt2pairs.DumpPairs 28_trimmedS6clean_hg19.bwt2pairs.REPairs 28_trimmedS6clean_hg19.bwt2pairs.RSstat 28_trimmedS6clean_hg19.bwt2pairs.SCPairs 28_trimmedS6clean_hg19.bwt2pairs.SinglePairs 28_trimmedS6clean_hg19.bwt2pairs.validPairs 28_trimmedS6clean_hg19.bwt2pairs_interaction.sam 29_trimmedS6clean_hg19.bwt2pairs.DEPairs 29_trimmedS6clean_hg19.bwt2pairs.DumpPairs 29_trimmedS6clean_hg19.bwt2pairs.REPairs 29_trimmedS6clean_hg19.bwt2pairs.RSstat 29_trimmedS6clean_hg19.bwt2pairs.SCPairs 29_trimmedS6clean_hg19.bwt2pairs.SinglePairs 29_trimmedS6clean_hg19.bwt2pairs.validPairs 29_trimmedS6clean_hg19.bwt2pairs_interaction.sam 30_trimmedS6clean_hg19.bwt2pairs.DEPairs 30_trimmedS6clean_hg19.bwt2pairs.DumpPairs 30_trimmedS6clean_hg19.bwt2pairs.REPairs 30_trimmedS6clean_hg19.bwt2pairs.RSstat 30_trimmedS6clean_hg19.bwt2pairs.SCPairs 30_trimmedS6clean_hg19.bwt2pairs.SinglePairs 30_trimmedS6clean_hg19.bwt2pairs.validPairs 30_trimmedS6clean_hg19.bwt2pairs_interaction.sam 31_trimmedS6clean_hg19.bwt2pairs.DEPairs 31_trimmedS6clean_hg19.bwt2pairs.DumpPairs 31_trimmedS6clean_hg19.bwt2pairs.REPairs 31_trimmedS6clean_hg19.bwt2pairs.RSstat 31_trimmedS6clean_hg19.bwt2pairs.SCPairs 31_trimmedS6clean_hg19.bwt2pairs.SinglePairs 31_trimmedS6clean_hg19.bwt2pairs.validPairs 31_trimmedS6clean_hg19.bwt2pairs_interaction.sam 32_trimmedS6clean_hg19.bwt2pairs.DEPairs 32_trimmedS6clean_hg19.bwt2pairs.DumpPairs 32_trimmedS6clean_hg19.bwt2pairs.REPairs 32_trimmedS6clean_hg19.bwt2pairs.RSstat 32_trimmedS6clean_hg19.bwt2pairs.SCPairs 32_trimmedS6clean_hg19.bwt2pairs.SinglePairs 32_trimmedS6clean_hg19.bwt2pairs.validPairs 32_trimmedS6clean_hg19.bwt2pairs_interaction.sam 33_trimmedS6clean_hg19.bwt2pairs.DEPairs 33_trimmedS6clean_hg19.bwt2pairs.DumpPairs 33_trimmedS6clean_hg19.bwt2pairs.REPairs 33_trimmedS6clean_hg19.bwt2pairs.RSstat 33_trimmedS6clean_hg19.bwt2pairs.SCPairs 33_trimmedS6clean_hg19.bwt2pairs.SinglePairs 33_trimmedS6clean_hg19.bwt2pairs.validPairs 33_trimmedS6clean_hg19.bwt2pairs_interaction.sam split.mRSstat split_allValidPairs split_allValidPairs.mergestat

./hic_results/matrix: split

./hic_results/matrix/split: iced raw

./hic_results/matrix/split/iced: 1000000 20000 40000 500000

./hic_results/matrix/split/iced/1000000: split_1000000_iced.matrix split_1000000_iced.matrix.biases

./hic_results/matrix/split/iced/20000: split_20000_iced.matrix split_20000_iced.matrix.biases

./hic_results/matrix/split/iced/40000: split_40000_iced.matrix split_40000_iced.matrix.biases

./hic_results/matrix/split/iced/500000: split_500000_iced.matrix split_500000_iced.matrix.biases

./hic_results/matrix/split/raw: 1000000 20000 40000 500000

./hic_results/matrix/split/raw/1000000: split_1000000.matrix split_1000000_abs.bed split_1000000_ord.bed

./hic_results/matrix/split/raw/20000: split_20000.matrix split_20000_abs.bed split_20000_ord.bed

./hic_results/matrix/split/raw/40000: split_40000.matrix split_40000_abs.bed split_40000_ord.bed

./hic_results/matrix/split/raw/500000: split_500000.matrix split_500000_abs.bed split_500000_ord.bed

./hic_results/pic: split

./hic_results/pic/split:

[caleblareau]

@skyingg-- taking a look at this now To clarify, it looks like you called your sample "split", right? That wasn't a feature of HiC-Pro or bowtie2? I think that I should be able to solve this case use by using a couple of regular expressions... Working on it now.

[skyingg]

Yes, I name the sample 'split'

@skyingg-- taking a look at this now To clarify, it looks like you called your sample "split", right? That wasn't a feature of HiC-Pro or bowtie2? I think that I should be able to solve this case use by using a couple of regular expressions... Working on it now.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/aryeelab/hichipper/issues/7#issuecomment-285958730, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AX3_uFU2YaqcWN0pJLmMPFAURjKUwTHwks5rlCdDgaJpZM4MU_5J.

[caleblareau]

Okay, I'll push a version in the next 4-6 hours (0.4.9 or 0.5.0 depending on how satisfied I am with things) that should work for this. Basically, I just use more regular expressions that don't make any assumptions about the number of files in these sub directories. I created some dummy input like you have shown and it seemed to work fine now.

Note that the tool has undergone a massive overhaul and I'm still editing the README. I'll make a minor note about the split files but probably won't show a detailed example. The tool should be stable though in another day or two. At that point, let me know if you aren't getting satisfactory results.

One note: if you do decide to use the new background correction algorithm, there's a chance that it will crash due to insufficient RAM for background correction (also depends on sequencing depth). If you have ChIP-Seq peaks or some other sort of peaks and can get around using our adaptive background peak calling, that may be best if you have to run it on your mac. Alternatively, using a unix server with more memory should work (again, assuming that you have > 100M reads in your sample and less than say 16 gigs of RAM).

[skyingg]

Hi caleblareau,

The new version does not work (I still use split as my sample name). The old version 0.4.20 works somehow and I could get what I need. Could please provide both of release?

lab2116s-iMac:mydata lab2116$ hichipper --out test my.yaml Wed Mar 29 20:58:24 CST 2017: Starting hichipper pipeline v0.5.0 Wed Mar 29 20:58:24 CST 2017: Parsing user parameters ERROR: Missing hic_results or bowtie_results files for split; either exclude the sample or manage the file architecture/input lab2116s-iMac:mydata lab2116$

[caleblareau]

Sorry being lazy on this; I finally tracked down the issue where HiC-Pro creates a allValidPairs file that is uniquely generated when input fastq files are split. I'll push v0.5.3 shortly that should finally hopefully fix this

[caleblareau]

@skyingg I'll wait to hear from you before I close this, but after running it on split files, everything seemed to work.

[clagiamba]

Hi,

How did this get solved at the end? Do we need to call the chunked files "split"?

Thank you, Claudia

[clagiamba]

Posting Caleb's reply below.

1) The split file issue (as you noted on GitHub) was solved a while back and is not the source of the problem. There isn’t a need to run this without the files being split nor do you need to change the names of anything.

2) The issue is that hichipper assumes that you have additional files that are presently missing from your execution… note:

http://hichipper.readthedocs.io/en/latest/content/Usage.html#more-typical-example

You don’t have DEpairs or the SCpairs files, which is what’s throwing the error. An update of HiC-Pro will fix this. Or, you could apply my hack and delete only one of the DumpPairs. This will make your output *Pairs files divisible by five, which will satisfy the error that is being thrown here:

https://github.com/aryeelab/hichipper/blob/2b8787c538148ddf81be0fca3a78175a0d627485/hichipper/hichipperHelp.py#L225

If you were to update your Hi-C Pro (2.7.6 is fairly old), then this should work automatically.

Hope this helps, -caleb