search

nservant / HiC-Pro

HiC-Pro: An optimized and flexible pipeline for Hi-C data processing
Other
386 stars 182 forks source link

Could not retrieve index file, all data goes dump #405

Closed OK1810 closed 3 years ago

OK1810 commented 3 years ago

Hello,

I would appreciate if you could give an idea on how to fix the issue that I came across: When running hicpro.sh, all reads go to dump folder and therefore Terminal exit with the error:

Run quality checks for all samples ...
Logs: logs/DataSet/make_Rplots.log
make: *** [hic_qc] Error 1
mv: cannot stat '/Users/belllab/Desktop/Peter/hicpro_result/hic_results/matrix/Dataset/raw/2000/Dataset_2000.matrix': No such file or directory
/Users/belllab/Desktop/Peter/hicpro.sh:26: no such file or directory: /Users/belllab/Desktop/Peter/hicpro_result/hic_results/matrix/Dataset/raw/2000/Dataset_2000.matrix
mv: cannot stat '/Users/belllab/Desktop/Peter/hicpro_result/hic_results/matrix/Dataset/raw/10000/Dataset_10000.matrix': No such file or directory
/Users/belllab/Desktop/Peter/hicpro.sh:26: no such file or directory: /Users/belllab/Desktop/Peter/hicpro_result/hic_results/matrix/Dataset/raw/10000/Dataset_10000.matrix
Inputs '/Users/belllab/Desktop/Peter/hicpro_result/hic_results/matrix' not found. Exit.
mv: cannot stat 'ICED/hic_results/matrix/Dataset/iced/': No such file or directory

From MergeSAM.log:

/Users/belllab/anaconda3/envs/py2/bin/python /usr/local/bin/HiC-Pro_2.11.4/scripts/mergeSAM.py -q 10 -t -v -f bowtie_results/bwt2/DataSet/GSF2784-WT_S1_R1_Sis_LAL14_AluI.bwt2merged.bam -r bowtie_results/bwt2/DataSet/GSF2784-WT_S1_R2_Sis_LAL14_AluI.bwt2merged.bam -o bowtie_results/bwt2/DataSet/GSF2784-WT_S1_Sis_LAL14_AluI.bwt2pairs.bam
[E::idx_find_and_load] Could not retrieve index file for 'bowtie_results/bwt2/DataSet/GSF2784-WT_S1_R1_Sis_LAL14_AluI.bwt2merged.bam'
[E::idx_find_and_load] Could not retrieve index file for 'bowtie_results/bwt2/DataSet/GSF2784-WT_S1_R2_Sis_LAL14_AluI.bwt2merged.bam'
....

What has been tried:

  1. Use current index, .bed and size files for another data -> the same error occurs
  2. Run another data with other index, .bed, size files on the same PC -> works fine
  3. Create new .bed, index, size files -> the same error occurs
  4. Increase amount of RAM -> the same error occurs

Size of genome: 2465177

Thank you for taking the time to look into this case.

nservant commented 3 years ago

Hi, Do you have the bwt2pairs.bam files ? I never saw this index error ... I guess it means that it would expect for the bam.bai files, but usually we do not need them (and they are actually not created). Is it just a warning ? or does the script stop ?

OK1810 commented 3 years ago

The script would continue after the index error but stop when the [hic_qc] Error 1 appeared. We were able to "fix" the problem, although we are still uncertain of exactly what was causing the problem. We re-downloaded the .fasta files for our chromosome from NCBI and used that to build our index, etc. Although there doesn't appear to be any differences between this new .fasta file and the original .fasta file, this seemed to do the trick.

Our best guess right now is that the original .fasta file was corrupted in some way, but unfortunately we don't have a better explanation at the moment.