deeptools / HiCExplorer

HiCExplorer is a powerful and easy to use set of tools to process, normalize and visualize Hi-C data.
https://hicexplorer.readthedocs.org
GNU General Public License v3.0
233 stars 70 forks source link

Tool for hicConvertFormat multiple .cool files in one folder #522

Closed sshen82 closed 4 years ago

sshen82 commented 4 years ago

Hi, I am wondering if there is a tool for simultaneously converting cool files, or is there anything wrong with my code. I tried to use shell to write a for loop but doesn't work. (This code works fine on Juicer when using "dump" and use $f.) If I only use one of the file in this folder and do hicConvertFormat, it works.

`FILE='ls ~/Desktop/Keles/Hi-C/Ecker/Human/10kb_cool'

for f in $FILE do hicConvertFormat -m $f --inputFormat cool --outputFormat ginteractions -o $f.tsv mv $f.tsv ~/Desktop/Keles/Hi-C/Ecker/Human/10kb_tsv break done`

Traceback (most recent call last): File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/hicmatrix/lib/cool.py", line 54, in load cooler_file = cooler.Cooler(self.matrixFileName) File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/cooler/api.py", line 80, in init self._refresh() File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/cooler/api.py", line 84, in _refresh with open_hdf5(self.store, *self.open_kws) as h5: File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/contextlib.py", line 81, in enter return next(self.gen) File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/cooler/util.py", line 576, in open_hdf5 fh = h5py.File(fp, mode, args, **kwargs) File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/h5py/_hl/files.py", line 408, in init swmr=swmr) File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/h5py/_hl/files.py", line 173, in make_fid fid = h5f.open(name, flags, fapl=fapl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5f.pyx", line 88, in h5py.h5f.open OSError: Unable to open file (unable to open file: name = '181218_21yr_2_A10_AD001_L23_10kb_contacts.cool', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/shensiqi/miniconda3/envs/hicexplorer/bin/hicConvertFormat", line 7, in main() File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/hicexplorer/hicConvertFormat.py", line 165, in main distance_counts, correction_factors = matrixFileHandlerInput.load() File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/hicmatrix/lib/matrixFileHandler.py", line 45, in load return self.matrixFile.load() File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/hicmatrix/lib/cool.py", line 64, in load log.info("The following nodes are available: {}".format(cooler.fileops.list_coolers(self.matrixFileName.split("::")[0]))) File "/Users/shensiqi/miniconda3/envs/hicexplorer/lib/python3.6/site-packages/cooler/fileops.py", line 145, in list_coolers raise OSError("'{}' is not an HDF5 file.".format(filepath)) OSError: '181218_21yr_2_A10_AD001_L23_10kb_contacts.cool' is not an HDF5 file. mv: rename 181218_21yr_2_A10_AD001_L23_10kb_contacts.cool.tsv to /Users/shensiqi/Desktop/Keles/Hi-C/Ecker/Human/10kb_tsv/181218_21yr_2_A10_AD001_L23_10kb_contacts.cool.tsv: No such file or directory

joachimwolff commented 4 years ago

Hi,

both exceptions have the very clear error message: OSError: Unable to open file (unable to open file: name = '181218_21yr_2_A10_AD001_L23_10kb_contacts.cool', errno = 2, error message = 'No such file or directory',

respectively:

OSError: '181218_21yr_2_A10_AD001_L23_10kb_contacts.cool' is not an HDF5 file.

This means that the paths you give hicConvertFormat are no valid ones, this is caused by your script and not by our software.

// Edit I think your problem are relative vs absolute paths. You make the ls on a specific folder, but give hicConvertFormat only the relative path within this folder. If you have not make sure that your script is actually running in that first folder, the relative paths lead to nowhere. Make either sure that this is matching or use absolute paths for all files.

Best,

Joachim

sshen82 commented 4 years ago

I see what is the problem, and I added a "cd" to that 10kb_cool directory, and the problem is solved. Thank you!