HDF5ExtError: HDF5 error back trace

[clsteam]

Welcome to the HiCExplorer GitHub repository! Before opening the issue please check that the following requirements are met :

• [x] Search whether this issue (or a similar issue) has been solved before using the search tab above. Link the previous issue if appropriate below. non previous issue

• [x] Paste your HiCExplorer version (hicInfo --version) and your python version (python --version) below. hicInfo v3.0 python v3.8.0

• [x] Have you checked our documentation on hicexplorer.readthedocs.io? yes

• [x] Do you use conda to install HiCExplorer? yes, i used conda

• [x] Do you use the latest HiCExplorer release? If not, please install it via a conda environment: conda create --name hicexplorer hicexplorer=3.6 python=3.8 -c bioconda -c conda-forge and activate the environment: conda activate hicexplorer. Retry your command. You can exit a conda environment via conda deactivate. To learn more about conda and environments, please consider the following documentation. not latest HiCExplorer release, because our server shared and used by multiple people

Retry your command, is it solved now? If not please continue with the following:

• [x] Paste the full HiCExplorer command that produces the issue below (ignore if you simply spotted the issue in the code/documentation).

  hicConvertFormat  \
  --matrices s_4000.matrix \
  --outFileName s_4000.h5 \
  --inputFormat hicpro \
  --outputFormat h5  \
  --bedFileHicpro   s_4000_abs.bed

• [x] Paste the output printed on screen from the command that produces the issue below (ignore if you simply spotted the issue in the code/documentation).

  Traceback (most recent call last):
  File "/miniconda3/envs/HiCExplorer/bin/hicConvertFormat", line 7, in <module>
  main()
  File "/miniconda3/envs/HiCExplorer/lib/python3.8/site-packages/hicexplorer/hicConvertFormat.py", line 182, in main
  matrixFileHandlerOutput.save(
  File "/miniconda3/envs/HiCExplorer/lib/python3.8/site-packages/hicmatrix/lib/matrixFileHandler.py", line 57, in save
  self.matrixFile.save(pName, pSymmetric, pApplyCorrection)
  File "/miniconda3/envs/HiCExplorer/lib/python3.8/site-packages/hicmatrix/lib/h5.py", line 121, in save
  with tables.open_file(filename, mode="w", title="HiCExplorer matrix") as h5file:
  File "/miniconda3/envs/HiCExplorer/lib/python3.8/site-packages/tables/file.py", line 315, in open_file
  return File(filename, mode, title, root_uep, filters, **kwargs)
  File "/miniconda3/envs/HiCExplorer/lib/python3.8/site-packages/tables/file.py", line 778, in __init__
  self._g_new(filename, mode, **params)
  File "tables/hdf5extension.pyx", line 492, in tables.hdf5extension.File._g_new
  tables.exceptions.HDF5ExtError: HDF5 error back trace
  
  File "H5F.c", line 444, in H5Fcreate
  unable to create file
  File "H5Fint.c", line 1364, in H5F__create
  unable to open file
  File "H5Fint.c", line 1615, in H5F_open
  unable to lock the file
  File "H5FD.c", line 1640, in H5FD_lock
  driver lock request failed
  File "H5FDsec2.c", line 941, in H5FD_sec2_lock
  unable to lock file, errno = 37, error message = 'No locks available'
  End of HDF5 error back trace

thanks, Yao

[joachimwolff]

• Please update the HiCExplorer version. You use 3.0 which is outdated. The current version is 3.7.2. With conda environments you don't mess with other people, that's one of it's advantages.
• Please try again with the current version.

[clsteam]

Because this is others large server, I don’t have permission to install the new versio. Secondly, I have tried to change the output file name and tried other methods, and tried many times, but finally failed to solve the problem. Therefore, I prefer to have a way to solve the problem in the current version.

[joachimwolff]

The error says the hdf5 library is unable to get a lock for the file. Does your filesystem support file locks? I guess that's something you need to check with the admins of your server; that's not a HiCExplorer issue.

[CocoMlle]

HI, I have the same issue and i didn't have it before :( i can't understand why it happened now ! i'm running hicexplorer on a serveur, with python 3.8. Hicexplorer was installed with conda, it's version 3.7.2. It's working until hicMergeMatrixBins steps. after that i have this error : hicMergeMatrixBins --matrix ../../../ANALYSIS/hicexplorer_results/GM12878/hicMatrix/GM12878_10kb.h5 --outFileName ../../../ANALYSIS/hicexplorer_results/GM12878/hicMatrix/GM12878.100bins.h5 --numBins 100 Traceback (most recent call last): File "/work/gad/shared/bin/miniconda3/envs/hicexplorer38/bin/hicMergeMatrixBins", line 7, in main() File "/work/gad/shared/bin/miniconda3/envs/hicexplorer38/lib/python3.8/site-packages/hicexplorer/hicMergeMatrixBins.py", line 275, in main hic = hm.hiCMatrix(args.matrix) File "/work/gad/shared/bin/miniconda3/envs/hicexplorer38/lib/python3.8/site-packages/hicmatrix/HiCMatrix.py", line 56, in init matrixFileHandler_load = self.matrixFileHandler.load() File "/work/gad/shared/bin/miniconda3/envs/hicexplorer38/lib/python3.8/site-packages/hicmatrix/lib/matrixFileHandler.py", line 51, in load return self.matrixFile.load() File "/work/gad/shared/bin/miniconda3/envs/hicexplorer38/lib/python3.8/site-packages/hicmatrix/lib/h5.py", line 27, in load with tables.open_file(self.matrixFileName, 'r') as f: File "/work/gad/shared/bin/miniconda3/envs/hicexplorer38/lib/python3.8/site-packages/tables/file.py", line 315, in open_file return File(filename, mode, title, root_uep, filters, kwargs) File "/work/gad/shared/bin/miniconda3/envs/hicexplorer38/lib/python3.8/site-packages/tables/file.py", line 778, in init self._g_new(filename, mode, params) File "tables/hdf5extension.pyx", line 492, in tables.hdf5extension.File._g_new tables.exceptions.HDF5ExtError: HDF5 error back trace

File "H5F.c", line 509, in H5Fopen unable to open file File "H5Fint.c", line 1652, in H5F_open unable to read superblock File "H5Fsuper.c", line 420, in H5F__super_read file signature not found

End of HDF5 error back trace

Unable to open/create file '../../../ANALYSIS/hicexplorer_results/GM12878/hicMatrix/GM12878_10kb.h5'

any idea? thanks for your help :)