Can't read output h5 file

[vtosevski]

Hi there, I am very excited to try cellbender but somehow I haven't gotten very far. I ran it on a little study with 18 samples (10x 3' v3 chemistry) and cellbender run goes smooth and log and pdf look good. I am unable, however, to read in the h5 output file and am getting the following error:

h5ls("cellbender_out_filtered.h5")
Error in H5Fopen(file, "H5F_ACC_RDONLY", native = native) : 
  HDF5. File accessibilty. Unable to open file.

What is the problem there? I am able to read other files fine (for instance, the default raw_feature_bc_matrix.h5 that was an input to cellbender.

I am on Ubuntu 18.04 Linux DESKTOP-01 5.4.0-47-generic #51~18.04.1-Ubuntu

and this is the listing of my CellBender environment:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
alabaster                 0.7.12                   pypi_0    pypi
babel                     2.8.0                    pypi_0    pypi
blas                      1.0                         mkl    anaconda
blosc                     1.20.0               hd408876_0    anaconda
bzip2                     1.0.8                h7b6447c_0    anaconda
ca-certificates           2020.7.22                     0    anaconda
cellbender                0.1                       dev_0    <develop>
certifi                   2020.6.20                py37_0    anaconda
chardet                   3.0.4                    pypi_0    pypi
cudatoolkit               10.2.89              hfd86e86_1  
cycler                    0.10.0                   pypi_0    pypi
docutils                  0.16                     pypi_0    pypi
freetype                  2.10.2               h5ab3b9f_0  
future                    0.18.2                   pypi_0    pypi
hdf5                      1.10.4               hb1b8bf9_0    anaconda
idna                      2.10                     pypi_0    pypi
imagesize                 1.2.0                    pypi_0    pypi
intel-openmp              2020.2                      254    anaconda
jinja2                    2.11.2                   pypi_0    pypi
joblib                    0.16.0                   pypi_0    pypi
jpeg                      9b                   h024ee3a_2  
kiwisolver                1.2.0                    pypi_0    pypi
lcms2                     2.11                 h396b838_0  
ld_impl_linux-64          2.33.1               h53a641e_7  
libedit                   3.1.20191231         h14c3975_1  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.1.0                hdf63c60_0  
libgfortran-ng            7.3.0                hdf63c60_0    anaconda
libpng                    1.6.37               hbc83047_0  
libstdcxx-ng              9.1.0                hdf63c60_0  
libtiff                   4.1.0                h2733197_1  
lz4-c                     1.9.2                he6710b0_1    anaconda
lzo                       2.10                 h7b6447c_2    anaconda
markupsafe                1.1.1                    pypi_0    pypi
matplotlib                3.3.1                    pypi_0    pypi
mkl                       2020.2                      256    anaconda
mkl-service               2.3.0            py37he904b0f_0  
mkl_fft                   1.1.0            py37h23d657b_0  
mkl_random                1.1.1            py37h0573a6f_0    anaconda
mock                      4.0.2                      py_0    anaconda
ncurses                   6.2                  he6710b0_1  
ninja                     1.10.1           py37hfd86e86_0  
numexpr                   2.7.1            py37h423224d_0  
numpy                     1.19.1           py37hbc911f0_0  
numpy-base                1.19.1           py37hfa32c7d_0  
olefile                   0.46                     py37_0  
openssl                   1.1.1g               h7b6447c_0    anaconda
opt-einsum                3.3.0                    pypi_0    pypi
packaging                 20.4                     pypi_0    pypi
pandas                    1.1.2                    pypi_0    pypi
pillow                    7.2.0            py37hb39fc2d_0  
pip                       20.2.2                   py37_0  
pygments                  2.7.0                    pypi_0    pypi
pyparsing                 2.4.7                    pypi_0    pypi
pyro-api                  0.1.2                    pypi_0    pypi
pyro-ppl                  1.4.0                    pypi_0    pypi
pytables                  3.6.1            py37h71ec239_0    anaconda
python                    3.7.9                h7579374_0  
python-dateutil           2.8.1                    pypi_0    pypi
pytorch                   1.6.0           py3.7_cuda10.2.89_cudnn7.6.5_0    pytorch
pytz                      2020.1                   pypi_0    pypi
readline                  8.0                  h7b6447c_0  
requests                  2.24.0                   pypi_0    pypi
scikit-learn              0.23.2                   pypi_0    pypi
scipy                     1.5.2                    pypi_0    pypi
setuptools                49.6.0                   py37_0  
six                       1.15.0                     py_0    anaconda
snappy                    1.1.8                he6710b0_0    anaconda
snowballstemmer           2.0.0                    pypi_0    pypi
sphinx                    3.2.1                    pypi_0    pypi
sphinx-argparse           0.2.5                    pypi_0    pypi
sphinx-autodoc-typehints  1.11.0                   pypi_0    pypi
sphinx-rtd-theme          0.5.0                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.2                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                    pypi_0    pypi
sphinxcontrib-htmlhelp    1.0.3                    pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-programoutput 0.16                     pypi_0    pypi
sphinxcontrib-qthelp      1.0.3                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.4                    pypi_0    pypi
sqlite                    3.33.0               h62c20be_0  
threadpoolctl             2.1.0                    pypi_0    pypi
tk                        8.6.10               hbc83047_0  
torchvision               0.7.0                py37_cu102    pytorch
tqdm                      4.49.0                   pypi_0    pypi
urllib3                   1.25.10                  pypi_0    pypi
wheel                     0.35.1                     py_0  
xz                        5.2.5                h7b6447c_0  
zlib                      1.2.11               h7b6447c_3  
zstd                      1.4.5                h9ceee32_0    anaconda

[sjfleming]

Hi @vtosevski , thanks for trying it out. Glad the tool ran okay for you. For loading and working with the data, there are a couple of options. Two of the methods (scanpy and pytables) are mentioned here

https://cellbender.readthedocs.io/en/latest/getting_started/remove_background/index.html#use-output-count-matrix-in-downstream-analyses

I'd recommend loading the data in scanpy for any downstream work.

import scanpy as sc

# load the data
adata = sc.read_10x_h5('cellbender_out_filtered.h5')

But if you just want to take a look at the h5 file itself, I'd recommend pytables, using something like

import tables

with tables.open_file('cellbender_out_filtered.h5') as f:
    print(f)  # display the structure of the h5 file

However, I do not know why the method you tried didn't work. I am not familiar with h5ls, so I'm not sure I can be super helpful with debugging the command you tried.

[vtosevski]

Thanks for the quick reply Stephen. h5ls is just a generic HDF5 list function from R rhdf5 package. I don't quite understand why I was able to read other h5 files but not cellbender. Anyhow, I went back to the basics after your reply and it turns out I was missing system library libhdf5-dev. After installing it, I am now able to read in cellbender output into Seurat. Thanks for doing a great work supporting the package! Vinko

[sjfleming]

Hi Vinko, thanks so much for posting that here, hopefully it can also help others who run into this issue!