Cannot load Brain_Non-Myeloid_facs.h5ad

[giovp]

Hi,

really nice resource and cool that you provide pre-processed anndata files. I downloaded the brain datasets: the myeloid one loads without any problem, the non-myeloid one throws an error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-d7f4867a0914> in <module>
----> 1 adata_nonmyeloid = sc.read_h5ad("/Users/giovanni.palla/Projects/spatial-scripts/dat/tabula-muris-senis/Brain_Non-Myeloid_facs.h5ad")

~/miniconda3/envs/scanpy-issues/lib/python3.6/site-packages/anndata/_io/h5ad.py in read_h5ad(filename, backed, as_sparse, as_sparse_fmt, chunk_size)
    427     _clean_uns(d)  # backwards compat
    428 
--> 429     return AnnData(**d)
    430 
    431 

~/miniconda3/envs/scanpy-issues/lib/python3.6/site-packages/anndata/_core/anndata.py in __init__(self, X, obs, var, uns, obsm, varm, layers, raw, dtype, shape, filename, filemode, asview, obsp, varp, oidx, vidx)
    296                 varp=varp,
    297                 filename=filename,
--> 298                 filemode=filemode,
    299             )
    300 

~/miniconda3/envs/scanpy-issues/lib/python3.6/site-packages/anndata/_core/anndata.py in _init_as_actual(self, X, obs, var, uns, obsm, varm, varp, obsp, raw, layers, dtype, shape, filename, filemode)
    495             self._raw = None
    496         elif isinstance(raw, cabc.Mapping):
--> 497             self._raw = Raw(self, **raw)
    498         else:  # is a Raw from another AnnData
    499             self._raw = Raw(self, raw._X, raw.var, raw.varm)

~/miniconda3/envs/scanpy-issues/lib/python3.6/site-packages/anndata/_core/raw.py in __init__(self, adata, X, var, varm)
     30             self._X = X
     31             self._var = _gen_dataframe(var, self.X.shape[1], ["var_names"])
---> 32             self._varm = AxisArrays(self, 1, varm)
     33         elif X is None:  # construct from adata
     34             self._X = adata.X.copy()

~/miniconda3/envs/scanpy-issues/lib/python3.6/site-packages/anndata/_core/aligned_mapping.py in __init__(self, parent, axis, vals)
    229         self._data = dict()
    230         if vals is not None:
--> 231             self.update(vals)
    232 
    233 

~/miniconda3/envs/scanpy-issues/lib/python3.6/_collections_abc.py in update(*args, **kwds)
    844                     self[key] = other[key]
    845             else:
--> 846                 for key, value in other:
    847                     self[key] = value
    848         for key, value in kwds.items():

ValueError: not enough values to unpack (expected 2, got 1)

I'm using scanpy 11.4.5.11 and anndata 0.7.1 With a quick check I could not figure out if it's an issue with anndata, was wondering if you observe the same problem.

Thanks a lot! Giovanni

[aopisco]

@giovp when did you download the h5ad objects?

[pangxueyu233]

@aopisco we downloaded the V2 database Brain_Non-Myeloid_facs.h5ad in figshare. And we tried loaded this obj in scanpy but errors happened

Python 3.6.10 (default, Dec 19 2019, 23:04:32)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> import pandas as pd
>>> import scanpy as sc
>>> Brain_Myeloid_facs=sc.read_h5ad("/mnt/data/scRNA_database_aging/h5ad_files/Brain_Myeloid_facs.h5ad")
/home/XXXXX/.local/lib/python3.6/site-packages/anndata/compat/__init__.py:161: FutureWarning: Moving element from .uns['neighbors']['distances'] to .obsp['distances'].

This is where adjacency matrices should go now.
  FutureWarning,
/home/XXXXX/.local/lib/python3.6/site-packages/anndata/compat/__init__.py:161: FutureWarning: Moving element from .uns['neighbors']['connectivities'] to .obsp['connectivities'].

This is where adjacency matrices should go now.
  FutureWarning,
>>> Brain_Non_Myeloid_facs=sc.read_h5ad("/mnt/data/scRNA_database_aging/h5ad_files/Brain_Non_Myeloid_facs.h5ad")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/XXXXX/.local/lib/python3.6/site-packages/anndata/_io/h5ad.py", line 429, in read_h5ad
    return AnnData(**d)
  File "/home/XXXXX/.local/lib/python3.6/site-packages/anndata/_core/anndata.py", line 321, in __init__
    filemode=filemode,
  File "/home/XXXXX/.local/lib/python3.6/site-packages/anndata/_core/anndata.py", line 530, in _init_as_actual
    self._raw = Raw(self, **raw)
  File "/home/XXXXX/.local/lib/python3.6/site-packages/anndata/_core/raw.py", line 32, in __init__
    self._varm = AxisArrays(self, 1, varm)
  File "/home/XXXXX/.local/lib/python3.6/site-packages/anndata/_core/aligned_mapping.py", line 232, in __init__
    self.update(vals)
  File "/usr/lib/python3.6/_collections_abc.py", line 846, in update
    for key, value in other:
ValueError: not enough values to unpack (expected 2, got 1)

And with the same codes I could load Brain_Myeloid_facs.h5ad easily. But cannot suit for Brain_Non_Myeloid_facs.h5ad

[aopisco]

@pangxueyu233 thanks for letting me know -- just to confirm did you use the latest objects here: https://figshare.com/articles/dataset/Tabula_Muris_Senis_Data_Objects/12654728 ? I have been able to use the objects without problems -- what are your scanpy + anndata versions?

[mousepixels]

@giovp Hi! Did you figure out a solution? I am having the same issue with the facs files but not the droplet files.

Edit: to clarify, only these files are troublesome: Bladder_facs.h5ad Brain_Non-Myeloid_facs.h5ad Heart_facs.h5ad Mammary_Gland_facs.h5ad

And here is error: Traceback (most recent call last): File "", line 1, in File "/Users/mark/miniconda3/lib/python3.7/site-packages/anndata/_io/h5ad.py", line 437, in read_h5ad return AnnData(d) File "/Users/mark/miniconda3/lib/python3.7/site-packages/anndata/_core/anndata.py", line 322, in init filemode=filemode, File "/Users/mark/miniconda3/lib/python3.7/site-packages/anndata/_core/anndata.py", line 531, in _init_as_actual self._raw = Raw(self, raw) File "/Users/mark/miniconda3/lib/python3.7/site-packages/anndata/_core/raw.py", line 32, in init self._varm = AxisArrays(self, 1, varm) File "/Users/mark/miniconda3/lib/python3.7/site-packages/anndata/_core/aligned_mapping.py", line 235, in init self.update(vals) File "/Users/mark/miniconda3/lib/python3.7/_collections_abc.py", line 846, in update for key, value in other: ValueError: not enough values to unpack (expected 2, got 1)

[aopisco]

@mousepixels you can't open the files from figshare?

[pangxueyu233]

Hi, @mousepixels , I could import the Brain_Non-Myeloid_facs.h5ad and others objs, from the link, https://figshare.com/articles/dataset/Tabula_Muris_Senis_Data_Objects/12654728 ,into seurat by ReadH5AD and scanpy by sc.read_h5ad. Thanks for your kindness and maintenance @aopisco

[mousepixels]

@aopisco yes i downloaded them right from figshare. I am only having issues with those 4 files though. Everything else works as expected. I'm going to try on a different computer/environment and will update shortly. I was using sc.read_h5ad()

edit: same error tried on different computer/environment

[jmontelongo25]

I loaded in the h5ad file using Seurat disk in Rstudio, and I received the following errors:

Unknown file type: h5ad'assay' not set, setting to 'RNA'Creating h5Seurat file for version 3.1.5.9900 Adding X as data Adding X as counts Cannot yet add feature-level metadata from compound datasets No cell-level metadata present, creating fake cell names Adding FACS.selection_categories to miscellaneous data Adding age_categories to miscellaneous data Adding cell_ontology_class_categories to miscellaneous data Adding cell_ontology_id_categories to miscellaneous data Adding free_annotation_categories to miscellaneous data Adding method_categories to miscellaneous data Adding mouse.id_categories to miscellaneous data Adding sex_categories to miscellaneous data Adding subtissue_categories to miscellaneous data Adding tissue_categories to miscellaneous data

Is there a way to ameliorate this?

[mousepixels]

So i figured out a temporary fix to reading those files.. i'm not 100% sure it is not messing up something with the anndata object, but the object seems normal from what I can tell:

I put a try/except around the line that was giving me an error in ~/miniconda3/lib/python3.7/site-packages/anndata/_core/aligned_mapping.py (line 235)

    try:
        if vals is not None:
            self.update(vals)
    except:
        print('miniconda3/lib/python3.7/site-packages/anndata/_core/aligned_mapping.py line 236 error skip')

This way it will act normally for all the files that didn't throw errors and will warn you when there is a file with an error. All i can think of is maybe there is a difference in the scanpy versions between people having and people not having issues.