search

aertslab / SCopeLoomR

R package (compatible with SCope) to create generic .loom files and extend them with other data e.g.: SCENIC regulons, Seurat clusters and markers, ...
MIT License
40 stars 15 forks source link

Cannot add Seurat Embeddings #7

Closed saeedfc closed 5 years ago

saeedfc commented 5 years ago

I am not able to add seurat clusterings either. It returns the following error. I guess the loom file created by pyscenic is not in the format used by scopeloomr. Error in h5attr(x = loom, which = GA_METADATA_NAME) : Attribute does not exist

dweemx commented 5 years ago

Hi, Theoretically, .loom files generated by pySCENIC should be compatible with SCopeLoomR. Regarding your first post it seems that there is something wrong when decompressing the MetaData global attribute (which is gzipped and base64 encoded). I tried to add an embedding to a in-house generated by pySCENIC .loom but unfortunately I cannot reproduce your error... If you like you can share your .loom so that I can have a better look on what's going wrong?

I notice also some inconsistency between your 2 posts: the MetaData (GA_METADATA_NAME) global attribute doesn't exist (second post) while in your first post it does exist:

Attributes: last_modified, title, MetaData, Genome, SCopeTreeL1, SCopeTreeL2, SCopeTreeL3,
saeedfc commented 5 years ago

Hi,

I shall close the first issue. 2nd post is the persisting issue. I also tried adding Seurat clusters. I shall share with you the loom file.

add_seurat_clustering(loom,so, default.clustering.resolution = "res.0.52", annotation = ident)
[1] "Seurat resolution 0.52"
[1] "Adding Seurat clusters..."
[1] "Adding default Seurat clusters..."
[1] "Clusterings already exists..."
Error in h5attr(x = loom, which = GA_METADATA_NAME) : 
  Attribute does not exist
In addition: Warning message:
In add_annotated_clustering(loom = loom, group = "Seurat", name = paste("Seurat, resolution",  :
  A default clustering has already been set. The current default clustering will be overwritten.
> add_embedding(loom,embedding = as.data.frame(so@dr$tsne@cell.embeddings), is.default = F, trajectory = NULL)
Error in paste0("Adding embedding ", name, "...") : 
  argument "name" is missing, with no default
> add_embedding(loom,embedding = as.data.frame(so@dr$tsne@cell.embeddings), is.default = F, trajectory = NULL, name = "Seurat")
[1] "Adding embedding Seurat..."
Error in h5attr(x = loom, which = GA_METADATA_NAME) : 
  Attribute does not exist
saeedfc commented 5 years ago

Hi,

As I said, the loom was created on a windows machine and I am using ScopeLoomR on Ubuntu. Now, I created loom file as well on Ubuntu with same script using pySCENIC. Now I am getting different error as below when I work with the loom file with ScopeLoomR. Thank you for the help!

> ident <- seurat.object@meta.data[,c(3,13)]
> loom <- open_loom("loom1.loom")
> loom
Class: H5File
Filename: /DATA/R projects/SCENIC/loom1.loom
Access type: H5F_ACC_RDWR
Attributes: last_modified, title, MetaData, Genome, SCopeTreeL1, SCopeTreeL2, SCopeTreeL3, CreationDate
Listing:
      name    obj_type dataset.dims dataset.type_class
 col_attrs   H5I_GROUP         <NA>               <NA>
 col_edges   H5I_GROUP         <NA>               <NA>
    layers   H5I_GROUP         <NA>               <NA>
    matrix H5I_DATASET 3387 x 13494          H5T_FLOAT
 row_attrs   H5I_GROUP         <NA>               <NA>
 row_edges   H5I_GROUP         <NA>               <NA>
> add_seurat_clustering(loom,so, default.clustering.resolution = "res.0.52", annotation = ident)
[1] "Seurat resolution 0.52"
[1] "Adding Seurat clusters..."
[1] "Adding default Seurat clusters..."
Error in loom$create_dataset(name = paste0("col_attrs/", key), robj = value,  : 
  Can't create dataset col_attrs/ClusterID - already exists!
> add_embedding(loom,embedding = as.data.frame(so@dr$tsne@cell.embeddings), is.default = F, trajectory = NULL, name = "Seurat")
[1] "Adding embedding Seurat..."
Error in memDecompress(from = base64enc::base64decode(what = gzb64c),  : 
  internal error -3 in memDecompress(2)
dweemx commented 5 years ago

Hi @saeedfc,

Regarding the .loom generated on Windows machine, I don't get why the global attributes are not saved in the .loom. Maybe you should report this bug in pySCENIC repo..

Regarding your last post, 1) Error 1

Error in loom$create_dataset(name = paste0("col_attrs/", key), robj = value,  : 
  Can't create dataset col_attrs/ClusterID - already exists!

This bug should be fixed in SCopeLoomR version 0.3.2. It should theoretically generate an error because you're trying to add a default clustering in the .loom while there is already one. You can circumvent this by providing the argument default.clustering.overwrite (see ?add_seurat_clustering).

2) Error 2

Error in memDecompress(from = base64enc::base64decode(what = gzb64c),  : 
  internal error -3 in memDecompress(2)

It turns out that for some reason the global MetaData attribute inside your .loom is not compressed (I don't know how that happened... because normally it should be compressed when using pySCENIC... something to report to pySCENIC maybe also). Nevertheless, SCopeLoomR version 0.3.2 should now handle your .loom (where MetaData saved as a simple json string).

saeedfc commented 5 years ago

Hi,

I assumed its a boolean argument. I tried the following. But I get a different error now.

>add_seurat_clustering(loom,so, default.clustering.resolution = "res.0.52", annotation = ident, default.clustering.overwrite = T)
Error in loom$attr_exists(attr_name = GA_METADATA_NAME) : 
  HDF5-API Errors:
    error #000: ../../../src/H5A.c in H5Aexists(): line 1693: not a location
        class: HDF5
        major: Invalid arguments to routine
        minor: Inappropriate type

    error #001: ../../../src/H5Gloc.c in H5G_loc(): line 173: invalid file ID
        class: HDF5
        major: Invalid arguments to routine
        minor: Bad value

Many thanks!

dweemx commented 5 years ago

Did you remove the loom object in your R session and reopened the loom file with open_loom function before running add_seurat_clustering?

saeedfc commented 5 years ago

I closed and opened a new session. Also used open_loom again. 

> add_seurat_clustering(loom,so, default.clustering.resolution = "res.0.52", annotation = ident, default.clustering.overwrite = T)
[1] "Seurat resolution 0.52"
[1] "Adding Seurat clusters..."
[1] "Adding default Seurat clusters..."
Error in loom$link_delete(name = paste0("col_attrs/", key)) : 
  HDF5-API Errors:
    error #000: ../../../src/H5L.c in H5Ldelete(): line 664: unable to delete link
        class: HDF5
        major: Links
        minor: Can't delete message

    error #001: ../../../src/H5L.c in H5L_delete(): line 2403: can't unlink object
        class: HDF5
        major: Symbol table
        minor: Unable to remove object

    error #002: ../../../src/H5Gtraverse.c in H5G_traverse(): line 869: internal path traversal failed
        class: HDF5
        major: Symbol table
        minor: Object not found

    error #003: ../../../src/H5Gtraverse.c in H5G_traverse_real(): line 641: traversal operator failed
        class: HDF5
        major: Symbol table
        minor: Callback failed

    error #004: ../../../src/H5L.c in H5L_delete_cb(): line 2356: can't delete self
        class: HDF5
        major: Symbol table
        minor: Can't delete message
In addition: Warning message:
In add_annotated_clustering(loom = loom, group = "Seurat", name = paste("Seurat, resolution",  :
  A default clustering has already been set. The current default clustering will be overwritten.

Many Thanks!

dweemx commented 5 years ago

This new bug should be fixed in version 0.3.3. Could you give it a go @saeedfc ? (remove your environment and use open_loom again).

saeedfc commented 5 years ago

Hi, Thanks it worked!