Closed pcm32 closed 2 years ago
Sorry for just getting around to this, @pcm32.
Hmm, I'm not sure I fully understand. You want to remove parts of the AnnData to make it smaller but still compatible with the Cell Browser?
Yes, exactly.
Interesting! I think for the row attributes (which are the genes?), the only attributes which I think are necessary are the gene symbol and maybe the accessions. For the columns, the cell IDs are obviously important, but other than that just the cluster/celltype annotation columns? You've already mentioned the embeddings.
That should be it for "essential" things: gene symbols/accessions, cell IDs, cluster/celltype, embeddings.
Keep a back-up of the original AnnData just in case. Let me know how it goes!
Hi Pablo, can you explain a bit why you think your anndata object would be too large? I can't think of a reason why there would be a problem exporting it, did you run into problems? If you do get errors, we can address them, but we've exported 2M cell atlases without any problem so far.
It may be slow upon the first load. This is due to me preloading all the meta info which I really shouldn't do anymore. So you may want to reduce your meta table to just the essentials or let me know, I should really remove the "preloadMeta()" call on the initial dataset load .
There used to be a problem in the Seurat exporter and matrices that are too big for R, but we've worked around this now using .mtx.gz files (like all other packages). I don't think Python ever had a problem with too large matrices.
It is mostly for the sake of portability and not using so much unnecessary space on Galaxy instances. Original AnnData I was given was 20 GB, I reduced it to 10 GB by removing the raw and to further 4 GB by saving it with compression. I just didn't want to be moving around and replicating a 20 GB file for visualisation purposes only.
Ah, I didn't know you could save without compression, I thought it was the default.
Yes, you can set "raw" to None if you don't want to show it, cbImportScanpy checks for None.
On Fri, Oct 30, 2020 at 2:03 PM Pablo Moreno notifications@github.com wrote:
It is mostly for the sake of portability and not using so much unnecessary space on Galaxy instances. Original AnnData I was given was 20 GB, I reduced it to 10 GB by removing the raw and to further 4 GB by saving it with compression. I just didn't want to be moving around and replicating a 20 GB file for visualisation purposes only.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/193#issuecomment-719539660, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACL4TMB7JQ4FJCUHTYWJHDSNK2TLANCNFSM4SGAEVLA .
I think we can close this for now? Feel free to reopen if there's more to discuss.
Hi there,
I was given a very large AnnData (~20 GBs) which includes of course tSNE and UMAP coords. I was wondering what parts I could remove safely from it to just feed it to CellBrowser through its AnnData converter (cbBuild or so, being passed an AnnData file). The idea is to make that AnnData more portable but still useful for viewing with CellBrowser.
Thanks!