search

scverse / anndata

Annotated data.
http://anndata.readthedocs.io
BSD 3-Clause "New" or "Revised" License
575 stars 152 forks source link

Output layer content but got killed #782

Open tjwu opened 2 years ago

tjwu commented 2 years ago

I am trying to output a table from h5ad file. These are codes.

import numpy as np import pandas as pd import anndata as ad from scipy.sparse import csr_matrix adata = ad.read('local.h5ad', backed='r') adata.to_df(layer='decontXcounts').to_csv('test.csv')

adata.X <HDF5 sparse dataset: format 'csr', shape (483152, 58559), type '<f4'>

Try to output the adata.X for parsing. adata.to_df(layer='decontXcounts').to_csv('test.csv')

But, the run got killed due to memory usage( I guess, since the file reaches 16G and only 71638 rows output). Any suggestions on output the adata.X? Thanks.

ivirshup commented 2 years ago

You could read through the sparse matrix and write it to the csv line by line, but if you can't hold the dense dataframe in memory it may be difficult to work with the csv.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. Please add a comment if you want to keep the issue open. Thank you for your contributions!