search

vjcitn / XenSCE

sketch of S4 methods for Xenium demonstration data in Open Storage Network
https://vjcitn.github.io/XenSCE/
0 stars 1 forks source link

ParquetDataFrame doesn't scale well, consider direct path and serialization solution #4

Open vjcitn opened 4 weeks ago

vjcitn commented 4 weeks ago

the DelayedArray approach seems much slower than duckdb-dplyr mediated solutions that work directly with parquet. a class union may be needed to facilitate direct work with path to parquet, and a general serialization solution like saveHDF5SummarizedExperiment will probably be needed

vjcitn commented 3 weeks ago

ingest_xen introduces Xen_SPEP which has worked for skin and prostate examples

 2141  wget https://mghp.osn.xsede.org/bir190004-bucket01/BiocXenData/xen_sk_spep.zip
 2142  wget https://mghp.osn.xsede.org/bir190004-bucket01/BiocXenData/xen_prost.zip

will need caching methods

vjcitn commented 3 weeks ago

the show method for spatialCoords would crash laptop when applied to a XenSPEP ... need to investigate

vjcitn commented 3 weeks ago

loadGeometry needs to be able to deal with paths other than .

vjcitn commented 3 weeks ago

should update build_demo