search

chanzuckerberg / single-cell-data-portal

The data portal supporting the submission, exploration, and management of projects and datasets to cellxgene.
MIT License
64 stars 14 forks source link

Represent spatial deep zoom bucket assets URIs in Data Portal DB #7303

Open nayib-jose-gloria opened 3 months ago

nayib-jose-gloria commented 3 months ago

As of schema 5.1.0, a new step in dataset processing is creating "spatial deep zoom" assets for Explorer for Visium datasets with fullres/hires images. These are stored in a spatial-deep-zoom bucket in s3

The URIs for these assets are not tracked in the data portal DB, unlike the URIs for all other artifacts (H5AD, RDS, CXG) we generate during dataset processing. These URIs should be tracked in our DB and associated with their respective dataset versions.

Any code touching these spatial artifacts URIs should be updated accordingly to also update the DB.

For further context: Not tracking spatial deep zoom assets in the DB led to an oversight that led to this bug