Given HuBMAP Dataset JSON, creates a Vitessce configuration.
This is a dependency of portal-ui and search-api.
Updates that are more than housekeeping should result in a new release:
VERSION.txt
.V=$(cat VERSION.txt); git tag $V; git push origin $V
.requirements.in
and rebuild requirements.txt
.requirements.txt
.Installing this package locally makes vis-preview.py
available:
$ cd portal-visualization
$ pip install .
...
$ src/vis-preview.py --help
usage: vis-preview.py [-h] (--url URL | --json JSON) [--assaytypes_url URL]
[--assets_url URL] [--token TOKEN] [--marker MARKER]
[--to_json]
Given HuBMAP Dataset JSON, generate a Vitessce viewconf, and load vitessce.io.
optional arguments:
-h, --help show this help message and exit
--url URL URL which returns Dataset JSON
--json JSON File containing Dataset JSON
--assaytypes_url URL AssayType service; default:
https://ingest.api.hubmapconsortium.org/assaytype/
--assets_url URL Assets endpoint; default:
https://assets.hubmapconsortium.org
--token TOKEN Globus groups token; Only needed if data is not public
--marker MARKER Marker to highlight in visualization; Only used in
some visualizations.
--to_json Output viewconf, rather than open in browser.
Data for the Vitessce visualization almost always comes via raw data that is processed by ingest-pipeline airflow dags.
Harvard often contributes our own custom pipelines to these dags that can be found in portal-containers.
The outputs of these pipelines are then converted into view configurations for Vitessce by the portal backend,
using code in this repo, when a Dataset
that should be visualized is requested in the client.
The view configurations are built using the Vitessce-Python API.
HuBMAP receives various imaging modalities (microscopy and otherwise). The processing is fairly uniform, and always includes running ome-tiff-pyramid + a pipeline for extracting byte offsets to optimize visualization load speeds of large imaging datasets. Vitessce is able to view OME-TIFF files directly via Viv. Two pipelines are commonly used for processing the image data with a more analytic orientation: Cytokit is used to produce segmentations (+ stitching if the input data is tiled) for downstream analysis and SPRM is one such analytic pipeline that does clustering and quantification. Below are common questions and answers for imaging modalities:
Currently, RNA-seq
data comes as AnnData
h5ad
files from Matt's pipeline. Vitessce is able to view AnnData
directly when saved as zarr
. In order to visualize the data, the following steps must be taken to alter the the incoming AnnData
h5ad
file:
obs
part of the store (so they may be visualized as pop-overs when hovered).zarr
store.These steps are exexuted by the anndata-to-ui
container that is run after Matt's pipeline; The view config is generated by RNASeqAnnDataZarrViewConfBuilder
.
Currently the portal backend cannot handle slide-seq
, which is a spatially resolved RNA-seq
assay, but its ViewConfBuilder
class will look be the same as RNASeqAnnDataZarrViewConfBuilder
, except for an additional spatial_polygon_obsm="X_spatial"
argument to the AnnDataWrapper
as well as a SPATIAL
vitessce component in the view config.
Currently only the (mis-named) h5ad-to-arrow pipeline is used to convert h5ad
AnnData
files to json
that contains only the scatterplot results of the scanpy analysis.
In the future, vitessce-python
(or something similar) should be used as a new container to process the SnapATAC
-backed (or other method of storage) peaks for visualization in Vitessce as genomic profiles.
See here for a demo what the final result will look like.
SNARE-seq
is a mix of the above two modalities and its processing and visualization is still TBD.