Closed qiagu closed 3 years ago
ilan-gold
commented
3 years ago I would recommend the bitmask approach if you have the image already in a TIFF. The only thing you need to add is OMEXML metadata to make it an OMETIFF and then pass in the appropriate bitmask flag along with it in the view configuration. Vitessce will handle the rest!
ilan-gold
commented
3 years ago The bitmask has similar features to the polygons (opacity, on/off) but can actually take advantage of multiple different segmentations (i.e channels in the OMETIFF) if you have them unlike the cells.json. QuPath geojson support is also planned if you look in our PR's.
qiagu
commented
3 years ago @ilan-gold Thanks for your suggestion!
I tried using bfconvert to convert my TIFF masks into OMETIFF and provided it to vitessce.io, together with an OMETIFF image and various cell / cell type info extracted from an anndata. As a result, I was able to see the masks layer. However, the masks layer was in grey scale, i.e., cell types were not represented.
I also tried providing X/Y centroids via spatial_centroid_obsm, without a bitmask image. I was able to get a segmentation layer which showed cells in irregular squares and cell types in different colors, respectively.
I wonder whether all I observed are expected or not. Besides, how can I get cell types display in the spatial component with bitmask provided?
ilan-gold
commented
3 years ago Can you provide your view config? I can try to help debug from there. You should be able to provide per-cell information just like before, except instead of polygons you have a bitmask so AnnData should be fine. In fact, centroids are needed to enable selection so it's probably best to include that. Unfortunately I don't have any public examples to share with you (we have ones from from HuBMAP but they are currently not released). I will share the view config though.
ilan-gold
commented
3 years ago Hoepfully this helps!
{
"coordinationSpace": {
"dataset": {
"A": "A"
}
},
"datasets": [
{
"files": [
{
"fileType": "anndata-cells.zarr",
"options": {
"factors": [
"obs/K-Means [Covariance] Expression",
"obs/K-Means [Mean-All-SubRegions] Expression",
"obs/K-Means [Mean] Expression",
"obs/K-Means [Shape-Vectors]",
"obs/K-Means [Texture]",
"obs/K-Means [Total] Expression"
],
"xy": "obsm/xy"
},
"requestInit": {
"headers": {
"Authorization": "Bearer TOKEN"
}
},
"type": "cells",
"url": "https://assets.hubmapconsortium.org/d7cfb9e8beba2b6e65ce587b0b74b5bb/anndata-zarr/reg1_stitched_expressions-anndata.zarr"
},
{
"fileType": "anndata-cell-sets.zarr",
"options": [
{
"groupName": "K-Means [Covariance] Expression",
"setName": "obs/K-Means [Covariance] Expression"
},
{
"groupName": "K-Means [Mean-All-SubRegions] Expression",
"setName": "obs/K-Means [Mean-All-SubRegions] Expression"
},
{
"groupName": "K-Means [Mean] Expression",
"setName": "obs/K-Means [Mean] Expression"
},
{
"groupName": "K-Means [Shape-Vectors]",
"setName": "obs/K-Means [Shape-Vectors]"
},
{
"groupName": "K-Means [Texture]",
"setName": "obs/K-Means [Texture]"
},
{
"groupName": "K-Means [Total] Expression",
"setName": "obs/K-Means [Total] Expression"
}
],
"requestInit": {
"headers": {
"Authorization": "Bearer TOKEN"
}
},
"type": "cell-sets",
"url": "https://assets.hubmapconsortium.org/d7cfb9e8beba2b6e65ce587b0b74b5bb/anndata-zarr/reg1_stitched_expressions-anndata.zarr"
},
{
"fileType": "anndata-expression-matrix.zarr",
"options": {
"matrix": "X"
},
"requestInit": {
"headers": {
"Authorization": "Bearer TOKEN"
}
},
"type": "expression-matrix",
"url": "https://assets.hubmapconsortium.org/d7cfb9e8beba2b6e65ce587b0b74b5bb/anndata-zarr/reg1_stitched_expressions-anndata.zarr"
},
{
"fileType": "raster.json",
"options": {
"images": [
{
"metadata": {
"isBitmask": false,
"omeTiffOffsetsUrl": "https://assets.hubmapconsortium.org/d7cfb9e8beba2b6e65ce587b….."
},
"name": "reg1_stitched_expressions",
"type": "ome-tiff",
"url": "https://assets.hubmapconsortium.org/d7cfb9e8beba2b6e65ce587b…..."
},
{
"metadata": {
"isBitmask": true,
"omeTiffOffsetsUrl": "https://assets.hubmapconsortium.org/d7cfb9e8beba2b6e65ce587b…..."
},
"name": "reg1_stitched_mask",
"type": "ome-tiff",
"url": "https://assets.hubmapconsortium.org/d7cfb9e8beba2b6e65ce587b…..."
}
],
"renderLayers": [
"reg1_stitched_expressions",
"reg1_stitched_mask"
],
"schemaVersion": "0.0.2",
"usePhysicalSizeScaling": false
},
"type": "raster"
}
],
"name": "SPRM",
"uid": "A"
}
],
"description": "",
"initStrategy": "auto",
"layout": [
{
"component": "spatial",
"coordinationScopes": {
"dataset": "A"
},
"h": 8,
"w": 7,
"x": 3,
"y": 0
},
{
"component": "description",
"coordinationScopes": {
"dataset": "A"
},
"h": 4,
"w": 3,
"x": 0,
"y": 8
},
{
"component": "layerController",
"coordinationScopes": {
"dataset": "A"
},
"h": 8,
"w": 3,
"x": 0,
"y": 0
},
{
"component": "cellSets",
"coordinationScopes": {
"dataset": "A"
},
"h": 7,
"w": 2,
"x": 10,
"y": 5
},
{
"component": "genes",
"coordinationScopes": {
"dataset": "A"
},
"h": 5,
"props": {
"variablesLabelOverride": "antigen"
},
"w": 2,
"x": 10,
"y": 0
},
{
"component": "heatmap",
"coordinationScopes": {
"dataset": "A"
},
"h": 4,
"props": {
"transpose": true,
"variablesLabelOverride": "antigen"
},
"w": 7,
"x": 3,
"y": 8
}
],
"name": "reg1_stitched_expressions",
"version": "1.0.1"
}Thanks a lot @ilan-gold
I'll look into your JSON configurations and do more experiments.
FYI, I use a simple python script to generate my config JSON. If possible, please have a look at https://github.com/ohsu-comp-bio/scimap_phenotyping/blob/main/scimap_phenotyping.py
ilan-gold
commented
3 years ago One thing that jumps out at me is that the mask and the image file have the same name - perhaps try changing that?
qiagu
commented
3 years ago Oh, yes. I have two separate scripts in my laptop for with / without bitmsk, respectively. I didn't fully test on this combined script yet. Sorry for that. Basic idea is there though.
ilan-gold
commented
3 years ago @qiagu It's really tough to say without an example dataset/view config for me to work with (even a screenshot might help). If the data is private, perhaps you could still share the view config for me to look at? Or you could email me privately a demo URL/data that I could debug - ilan_gold@hms.harvard.edu is my email. Other than that, this is what our script looks like for public data similar to yours that will (hopefully soon) be available to the public: https://github.com/hubmapconsortium/portal-ui/blob/6c53a966161b5f3441ff14405bdd8f67dde1d61c/context/app/api/vitessce_confs/base_confs.py#L363-L406
@keller-mark Am I missing anything here? Any ideas?
qiagu
commented
3 years ago @ilan-gold Thanks for your quick feedback and useful information. I tried to use your JSON on http://beta.vitessce.io/app. It didn't work, unfortunately. I look forward to seeing what the bitmask should look like. My config json is as follows, which only gives me single-color mask layer, even though cell types are provided. BTW, I updated the python script and now use React component as the backend for my tool. https://github.com/ohsu-comp-bio/scimap_phenotyping/blob/main/vitessce_spatial.py#L43-L47
{
"version": "1.0.0",
"name": "dataset01",
"description": "Description of dataset01",
"datasets": [
{
"uid": "A",
"name": "",
"files": [
{
"type": "raster",
"fileType": "raster.json",
"options": {
"schemaVersion": "0.0.2",
"usePhysicalSizeScaling": false,
"images": [
{
"name": "OMETIFF",
"type": "ome-tiff",
"url": "http://localhost/A/0/image01.ome.tiff",
"metadata": {
"omeTiffOffsetsUrl": "http://localhost/A/0/image01.offsets.json",
"isBitmask": false
}
},
{
"name": "MASKS",
"type": "ome-tiff",
"url": "http://localhost/A/0/masks01.ome.tiff",
"metadata": {
"omeTiffOffsetsUrl": "http://localhost/A/0/masks01.offsets.json",
"isBitmask": true
}
}
],
"renderLayers": [
"OMETIFF",
"MASKS"
]
}
},
{
"type": "cells",
"fileType": "anndata-cells.zarr",
"url": "http://localhost/A/1/anndata.zarr",
"options": {
"xy": "obsm/XY_centroid",
"mappings": {
"UMAP": {
"key": "obsm/X_umap",
"dims": [
0,
1
]
}
}
}
},
{
"type": "cell-sets",
"fileType": "anndata-cell-sets.zarr",
"url": "http://localhost/A/1/anndata.zarr",
"options": [
{
"groupName": "Leiden",
"setName": "obs/leiden"
}
]
},
{
"type": "expression-matrix",
"fileType": "anndata-expression-matrix.zarr",
"url": "http://localhost/A/1/anndata.zarr",
"options": {
"matrix": "X"
}
}
]
}
],
"coordinationSpace": {
"dataset": {
"A": "A"
},
"embeddingType": {
"A": "UMAP"
}
},
"layout": [
{
"component": "spatial",
"coordinationScopes": {
"dataset": "A"
},
"x": 0.0,
"y": 0,
"w": 3.0,
"h": 12
},
{
"component": "cellSets",
"coordinationScopes": {
"dataset": "A"
},
"x": 6.0,
"y": 3.0,
"w": 6.0,
"h": 3.0
},
{
"component": "scatterplot",
"coordinationScopes": {
"dataset": "A",
"embeddingType": "A"
},
"x": 6.0,
"y": 6.0,
"w": 6.0,
"h": 6.0
},
{
"component": "status",
"coordinationScopes": {
"dataset": "A"
},
"x": 6.0,
"y": 0.0,
"w": 6.0,
"h": 3.0
},
{
"component": "layerController",
"coordinationScopes": {
"dataset": "A"
},
"x": 3.0,
"y": 0,
"w": 3.0,
"h": 12
}
],
"initStrategy": "auto"
}Oh, our datasets are usually large, in GBs and they are kept in internal server. I don't know how to share with you for now. Sorry for that.
ilan-gold
commented
3 years ago The JSON I provided includes non public data that is behind a login access. But beyond that beta.vitessce.io might not be updated for the bitmask functionality. Have you tried vitessce.io with your config!
qiagu
commented
3 years ago Now I see the difference b/w beta.vitessce.io and vitessce.io. The result from vitessce.io was reported in the beginning of this post. It's the same as the one that local React component produced.
ilan-gold
commented
3 years ago Ok, I am going to do some digging during the week, but could you check the dtype for your mask? In addition, could you check that the obs ids for the AnnData store match that of your mask?
qiagu
commented
3 years ago check that the obs ids for the AnnData store match that of your mask
Ha, probably that's the key. My AnnData was generated using Scimap which maps the cell IDs to some text for other purpose.
ilan-gold
commented
3 years ago Rad! Let me know how that works out - you can still use factors for displaying the non-numeric ids. I'll be sure to add this as a note to the docs.
qiagu
commented
3 years ago After refactoring the cell IDs, the bitmask works great. @ilan-gold, thank you very much for helping me clear this issue out.
ilan-gold
commented
3 years ago No problem, wish I had thought of it sooner!
ilan-gold
commented
3 years ago Going to close this. I added a note to our beta docs site.
Hi, I am trying to use vitessce to display cell phenotyping together with ome-tiff images. What I got for the cell segmentation masks is a tiff image labeled with different ID number for each cell. I wonder how to prepare this kind of mask into
cells.json. I also notice that the OmeTiffWrapper is able to take bitmask. Does bitmask work the same as the poly outline? If not, which one is preferred?Thanks!