How to specify custom palette for a category?

[olgabot]

Hello! cellxgene is looking great! I just got a brief demo from @colinmegill and one of my questions is, how can one specify a custom color palette for a particular category? For example, for the paper we had this palette in a centralized place that we could grab, but could also have as a column in the scanpy anndata object. If there was e.g. cell_ontology_class and cell_ontology_class_color columns, would that work? Warmest, Olga

[colinmegill]

Hi Olga!

Great question. It's also a great suggestion, since we currently don't support this functionality. I'll keep our thoughts current on this thread as we think through this.

Thanks!

[hejing3283]

Hi, Just want to know how this is going on. For the same sample, we usually have other data types which comes in different color schema. It would be very nice cellxgene would have the option for the user to specify customized color palette. It will help a lot to draw inference based on orthogonal data.

Thanks!

[colinmegill]

Hi @hejing3283! Thanks for this. The team is interested in this feature and is gathering use cases to make sure we meet the most needs.

Could you say more about how you think you might want to specify a custom color palette?

[hejing3283]

hi,

For continue variables, I would love to specify the low, medium and high for the value. For categorical variables, I would love to give it a list of color names or RBG/HEX values in order. It’s even better it allows us to provide several lists of colors so that we can choose one in the browser. Do you think this is possible?

Best, Jing

On Jun 8, 2019, at 10:16, Colin Megill notifications@github.com wrote:

Hi @hejing3283! Thanks for this. The team is interested in this feature and is gathering use cases to make sure we meet the most needs.

Could you say more about how you think you might want to specify a custom color palette?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[colinmegill]

@wikiselev && @cakirb add support to this issue (thanks for the feedback!) and points to https://bioconductor.org/packages/release/bioc/vignettes/iSEE/inst/doc/ecm.html as a potential pattern.

[bjstewart1]

It would be really great to add this functionality to this great tool!

[dburkhardt]

I can think of two ways I'd like to be able to add colormaps:

1. For continuous data like gene expression values, provide a dropdown menu in-app for common perceptually-uniform colormaps like viridis, inferno, magma, plasma, cviridis
2. When I create the AnnData, let me pass a colormap in the adata.uns container (e.g. as C_sample_labels for the obs['sample_labels'] array). I like to use dictionaries for colormaps for categorical variables otherwise I would use a matplotlib-compatible continuous colormap

[brianpenghe]

If it's hard to implement this, can we at least have a red vs blue scheme as default? Because the current default is so hard to display quantitative differences!

[colinmegill]

Thanks @brianpenghe — assuming this would be useful to highlight differential expression results on a continuous scale? Is this the scale you're thinking of:

https://matplotlib.org/3.1.1/tutorials/colors/colormaps.html

cc @sidneymbell

[brianpenghe]

Yes @colinmegill That would be ideal!

[dburkhardt]

I think changing the default color map is good, but RdYlBu might not be the best choice.

There’s actually a fair amount of research on this from the visualization community, and although people prefer rainbow color maps like RdYlBu, they’re not actually that good for quantitative comparisons.

I like this article from IEEE VIS 2019 (https://ieeexplore.ieee.org/document/8494817). They compare the accuracy of domain experts at quantifying differences in global temperatures on a map using various color scales. They find that people perform best with color maps with monotonic luminance scales (such as Blues or Viridis) despite subjects expressing greater preference for rainbow colormaps (like RdYlBu). Interestingly, subjects also reported perceiving themselves as having higher accuracy using rainbow colormaps despite performing worse with rainbow compared to monotonic luminance colormaps (like Blues / Viridis).

I also like this blog by Kenneth Moreland (https://www.kennethmoreland.com/color-advice/), who wrote a short paper called "Why We Use Bad Color Maps and What You Can Do About It"

It would be great to see a change in the colormap in cellxgene, but my reading of the literature suggests not using RdYlBu.

[jenzopr]

Regarding custom colors on launch and in line with @dburkhardt comment: Scanpy allows to color their plots by custom palettes when stored in adata.uns["{var}_colors"] where {var} is the name of the categorial variable. Maybe cellxgene could adopt this behaviour and check for colors in adata.uns before falling back to the default colors.

[brianpenghe]

One more thing to add: UCSC has implemented their browser. They do have a nice default color palette. https://cells.ucsc.edu/

[jenzopr]

Would someone from the dev team give an update on this one?

[colinmegill]

@jenzopr thanks for surfacing this issue again, a lot of users want it but it hasn't made its way into our roadmap yet.

[jenzopr]

Hi Colin! Thanks for the heads up. It would be a miss, if you asked me. We envision using cellxgene complimentary to our published work - and having colors matched between the paper and the cellxgene would be great! Unfortunately, none of our group members is an expert in JS, so I guess contributing code is out of reach for us. Is there another way how we could help you with that?

[colinmegill]

Thanks so much for offering @jenzopr!

Currently, the application is set up to flexibly render colors by sampling rainbow. No matter how many values show up, we'll assign a color. Our codepath for setting a color:

Scale:

https://github.com/chanzuckerberg/cellxgene/blob/3660a6cc27d3f4d08d5df3d714d4003b28dab90c/client/src/util/stateManager/colorHelpers.js#L39

Reducer:

https://github.com/chanzuckerberg/cellxgene/blob/master/client/src/reducers/colors.js

Value:

https://github.com/chanzuckerberg/cellxgene/blob/master/client/src/components/categorical/value.js#L498

The most straightforward way to do this would be to map a categorical label, ie., category is tissue and label is lung, to a color specified as hex #dc9b1c, etc. If present, we would consume this map in value.js for the legend swatch:

and in graph.js for the cells:

Is that an acceptable API to you, to have a map between label and color, and then we can reference the map between cell and label?

[jenzopr]

Thats excellent! Thanks for the detailed explanation and pointing me to the relevant pieces. I'm happy to try! 🎉

[prete]

Hi I've added the --experimental-palette-file option in my dev fork. Not sure if that's the way you planned to go.

Server side: added the --experimental-palette-file click option to launch.py and then load the palette as a dictionary when creating the schema in scanpy_engine.py. Client side: colorHelper.js check if the palette is present and use d3.scaleOrdinal(palette) with the palette instead of d3.scaleSequential(interpolateRainbow)

palette.csv example:

louvain, 0, #7fc97f
louvain, 1, #beaed4
louvain, 2, #fdc086
louvain, 3, #ffff99
louvain, 4, #386cb0
louvain, 5, #f0027f
louvain, 6, #bf5b17
louvain, 7, #666666

Another way can be add the unstructured annotations (uns) to the schema and use the keys that match something like category_palette to store the colours of each category and then load them automatically or use a flag like --use-uns-colors or something like that.

[mweiden]

The core of this issue, custom colors for category labels, is closed. The issue of custom colormaps (including divergent colormaps for differential expression) is still open, however. I will close this issue and open up a new issue for colormaps to separate the concerns.