vanvalenlab / deepcell-label

Cloud-based data annotation tools for biological images
https://label.deepcell.org
Other
73 stars 14 forks source link

Optimize canvas rendering #412

Closed ykevu closed 1 year ago

ykevu commented 1 year ago

For larger images with many cells, canvas rendering updates can become quite slow, largely due to the way overlaps are handled. We can either migrate to a higher level library/framework such as deck.gl or utilize more potential optimizations in the canvas rendering kernels. In the short term, one method of handling this would be using a modified label format where overlap values are strictly negative, and enforcing that a cell id must map to that value id as well, for instance:

[{cell: 0, value: 0, t: 0, c: 0}, 
{cell: 1, value: 1, t: 0, c: 0}, 
{cell: 0, value: -1, t: 0, c: 0}, 
{cell: 1, value: -1, t: 0, c: 0}]

Then, we can store a list of overlaps such as overlaps=[[0, 1]], where the i th index in overlaps corresponds to value 0 - i - 1, and the labeled and cell types canvases can separately handle positive values (just index into the colormap) and negative values (index into overlaps to obtain the list of overlapping cells present, then index into colormap for each cell).

Currently, we always iterate through the number of cells in the canvas for each pixel, giving us O(n) where n is the number of cells. In the worst case with this new optimization, every pixel contains every cell, in which case we are still O(n). However, we would expect that we are not handling images of this kind, but instead images where <10% of the pixels are overlapping, so our average time complexity is significantly better.

ykevu commented 1 year ago

Though this strategy could work if we are OK with overhauling the data format, a short term solution for cell types in particular is offered in #437.

ykevu commented 1 year ago

An alternative: potential cell matrix format, with two arrays:

cellMapping: [[0], [1], [2, 3]]

numCellsArray: [1, 1, 2]

The i th array in cellMapping represents a list of the cells that map to value i The j th element of numCellsArray represents the number of cells that map to value j

We can then have a kernel something like:

value = labelArray[y][x]
numCells = numCellsArray[value]
for (let i = 0; i < numCells; i++) {
    cell = cellMapping[numCells[i]]
    color = colorMap[cell]
    appendColorToOverlap(color)
}
ykevu commented 1 year ago

Implemented with #502