qupath / qupath-extension-instanseg

The official QuPath extension for InstanSeg
https://github.com/instanseg/instanseg
Apache License 2.0
10 stars 5 forks source link

Figure out preferred interTileBounds value #66

Open petebankhead opened 2 months ago

petebankhead commented 2 months ago

The interTilePadding() option is an important one for determining the final output and performance.

We need to choose:

  1. a sensible default
  2. whether to expose it through the user interface

Until recently, the value had been 40, and used in combination with a boundary parameter. It was then increased to 80 to reduce overlapping artifacts.

But now the tile merging/overlap-fixing code has been rewritten. The boundary parameter isn't really used; we now discard all objects that touch inter-tile boundaries, but keep others that are nearby to handle later (although maybe this should be changed as well...).

The relevance to performance is that we can basically reduce the 'unique' region covered by each tile by 2 x padding in both width and height.

So for a 512 x 512 tile and a padding of 80, the unpadded region is 352 x 352 pixels... which contains only about 47% of the pixels within the tile.

If we reduce the padding to 40, then we have 432 x 432 pixels, which is over 70% of the original area.

If we keep the padding at 40 and increase the tile size to 1024 x 1024 pixels, then we retain about 85% of the area.

And having more of the area outside the padding suggests we are processing fewer pixels twice and having fewer overlaps to resolve, speeding things up in both cases.

However, setting interTilePadding(0) results in us losing objects completely at tile boundaries. So the 'right' value lies somewhere in between, and probably relates to

  1. the receptive field of InstanSeg (most relevantly, the extent to which its convolutions cross the tile boundary into 'unknown' pixels)
  2. the bounding box of the largest objects we need to detect

Anywhere, here's a script that helps explore different options. Applied to the LuCa image, I have the sense we can decrease the padding much more (but probably not below about 16).

My questions are really then,

  1. which default we should use
  2. whether we should incorporate this value into the UI as an adjustable parameter

If we go with 2, I would propose offering multiples of 16 - since InstanSeg requires the image dimensions in steps of 32. So we might have a combo box with 0, 16, 32, 48, 64, 80, 96, 112, 128 (or some subset of those).

long startTime = System.currentTimeMillis()

def results = qupath.ext.instanseg.core.InstanSeg.builder()
    .modelPath("/path/to/fluorescence_nuclei_and_cells") // Change me!
    .device("mps")         // Change me!
    .allInputChannels()
    .outputChannels()      // Change as needed (0 for nuclei, 1 for cells)
    .tileDims(512)
    .nThreads(4)
    .makeMeasurements(true)
    .randomColors(false)
    .interTilePadding(16)  // Change me!
    .build()
    .detectObjects()

long endTime = System.currentTimeMillis()

println "-------"
println "Results"
println "-------"
println results

// Print some summary measurements from detections
def detections = getDetectionObjects()
def bounds = detections.collect {p -> getMaxBounds(p.getROI())}
def areas = detections.collect {p -> p.getROI().getArea()}

println "Num detections: \t" + detections.size()
println "Average area: \t" + GeneralTools.formatNumber(areas.average(), 2) + " (${areas.min()} - ${areas.max()})"
println "Average bounds: \t" + GeneralTools.formatNumber(bounds.average(), 2) + " (${bounds.min()} - ${bounds.max()})"
println "Processing time: \t" + GeneralTools.formatNumber((endTime - startTime)/1000.0, 2)

double getMaxBounds(qupath.lib.roi.interfaces.ROI roi) {
    return Math.max(roi.getBoundsWidth(), roi.getBoundsHeight())
}