It's sometimes desirable to have the ability to select the max number of instances that a multi-instance model will return. Currently, we implement this through tracking, but sometimes we might want to do this without having to run the tracker.
Use cases:
On-demand (e.g., in GUI or programmatically) instance count filtering
During HITL training when we don't track contiguous frames
(This issue) In exported inference models so the logic doesn't have to be reimplemented downstream
For performance: extraneous detections in the centroid stage can really slow down top-down models, which scale roughly linearly with number of instances
The problem is that there are several strategies for selecting among N instance detections.
717 should solve this in a more general form by providing standalone filtering functions that operate on single LabeledFrames.
This issue proposes a smaller and less general solution to this that will work for some of the use cases.
The idea is to implement this with tensorflow graph compatible ops within the InferenceLayer/InferenceModel subclasses. It's less general, but compatible with exported models.
It's sometimes desirable to have the ability to select the max number of instances that a multi-instance model will return. Currently, we implement this through tracking, but sometimes we might want to do this without having to run the tracker.
Use cases:
The problem is that there are several strategies for selecting among N instance detections.
717 should solve this in a more general form by providing standalone filtering functions that operate on single
LabeledFrame
s.This issue proposes a smaller and less general solution to this that will work for some of the use cases.
The idea is to implement this with tensorflow graph compatible ops within the
InferenceLayer
/InferenceModel
subclasses. It's less general, but compatible with exported models.This could go here: https://github.com/talmolab/sleap/blob/6cac6519208dbc77a89a1e7fb019fed03d9514ac/sleap/nn/inference.py#L2034
Or even better, here during centroid detection/cropping: https://github.com/talmolab/sleap/blob/6cac6519208dbc77a89a1e7fb019fed03d9514ac/sleap/nn/inference.py#L1659-L1660
Where we could use
tf.math.top_k
on the peak values like: