constantinpape / elf

Utils and convenience functions for large-scale bio-image analysis.
MIT License
48 stars 18 forks source link

Enable linking across missing segments with motile based tracking #85

Open constantinpape opened 10 months ago

constantinpape commented 10 months ago

The default tracking with motile from https://github.com/constantinpape/elf/blob/master/elf/tracking/motile_tracking.py#L5, which just uses the overlap to construct edge costs, works well for many tracking problems. However, it is currently limited by the fact that it cannot cross gaps (i.e. can only link objects if they are present in each consecutive frame or slice). Such gaps often happen due to:

We should extend the tracking functionality so that objects can be tracked with gaps. I can see two possible implementations for this.

1. Including edges that link objects across more than one frame in the tracking problem:

This could be done by extending compute_edges_from_overlap to include a parameter that determines the maximal gap to be closed (in terms of frames), and then computing the edges and overlap for the corresponding adjacent frames instead of just the nearest adjacent frame. This should also include an option for a penalty that grows with the linking distance, so that links with smaller gaps are preferred.

2. Adding new functionality that links tracklets

This would add a second layer problem on top of the first tracking problem, where the nodes correspond to tracklets (the result from the first tracking run), and edges can link the tracklets and can allow linking across gaps. The advantage of this approach is that more expressive features can be computed for the edges between tracklets, e.g. based on how well the local trajectories align.

I think we should go with the first option for now:

We should then also add a function that post-processes the gaps after tracking and inserts the interpolated segmentation masks.

constantinpape commented 10 months ago

The compute_edges functionality was now moved here: https://github.com/constantinpape/elf/blob/master/elf/tracking/tracking_utils.py