search

trackmate-sc / TrackMate

TrackMate is your buddy for your everyday tracking.
https://imagej.net/plugins/trackmate
GNU General Public License v3.0
169 stars 76 forks source link

Intersection Over Union (IoU) as feature penalty #238

Open 15bonte opened 2 years ago

15bonte commented 2 years ago

I was wondering if it was possible to add IoU as a feature penalty for the LAP tracker. Indeed, when working with cellpose and cells that do not really change between frames, it makes sense to consider the overlap between cells of consecutive frames.

Would it be something easy to add?

imagesc-bot commented 1 year ago

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/overlap-tracker-in-trackmate-links-2-spots-in-the-same-frame/80680/6

15bonte commented 1 year ago

Hi @tinevez,

As I am very interested in this feature, I may be motivated enough to try to implement it!

Yet, I would like to have your opinion on the best way to do so. Actually IoU would be quite different from other penalties since it is not a normalizeDiffTo between two spot features, but a score computed from the spot geometry.

My first idea would be to add a weight for this IoU penalty, set to 0 by default, and in computeLinkingCostFor add something like

final double iou_penalty = s0.computeIoU( s1 ); penalty += factor_iou * 1.5 * iou_penalty;

This computeIoU could be inspired by functions from the overlap tracker.

What is your opinion on this?

tinevez commented 1 year ago

Excellent!

I think it would be wise not to make it a penalty, but a basic cost. For this I would do another tracker, separate from the ones that are based on distance. I don't think mixing distance and IoU would work well.

It should not be crazy hard to at least test the idea: We can use the Jaqaman trackers (LAP and simple LAP) as basis. They use a CostFunction that can be derived to return the IoU instead of the distance. See: https://github.com/trackmate-sc/TrackMate/blob/master/src/main/java/fiji/plugin/trackmate/tracking/jaqaman/costfunction/CostFunction.java

The quickest way to implement these modification is IMHO to subclass the SparseLAPFrameToFrameTracker tracker. One of the method returns the cost function. You can just substitute the new one in the child class, and it should give new a tracker based on the IoU.

https://github.com/trackmate-sc/TrackMate/blob/master/src/main/java/fiji/plugin/trackmate/tracking/jaqaman/SparseLAPFrameToFrameTracker.java#L270-L284

It's cool no? I mean, the extensibility.

15bonte commented 1 year ago

Sounds clear!

I will try it in the days to come, and let you know if I have some issues.

Thanks a lot!

15bonte commented 1 year ago

Hi @tinevez ,

I have created a pull request here: https://github.com/trackmate-sc/TrackMate/pull/277.

I tried to follow your advice, please tell me if anything should be modified. In particular the name may be improved, and there might be some code duplicated from the overlap tracker since I was not sure of the best place to store it.

Thanks!