search

quantumjot / btrack

Bayesian multi-object tracking
https://btrack.readthedocs.io
MIT License
309 stars 50 forks source link

Correct way to add parameters in objects #321

Closed Sagnik700 closed 1 year ago

Sagnik700 commented 1 year ago

Hello,

I am trying to add user defined parameters to the objects computed by the function segmentation_to_objects and I am doing it by iterating through the objects list and adding the parameter as obj.properties = {"seg_id": seg_id, "embedding": embeddings[embedding]}. If I print an object after this, I can see my parameter which got added, and it's also visible when I print a tracklet after computing the tracks with tracker.track(tracking_updates=TRACKING_UPDATES). My TRACKING_UPDATES list has the visual value in it. But even if I do not add any custom feature to my objects, the results are exactly the same as of when the custom features are added with the process specified above. So I am wondering if I am missing any step in between or do I have to keep something else is mine within this workflow. My dataset is Hela from CellTrackingChallenge. Posting the script output below:

Without adding custom parameters: [INFO][2023/05/11 12:44:48 PM] Localizing objects from segmentation... [INFO][2023/05/11 12:44:50 PM] Objects are of type: <class 'dict'> [INFO][2023/05/11 12:44:50 PM] ...Found 1107 objects in 84 frames. [INFO][2023/05/11 12:44:50 PM] Loaded btrack: E:\Pythoncodes\venv1\lib\site-packages\btrack\libs\libtracker.DLL [INFO][2023/05/11 12:44:50 PM] btrack (v0.5.0) library imported [INFO][2023/05/11 12:44:50 PM] Starting BayesianTracker session [INFO][2023/05/11 12:44:50 PM] Loading configuration file: ./cell_config.json [INFO][2023/05/11 12:44:50 PM] Objects are of type: <class 'list'> [INFO][2023/05/11 12:44:50 PM] Starting tracking... [INFO][2023/05/11 12:44:50 PM] Update using: ['VISUAL'] [INFO][2023/05/11 12:44:50 PM] Tracking objects in frames 0 to 84 (of 84)... [INFO][2023/05/11 12:44:50 PM] - Timing (Bayesian updates: 0.00ms, Linking: 0.00ms) [INFO][2023/05/11 12:44:50 PM] - Probabilities (Link: 0.00000, Lost: 0.00000) [INFO][2023/05/11 12:44:50 PM] SUCCESS. [INFO][2023/05/11 12:44:50 PM] - Found 1024 tracks in 84 frames (in 0.0s) [INFO][2023/05/11 12:44:50 PM] - Inserted 50 dummy objects to fill tracking gaps [INFO][2023/05/11 12:44:50 PM] Loading hypothesis model: cell_hypothesis [INFO][2023/05/11 12:44:50 PM] Calculating hypotheses (relax: True)... [INFO][2023/05/11 12:44:50 PM] Setting up constraints matrix for global optimisation... [INFO][2023/05/11 12:44:50 PM] Using GLPK options: {'tm_lim': 60000}... [INFO][2023/05/11 12:44:50 PM] Optimizing... [INFO][2023/05/11 12:44:57 PM] Optimization complete. (Solution: optimal) [INFO][2023/05/11 12:44:57 PM] - Fates.FALSE_POSITIVE: 73 (of 1024) [INFO][2023/05/11 12:44:57 PM] - Fates.LINK: 858 (of 1799) [INFO][2023/05/11 12:44:57 PM] - Fates.DIVIDE: 34 (of 852) [INFO][2023/05/11 12:44:57 PM] - Fates.INITIALIZE_BORDER: 4 (of 43) [INFO][2023/05/11 12:44:57 PM] - Fates.INITIALIZE_FRONT: 11 (of 52) [INFO][2023/05/11 12:44:57 PM] - Fates.INITIALIZE_LAZY: 10 (of 929) [INFO][2023/05/11 12:44:57 PM] - Fates.TERMINATE_BORDER: 8 (of 42) [INFO][2023/05/11 12:44:57 PM] - Fates.TERMINATE_BACK: 46 (of 83) [INFO][2023/05/11 12:44:57 PM] - Fates.TERMINATE_LAZY: 5 (of 899) [INFO][2023/05/11 12:44:57 PM] - TOTAL: 5723 hypotheses [INFO][2023/05/11 12:44:57 PM] Completed optimization with 166 tracks [INFO][2023/05/11 12:44:57 PM] Writing out CSV files to: ../results_updated/1/results_v1.csv [INFO][2023/05/11 12:44:58 PM] Ending BayesianTracker session

Adding custom parameters: [INFO][2023/05/11 12:48:07 PM] Localizing objects from segmentation... [INFO][2023/05/11 12:48:10 PM] Objects are of type: <class 'dict'> [INFO][2023/05/11 12:48:10 PM] ...Found 1107 objects in 84 frames. [INFO][2023/05/11 12:48:10 PM] Loaded btrack: E:\Pythoncodes\venv1\lib\site-packages\btrack\libs\libtracker.DLL [INFO][2023/05/11 12:48:10 PM] btrack (v0.5.0) library imported [INFO][2023/05/11 12:48:10 PM] Starting BayesianTracker session [INFO][2023/05/11 12:48:10 PM] Loading configuration file: ./cell_config.json [INFO][2023/05/11 12:48:10 PM] Objects are of type: <class 'list'> [INFO][2023/05/11 12:48:10 PM] Starting tracking... [INFO][2023/05/11 12:48:10 PM] Update using: ['VISUAL'] [INFO][2023/05/11 12:48:10 PM] Tracking objects in frames 0 to 84 (of 84)... [INFO][2023/05/11 12:48:10 PM] - Timing (Bayesian updates: 0.00ms, Linking: 0.00ms) [INFO][2023/05/11 12:48:10 PM] - Probabilities (Link: 0.00000, Lost: 0.00000) [INFO][2023/05/11 12:48:10 PM] SUCCESS. [INFO][2023/05/11 12:48:10 PM] - Found 1024 tracks in 84 frames (in 0.0s) [INFO][2023/05/11 12:48:10 PM] - Inserted 50 dummy objects to fill tracking gaps [INFO][2023/05/11 12:48:10 PM] Loading hypothesis model: cell_hypothesis [INFO][2023/05/11 12:48:10 PM] Calculating hypotheses (relax: True)... [INFO][2023/05/11 12:48:10 PM] Setting up constraints matrix for global optimisation... [INFO][2023/05/11 12:48:10 PM] Using GLPK options: {'tm_lim': 60000}... [INFO][2023/05/11 12:48:10 PM] Optimizing... [INFO][2023/05/11 12:48:17 PM] Optimization complete. (Solution: optimal) [INFO][2023/05/11 12:48:17 PM] - Fates.FALSE_POSITIVE: 73 (of 1024) [INFO][2023/05/11 12:48:17 PM] - Fates.LINK: 858 (of 1799) [INFO][2023/05/11 12:48:17 PM] - Fates.DIVIDE: 34 (of 852) [INFO][2023/05/11 12:48:17 PM] - Fates.INITIALIZE_BORDER: 4 (of 43) [INFO][2023/05/11 12:48:17 PM] - Fates.INITIALIZE_FRONT: 11 (of 52) [INFO][2023/05/11 12:48:17 PM] - Fates.INITIALIZE_LAZY: 10 (of 929) [INFO][2023/05/11 12:48:17 PM] - Fates.TERMINATE_BORDER: 8 (of 42) [INFO][2023/05/11 12:48:17 PM] - Fates.TERMINATE_BACK: 46 (of 83) [INFO][2023/05/11 12:48:17 PM] - Fates.TERMINATE_LAZY: 5 (of 899) [INFO][2023/05/11 12:48:17 PM] - TOTAL: 5723 hypotheses [INFO][2023/05/11 12:48:17 PM] Completed optimization with 166 tracks [INFO][2023/05/11 12:48:17 PM] Writing out CSV files to: ../results_updated/1/results_v1.csv [INFO][2023/05/11 12:48:19 PM] Ending BayesianTracker session

quantumjot commented 1 year ago

Hi @Sagnik700 - cool that you're trying this out. This feature is a bit experimental, to say the least. One immediate thought is that you need to tell the tracker which visual features to use, something like:

with btrack.BayesianTracker() as tracker:
    ...
    tracker.tracking_updates = ["MOTION", "VISUAL"]
    tracker.features = ["embedding",]

LMK if you still don't see any difference.

Also, I see that you're using v0.5.0, so it may be worth upgrading to the latest version, incase there was a bug in that earlier version.

Sagnik700 commented 1 year ago

Hey @quantumjot , thanks for the quick reply. I tried out what you suggested and also updated the version, but the issue still persists. Also, even if I put some fictitious string in the tracker.features like tracker.features = ["random_something",] it does not throw any error. I have put this tracker.features line before tracker.track. Let me know if you can thing of some other step to understand more about this issue. Thanks in advance!!

quantumjot commented 1 year ago

Ah, it's critical that this line (tracker.features = ...) is before tracker.append(objects)

Sagnik700 commented 1 year ago

@quantumjot ok I get you. Now I can see some differences, thanks. But the number of tracks generated are a lot, so I think I have to play around with the HypothesisModel parameters a little to find a good result. Is there anything else you would suggest?

Sagnik700 commented 1 year ago

Closing the issue now since it is answered. Thanks