Cannot generate multi-generational tracks.

[OCKI96]

Hi @quantumjot,

I am trying to generate some tracks over a 4 day imaging experiment. While the individual tracks appear to be accurate, I am unable to obtain very long or multi-generational tracks. I have played around with HypothesisModel parameters to try and improve this but always seem to get only one generation - do you have any tips and tricks for improving this?

Let me know if you need any further info and thanks in advance for your help!

Ollie

[quantumjot]

Hi Ollie - would you be able to post the tracking log? It really helps to debug

[OCKI96]

`[INFO][2022/12/09 02:17:45 PM] Loaded btrack: /Users/ingeo/opt/anaconda3/envs/Image_analysis_OI/lib/python3.9/site-packages/btrack/libs/libtracker.dylib
[INFO][2022/12/09 02:17:45 PM] btrack (v0.4.6) library imported
[INFO][2022/12/09 02:17:45 PM] Starting BayesianTracker session
[INFO][2022/12/09 02:17:45 PM] Loading configuration file: /Volumes/lab-santoss/home/users/ingeo/Analysis/Image_analysis/OI-Live-Cell-Imaging/cell_config.json
[INFO][2022/12/09 02:17:45 PM] Objects are of type: <class 'list'>
[WARNING][2022/12/09 02:17:47 PM] `track_interactive` will be deprecated. Use `track` instead.
[INFO][2022/12/09 02:17:47 PM] Starting tracking... 
[INFO][2022/12/09 02:17:47 PM] Tracking objects in frames 0 to 99 (of 384)...
[INFO][2022/12/09 02:17:48 PM]  - Timing (Bayesian updates: 5.01ms, Linking: 4.32ms)
[INFO][2022/12/09 02:17:48 PM]  - Probabilities (Link: 0.99773, Lost: 0.70083)
[INFO][2022/12/09 02:17:48 PM]  - Stats (Active: 1152, Lost: 30705, Conflicts resolved: 12350)
[INFO][2022/12/09 02:17:48 PM] Tracking objects in frames 100 to 199 (of 384)...
[INFO][2022/12/09 02:17:50 PM]  - Timing (Bayesian updates: 18.55ms, Linking: 13.27ms)
[INFO][2022/12/09 02:17:50 PM]  - Probabilities (Link: 0.99998, Lost: 0.68732)
[INFO][2022/12/09 02:17:50 PM]  - Stats (Active: 2489, Lost: 70083, Conflicts resolved: 32736)
[INFO][2022/12/09 02:17:50 PM] Tracking objects in frames 200 to 299 (of 384)...
[INFO][2022/12/09 02:18:03 PM]  - Timing (Bayesian updates: 135.14ms, Linking: 48.33ms)
[INFO][2022/12/09 02:18:03 PM]  - Probabilities (Link: 0.99995, Lost: 0.99995)
[INFO][2022/12/09 02:18:03 PM]  - Stats (Active: 6761, Lost: 143486, Conflicts resolved: 83225)
[INFO][2022/12/09 02:18:03 PM] Tracking objects in frames 300 to 384 (of 384)...
[INFO][2022/12/09 02:18:34 PM]  - Timing (Bayesian updates: 294.75ms, Linking: 86.40ms)
[INFO][2022/12/09 02:18:34 PM]  - Probabilities (Link: 1.00000, Lost: 0.99208)
[INFO][2022/12/09 02:18:34 PM] SUCCESS.
[INFO][2022/12/09 02:18:34 PM]  - Found 79364 tracks in 384 frames (in 0.0s)
[INFO][2022/12/09 02:18:36 PM]  - Inserted 49797 dummy objects to fill tracking gaps
[INFO][2022/12/09 02:18:36 PM] Loading hypothesis model: cell_hypothesis
[INFO][2022/12/09 02:18:36 PM] Calculating hypotheses (relax: True)...
[INFO][2022/12/09 02:18:37 PM] Setting up constraints matrix for global optimisation...
[INFO][2022/12/09 02:18:37 PM] Using GLPK options: {'tm_lim': 60000}...
[INFO][2022/12/09 02:18:55 PM] Optimizing...
[WARNING][2022/12/09 02:20:01 PM] Optimizer returned status: feasible
[WARNING][2022/12/09 02:20:01 PM] Optimization failed.
GLPK Integer Optimizer 5.0
317456 rows, 256581 columns, 355726 non-zeros
256581 integer variables, all of which are binary
Preprocessing...
158728 rows, 256581 columns, 355726 non-zeros
256581 integer variables, all of which are binary
Scaling...
 A: min|aij| =  1.000e+00  max|aij| =  1.000e+00  ratio =  1.000e+00
Problem data seem to be well scaled
Constructing initial basis...
Size of triangular part is 158728
Solving LP relaxation...
GLPK Simplex Optimizer 5.0
158728 rows, 256581 columns, 355726 non-zeros
*     0: obj =   1.518559702e+05 inf =   0.000e+00 (2402)
*  2107: obj =   1.485396393e+05 inf =   0.000e+00 (229)
*  2337: obj =   1.485202866e+05 inf =   0.000e+00 (0)
OPTIMAL LP SOLUTION FOUND
Integer optimization begins...
Long-step dual simplex will be used
+  2337: mip =     not found yet >=              -inf        (1; 0)
+  2356: mip =     not found yet >=   1.485207690e+05        (19; 0)
+  2357: >>>>>   1.485302866e+05 >=   1.485207690e+05 < 0.1% (19; 0)
+  2375: mip =   1.485302866e+05 >=   1.485215366e+05 < 0.1% (33; 3)
+  2377: >>>>>   1.485298220e+05 >=   1.485215366e+05 < 0.1% (34; 3)
+  2395: mip =   1.485298220e+05 >=   1.485223220e+05 < 0.1% (47; 7)
+  2419: mip =   1.485298220e+05 >=   1.485223220e+05 < 0.1% (63; 9)
+  2442: mip =   1.485298220e+05 >=   1.485223220e+05 < 0.1% (79; 11)
+  2464: mip =   1.485298220e+05 >=   1.485227866e+05 < 0.1% (95; 13)
+  2486: mip =   1.485298220e+05 >=   1.485227866e+05 < 0.1% (111; 14)
+  2509: mip =   1.485298220e+05 >=   1.485227866e+05 < 0.1% (127; 16)
+  2532: mip =   1.485298220e+05 >=   1.485227866e+05 < 0.1% (143; 18)
+  2556: mip =   1.485298220e+05 >=   1.485227866e+05 < 0.1% (160; 20)
+  2577: mip =   1.485298220e+05 >=   1.485235720e+05 < 0.1% (177; 21)
Time used: 60.1 secs.  Memory used: 368.5 Mb.
+  2591: mip =   1.485298220e+05 >=   1.485235720e+05 < 0.1% (186; 23)
TIME LIMIT EXCEEDED; SEARCH TERMINATED
[INFO][2022/12/09 02:20:10 PM] Ending BayesianTracker session`

[quantumjot]

Ah - ok, I see. The optimiser timed out (DNF) so you didn't get any lineage information. This is usually because your parameters lead to a very large number of hypotheses to evaluate. You can try letting the optimizer run for a bit longer:

# run the optimizer for 10 minutes before timing out
tracker.optimize(options={"tm_lim": 60_000 * 10})

or try tweaking some of the parameters to improve the tracking. Generating 79364 tracks from only 384 frames seems like a lot. Selecting parameters is problem specific, so hard to give universal advice!

[OCKI96]

This would be parameters in the motion model initially? Thanks for getting back to me so quickly!

[quantumjot]

Yes, the parameters of the "motion model" are a good place to start. Also, is this 2D or 3D data? Sometimes setting the tracking volume (tracker.volume) incorrectly can lead to a large number of lost tracks...