Output videos for tracking

[cmalinmayor]

I'm super excited about using this tool to generate ground truth for a segmentation and tracking pipeline I am developing - I played around with it a bit and the images are impressive. However, I need videos with tracking ground truth, rather than single images. I'm planning to fork and adapt the code to do that this week - do you have any tips before I dig into it? And would you be interested in merging this feature back into the main code base?

It seems that rendering multiple frames from the simulation would be straightforward, but I also need the lineage information, and in my initial experimentation I didn't see that anywhere obvious. I'll look more, but I thought I'd ask too. Thanks!

[georgeoshardo]

Hi Caroline, thank you for your interest in SyMBac!

We actually already have the ability to export video and lineage tracking data for each cell (to train lineage tracking models), but currently it's not explicitly explained how to do so. All the data currently exists in the form of mother daughter relations and xy positions in the cell_timeseries pickle which can be exported, but it's not documented. I can share some additional information with you after I talk to the person in our lab who is mostly using this functionality. Are you interested in movies of mother machine or agar pad?

[georgeoshardo]

Additionally I always appreciate pull requests, we are currently working on the "version 2" which will incorporate the ability to simulate cells in any geometry, and simulate cells as soft-bodied (allowing for them to bend and squash against each other), and for which we want lineage tracking information to be more explicitly accessible. So any substantial code modifications to push is very useful!

[cmalinmayor]

Thank you for the fast response, and I'm very glad to hear that it is already at least theoretially supported! I'm interested in mother machine data. I did find the daughter and parent sections of the Cell class and where they are set in the simulation code, but would definitely appreciate some guidance. It seems like there is only one daughter (I suppose the same cell continues as the other daughter?), and it looked like the parents are set to copies of the same cells 😆 so I'm clearly misunderstanding something.

[georgeoshardo]

Yes indeed, the relations are set hierarchically within the cell object, so for example to access a cell's great-great-grandmother would be cell.mother.mother.mother.mother, not super convenient. I will push a commit soon which will add an option to export a lineage tree, although can you suggest what the canonical data-structure for this sort of thing should look like?

[cmalinmayor]

can you suggest what the canonical data-structure for this sort of thing should look like?

"Canonical" is not really decided in the field, unfortunately. I prefer a graph (usually networkx) where the node ids correspond to segmentation label ids, and the edges represent parent/daughter relationships and identity across time. If the segmentation ids are consistent for the cells of the same identity across time (for example, if they matched the Cell IDs), then just mother/daughter links (in a graph, or in a simple text file) is sufficient.

[georgeoshardo]

Perfect, thank you! I shall implement this :)

[georgeoshardo]

Hi Caroline

I've implemented the ability to output lineages over time, and simple family trees (which exclude temporal info and just give the divisions, to see an overview of the output) in https://github.com/georgeoshardo/SyMBac/commit/a976458a4e3db5c293a2a6815803b705e7b42a9d

There is an example notebook in https://github.com/georgeoshardo/SyMBac/blob/main/SyMBac/MM_lineage_example.ipynb (additionally the final output renders frames from a simulation temporally)

Also now by default, all masks are rendered in a temporally coherent manner, with the mother cell getting mask value 1, and maintaining it, while spawning new cells with ever increasing mask values.

After you run a simulation you can do (as shown in the notebook):

from SyMBac.lineage import Lineage
my_lineage = Lineage(my_simulation) #Generates the lineage trees
my_lineage.temporal_lineage_graph #Returns the full relations across time
my_lineage.family_tree_graph #Returns a family tree with no temporal info
my_lineage.plot_family_tree()  #Plots a family tree
my_lineage.all_cell_data_df #A pandas dataframe with all attributes from all cells in the simulation

You can also do

my_simulation.draw_simulation_OPL(do_transformation=True, label_masks=True)
my_simulation.visualise_in_napari()

and browse the masks and pre-render output to check that the labels are indeed temporally coherent.

I hope I have not introduced any new bugs, but please let me know if you run into any.

[cmalinmayor]

Amazing, thanks Georgeos! I will check it out Monday and let you know if I have any questions. I really appreciate the quick help!

[cmalinmayor]

I ran the example notebook you provided, and other than #46, which was not related to the lineage functionality, it worked great! If I come across anything else, I'll open a new issue. Thank you so much @georgeoshardo 🎉

[georgeoshardo]

Awesome! Glad it worked!