Example usage of only tracking algorithm

[orena1]

Hello, I have a 4D data (3DxT) and it is already segmented, meaning that for each T, I have 3D masks. Is there any instructions of using the algorithm + model on already segmented data?

I think a simple example like this: https://btrack.readthedocs.io/en/latest/user_guide/simple_example.html

Might be useful. Thanks a lot, Oren

[WenChentao]

Thank you for your question. Currently, this feature is not available, but I appreciate your interest in using it. I am considering two approaches for implementing this feature: using only segmentation results as inputs, or using segmentation results along with a probability map of cell regions. If the segmentation results are highly accurate, using segmentation results alone may be sufficient. However, if there are mistakes in the segmentation, adding a probability map generated by a deep network (or at least the raw cell images) may help improve the accuracy of the final cell positions.

[orena1]

Thanks @WenChentao, I actually did not know that the model also use probability maps. I'll get in the details of my pipeline, and maybe it will help. We use 2p microscope to get long (T) recording of volumes (About 2Hz per plane). I average 40 time-steps from the raw recordings to get a clear cell image in the volumes. (for example if I have raw data of [1000,30,100,100] = [T,Z,X,Y], I mean every 40 frames to get to data of [25,30,100,100])

We then use cellpose to extract the masks of the cells in the volumes, and when I say use cellpose I mean the whole pipeline that includes correcting the labels and fine-tunning the model. Not sure if cellpose segmentation model is the best (especially for 3D), but the gui that they have make it much easier to correct the segmentation and teach ppl how to do the segmentation.

As far as I know, cellpose output but masks and probability maps, therefor I can supply both.

I hope it helps, let me know if I can help in anyway.

[WenChentao]

Thank you for providing more details, @orena1.

Our program (v5.0.2-alpha) currently uses StarDist3D for segmentation and stores the segmented cell positions (2D array of shape (cell_number, 3)) and probability maps of cell regions (3D array of shape (size_height, size_width, size_depth)) in the 'seg' folder with filenames like 'coordst0001.npy' and 'probt0001.npy' (Related codes)

If you can prepare these two types of files by yourself (with CellPose, for example), you can use our track_stardist_single_mode.ipynb or track_stardist_ensemble_mode.ipynb notebooks for tracking without needing to perform segmentation (Section B in these notebooks).

I will consider adding this feature to the program in the future. However, I cannot guarantee a specific timeline for this development.

[orena1]

Thanks @WenChentao do you have an example of those 'coordst0001.npy' and 'probt0001.npy' files so I can try to recreate them?

[WenChentao]

@orena1 Here are the samples of my demo data in Jupyter notebook:

np.load("worm1/seg/coords0001.npy").shape # (number_cells, 3) (163, 3)

np.load("worm1/seg/coords0001.npy")

array([[262, 472, 7], [362, 560, 7], [220, 274, 9], ... [224, 256, 8]], dtype=int64)

np.load("worm1/seg/prob0001.npy").shape # (height, width, depth)

(256, 512, 21)

Please note that the order of the three coordinates in each row should match the order of the probability array. The coordinates in each row represent positions in voxel units, and the probability values should fall between 0 and 1.

Is this what you were looking for?

If you want to know more details, I recommend that you actually install my package and run the tracking notebooks using my demo data, which is publicly available.