[BUG] Shape of tracks layer changes after export and re-loading

[p-j-smith]

Describe the bug I started working on exporting the tracks in the plugin, but came across an issue whereby the shape of the tracks layer changes after exporting and re-loading.

Before saving the data, the tracks data is 4D. However, after exporting and loading it becomes 5D. This means the exported data can't be visualised on top of the original segmentation, because they have different axes.

Your setup: btrack_version: 0.6.4.dev29 system_platform: macOS-13.4.1-x86_64-i386-64bit system_python: 3.10.12 Comments To reproduce:

import skimage.io

import btrack
from btrack import datasets

CONFIG_FILE = datasets.cell_config()
SEGMENTATION_FILE = datasets.example_segmentation_file()
OBJECTS_FILE = datasets.example_track_objects_file()
FEATURES = [
    "area", 
    "major_axis_length", 
    "minor_axis_length", 
    "orientation", 
    "solidity",
]

segmentation = skimage.io.imread(SEGMENTATION_FILE)
objects = btrack.utils.segmentation_to_objects(
    segmentation, 
    properties=tuple(FEATURES), 
)

# initialise a tracker session using a context manager
with btrack.BayesianTracker() as tracker:

    # configure the tracker using a config file
    tracker.configure(CONFIG_FILE)
    tracker.max_search_radius = 50
    tracker.tracking_updates = ["MOTION", "VISUAL"]
    tracker.features = FEATURES

    # append the objects to be tracked
    tracker.append(objects)

    # set the tracking volume
    tracker.volume=((0, 1600), (0, 1200))

    # track them (in interactive mode)
    tracker.track(step_size=100)

    # generate hypotheses and run the global optimizer
    tracker.optimize()

    # get the tracks in a format for napari visualization
    original_tracks, original_properties, original_graph = tracker.to_napari()

    # export the track data 
    tracker.export("sample_tracks.h5", obj_type="obj_type_1")

with btrack.io.HDF5FileHandler(
    "sample_tracks.h5", "r", obj_type='obj_type_1',
) as reader:

    exported_tracks, exported_properties, exported_graph = btrack.utils.tracks_to_napari(
        reader.tracks,
    )

print(f"{original_tracks.shape=}")
print(f"{exported_tracks.shape=}")

which prints:

original_tracks.shape=(52996, 4)
exported_tracks.shape=(52996, 5)

[quantumjot]

Most likely it's related to this: https://github.com/quantumjot/btrack/blob/9f22f4d056aa55e8022f490c462f89240e21f89a/btrack/utils.py#L160-L163

[p-j-smith]

It is indeed related to that. The issue is that in the above example, the volume is set be two dimensional:

tracker.volume=((0, 1600), (0, 1200))

but not all z values are 0. When tracker.to_napari() is called, the dimensionality is determined from the volume we have set:

https://github.com/quantumjot/btrack/blob/ee54546fd0e0f15c98689331e0a452bca14ec12b/btrack/core.py#L688

However, when the tracks are exported to HDF the information about the volume is lost. So after loading tracks from the HDF file, when we convert the tracks to a napari layer the z values are used to determine the dimensionality:

https://github.com/quantumjot/btrack/blob/ee54546fd0e0f15c98689331e0a452bca14ec12b/btrack/utils.py#L193-L195

As not all z values are 0, it is assumed the tracks are 3D.

I'm not sure what the best solution is, but potential ideas are:

• raise an error if a 2D volume is set but not all z values are 0
• set all z values to 0 if a 2D volume is set
• store volume information when exporting to HDF

Or perhaps there's a better solution??

[quantumjot]

Yes - this is a problem that I've been meaning to deal with for some time! It's weird that there are some non-zero values, but I expect that may be due to dummies? Perhaps we could filter those out and everything would just work

I expect that option 2 is the easiest to implement, but I also like 1 & 3

[quantumjot]

Yep, it's due to the dummy objects that are inserted into the tracks:

[quantumjot]

So... you could replace this:

https://github.com/quantumjot/btrack/blob/ee54546fd0e0f15c98689331e0a452bca14ec12b/btrack/utils.py#L193-L195

with

z = np.concatenate([np.asarray(track.z)[~np.asarray(track.dummy)] for track in tracks])

and it'll likely fix things for now.