Trajectory overlay on video

[alix-pham]

Hello Trackpy team!

I have been trying to check how accurate is the tracking algorithm from TrackPy, and for this, I would like to have the overlay of the trajectory (e.g. the one given by plot_traj) over the whole timelapse. This way I could see and validate the accuracy of the tracking. I have tried making an animation of the plot_traj output, but it doesn't work; I also tried to download each plot_traj with superimposing each frame and then fetching them to make a video out of them, but it takes a while; I have tried to download the plot_traj output as an image to overlay on each frame but I can't manage to align it properly on my images (even using plt.axis('off'), plt.margins(0) and fig.tight_layout())

Can you think of a not-too-long, easy-to-manipulate way of seeing the trajectory overlay on a video/timelapse/tif file of all the frames, in order to validate the tracking? Did I miss something?

Thank you very much in advance.

[stevenvanuytsel]

Hi,

I’m not part of the trackpy team but I wrote a function that simply plots the tracks on top of the stack using matplotlib, and implemented a slider that allows me to navigate through the frames. I’d be happy to share the code if you'd want it.

Cheers, Steven

On 20 Oct 2021, at 10:09, alix-pham @.***> wrote:

Hello Trackpy team!

I have been trying to check how accurate is the tracking algorithm from TrackPy, and for this, I would like to have the overlay of the trajectory (e.g. the one given by plot_traj) over the whole timelapse. This way I could see and validate the accuracy of the tracking. I have tried making an animation of the plot_traj output, but it doesn't work; I also tried to download each plot_traj with superimposing each frame and then fetching them to make a video out of them, but it takes a while; I have tried to download the plot_traj output as an image to overlay on each frame but I can't manage to align it properly on my images (even using plt.axis('off'), plt.margins(0) and fig.tight_layout())

Can you think of a not-too-long, easy-to-manipulate way of seeing the trajectory overlay on a video/timelapse/tif file of all the frames, in order to validate the tracking? Did I miss something?

Thank you very much in advance.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/soft-matter/trackpy/issues/682, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKMY2EYNEPCKCJX2W6FSCLDUH2BMJANCNFSM5GLDQYEA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[alix-pham]

Hi Steven!

Wow yes, that would be amazing. Though I'm a bit scared that it would take a very long time to update? But I would be very happy to try it. Thanks!

Do you know if it can be exported afterwards?

Best,

Alix

[stevenvanuytsel]

Hi Alix,

I think it depends on the size of the images and the number of tracks that need to be plotted. I’ve not had any issues using 128x128 px frames and up to 15 tracks per frame. Unfortunately it can’t readily be exported as a movie, I didn’t try and implement it as I don’t really need it. I’m sure that there ought to be a way but other people are definitely in a better position to comment/help there if needed.

See my code below (I’m not a computer scientist so my code can definitely be improved upon but this works for me).

def view_tracks(stack, tracking_dataframe, diameter, **kwargs): _plot_style_1 = dict(linewidth=1, alpha=0.7) _plot_style_2 = dict(markersize=diameter+3, markeredgewidth=0.5, markerfacecolor='none', markeredgecolor='r', marker='o', linestyle='none') fontP = FontProperties() fontP.set_size('xx-small')

vmin = kwargs.get('vmin', 0)
vmax = kwargs.get('vmax', stack.max())
cmap = kwargs.get('cmap', 'gray')

fig, ax = plt.subplots()

# Initiate the figure at the middle frame
fig.index = stack.shape[0]//2

features = tracking_dataframe.loc[tracking_dataframe['frame']==fig.index][['x', 'y']]

# Group tracks by particle and plot them
for particle in tracking_dataframe[‘particle’].unique():
    ax.plot(tracking_dataframe.loc[tracking_dataframe[‘particle’]==particle, 'x'], tracking_dataframe.loc[tracking_dataframe[‘particle’]==particle, 'y'], **_plot_style_1, label = particle)

im = ax.imshow(stack[fig.index, :, :], cmap=cmap, vmin=vmin, vmax=vmax)
centroids = ax.plot(features['x'], features['y'], **_plot_style_2)[0]
ax.legend(loc='upper right', bbox_to_anchor=(1.05, 1), prop=fontP)

# Make a slider
sld_axframe = plt.axes([0.20, 0.02, 0.65, 0.03])
slider = Slider(sld_axframe, 'Frame', 0, stack.shape[0]-1, valinit=fig.index, valfmt='%i')

def _onscroll(event):
    fig = event.canvas.figure
    if event.button == 'up':
        fig.index = (fig.index-1) % stack.shape[0]
    else:
        fig.index = (fig.index+1) % stack.shape[0]
    slider.set_val(fig.index)
    _update(fig.index)

def _update(frame):
    fig.index = int(frame)
    features = tracking_dataframe[tracking_dataframe['frame']==fig.index][['x', 'y']]
    im.set_data(stack[int(frame), :, :])
    centroids.set_data(features['x'], features['y'])
    ax.set_ylabel('slice %s' % int(frame))
    im.axes.figure.canvas.draw()

slider.on_changed(_update)
fig.canvas.mpl_connect('scroll_event', _onscroll)
plt.show()

Steven

On 20 Oct 2021, at 16:50, alix-pham @.***> wrote:

bit

[alix-pham]

Thank you very much, I will try your code! However, my data is 1500x1500 px with approx. 400 frames, and 50 to 200 tracks...

Thanks again, Best,

Alix

[alix-pham]

"ValueError: Image size of 350x88421 pixels is too large. It must be less than 2^16 in each direction." Oups...

I have tried to save the plot_traj output to overlay it on all the frames, save them and make a video of them (which prevents from updating and calculating each time you change frame), but I can't seem to have a perfect registration: the overlay image is a bit smaller than the frame and I don't know how to make it right... (t is the trajectory dataframe, im the original image)

from skimage.io import imsave,imread, imshow from skimage.transform import resize,rescale import trackpy as tp import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(10,10)) tp.plot_traj(t, ax=ax) plt.axis('off') plt.margins(0) fig.tight_layout() fig.savefig('overlay.png', transparent=True, bbox_inches='tight')

fig, ax = plt.subplots(figsize=(10,10)) overlay = imread('overlay.png') overlay = resize(overlay, im[0].shape, preserve_range = True) overlay = overlay.astype(np.uint8) ax=plt.imshow(im[0], cmap='gray') ax=plt.imshow(overlay) plt.axis('off')

[alix-pham]

Nevermind, I just needed to add 'pad_inches=0' so now it works. But I would have loved to have a ready-made function to be able to see the overlay on a set of frames instead of only one!

Thanks for the help! Best, Alix

[alix-pham]

path = 'image_folder'
if not os.path.exists(path):
    os.mkdir(path)

fig, ax = plt.subplots(figsize=(10,10))
tp.plot_traj(t, ax=ax)
plt.axis('off')
plt.margins(0)
fig.tight_layout()
fig.savefig(path+'/overlay.png', transparent=True, bbox_inches='tight', pad_inches = 0)

for i in range(t_stack):
    fig, ax = plt.subplots(figsize=(20,20))
    overlay = imread(path+'/overlay.png')
    overlay = resize(overlay, im[0].shape, preserve_range = True)
    overlay = overlay.astype(np.uint16)
    ax=plt.imshow(im[i], cmap='gray')
    ax=plt.imshow(overlay)
    plt.axis('off')
    fig.savefig(path+'/'+filename+'_'+f'{i:06d}'+'.png', bbox_inches='tight', pad_inches = 0)

frames = [] # for storing the generated images
fig, ax = plt.subplots(figsize=(20,20))
for i in range(t_stack):
    ax = plt.imshow(imread(path+'/'+filename+'_'+f'{i:06d}'+'.png'),animated=True)
    plt.axis('off')
    frames.append([ax])

ani = animation.ArtistAnimation(fig, frames, interval=50, blit=True,
                                repeat_delay=1000)
ani.save('movie.mp4')
for file_name in glob.glob(path+"/*.png"):
    os.remove(file_name)