Label 3D Steps

[verpeutlab]

Hello. I am asking if you could provide me with instructions on how to label frames with Label3D. I see that a list of gestures and hotkeys are provided in the manual, the README file, and in example.m. However, if you could provide a series of steps on how I can use these commands to ensure that the frames are both labeled and saved properly, that would be great! I have been labeling frames with Label3D, but I keep running into a couple issues.

The first time I labeled frames, I labeled the markers in the places where they were visible at each camera angle of the frame using the clickgesture. I frequently saved my progress by using the shift+s command. However, I was unable to export this data to DANNCE format, and I noticed that the field titled data_3D inside the file that was generated only had NaNvalues inside it.

I started to label the frames a second time, and I realized that if I pressed t to triangulate, values would be saved in data_3D. However, the triangulation on my images is not accurate. I have attached a couple images to show you what happens when I press t to triangulate. These images are screenshots of Frame 1 and Frame 5, respectively.

In the first image, I pressed t to triangulate after I added the left ear mark, the right ear mark and the snout mark to each image. I was unsure if this inaccuracy in triangulation was due to a lack of saved data. In the second image, I first labeled four frames, saved them, and then pressed t to triangulate on the fifth frame. I still experienced inaccuracies in my triangulation data. I am not exactly sure what is causing this, as I have updated MATLAB to the latest version, 2021b.

Do you know what is causing the errors in my triangulation? If you could also provide me with a series of steps on how to properly label the frames with the available gestures and hotkeys, that would be great!

Note that each camera angle is duplicated in these images, as I am only using three cameras. Do you also know how I can make this work with three cameras? If not, I can just keep the duplicated images, as I do not mind labeling both.

[diegoaldarondo]

Yeah, happy to help.

I started to label the frames a second time, and I realized that if I pressed t to triangulate, values would be saved in data_3D.

Yes, this is correct. To triangulate a point you'll need to label the keypoint in at least two distinct cameras. The software will only save keypoints that are triangulated.

The brief order of operations is:

for each keypoint:
    Label keypoint in n>=2 perspectives with click/drag
    triangulate with 't'
    save (ctrl+s, or autosave)

I frequently saved my progress by using the shift+s command. However, I was unable to export this data to DANNCE format

If you want to export to dannce format, you can use the function Label3D.exportDannce. I recommend also passing a cell array with your camera names (if you're not using the default "Camera1","Camera2"... convention) to this function to avoid potential issues with dannce down the line, like so labelGui.exportDannce('cameraNames', {'CameraName1','CameraName2','CameraName3'})

However, the triangulation on my images is not accurate. I have attached a couple images to show you what happens when I press t to triangulate. These images are screenshots of Frame 1 and Frame 5, respectively.

This is almost definitely a sign of an inaccurate calibration. In fact, we actually use this kind of testing as a way to easily detect faulty calibrations in our own data and across many other datasets. I would try to recalibrate your cameras, particularly focusing on the extrinsic calibration steps. These are my recommendations for calibration https://github.com/diegoaldarondo/Label3D/issues/14#issuecomment-867783686

Note that each camera angle is duplicated in these images, as I am only using three cameras. Do you also know how I can make this work with three cameras?

Everything should work just fine with 3 cameras (in general it works with n>=2 cameras). When you initialize Label3D just ensure that the params, videos, and optional sync cell arrays are of the same length as the number of cameras you want to include. So videos{1} and params{1} will contain the videos and camera parameters for the first camera, and so on.