Steps After Frame Synchronization

[verpeutlab]

Hello, I have just synchronized the cameras I have using makeSyncFiles.py. One files was generated for each camera. What do I do with these files after they are generated, and how can I use them to label my data with Label3D? (ex: do I have to run them through any other scripts, and if so, which ones?)

[diegoaldarondo]

You can load all of the sync files in camera order and put their contents into a cell array (let's call that variable sync).

Each element in sync will contain the struct with 4 fields produced by makeSyncFiles.py

You can then pass this variable as a keyword argument to Label3D during initialization (Label3D(...,'sync',sync, 'framesToLabel', framesToLabel)) or just assign the sync field to your Label3D instance (labelGui.sync = sync). When you exportDannce , it will save the sync information in the appropriate format for dannce.

[verpeutlab]

Thank you! I just have questions regarding when I should construct label3d_dannce.mat for my data and what it should include.

First, should I construct label3d_dannce.mat before or after I hand-label a set of 100 frames in Label3D? I believe fields like data_3d inside structures like labelDatawill not have values until after I hand-label the frames in Label3D, but I also need cell arrays like syncand paramsin order for Label3D to run.

Secondly, if I need to construct label3d_dannce.mat before I start to hand-label the frames in Label3D, I am wondering what dimensions I should make the fields data_frameand data_sampleID. What is the relationship between the size of data_frame and data_sampleID and the number of frames in the dataset? I noticed that both fields were in a (1,81) array in label3d_dannce.mat and in a (1,10) array inlabel3d_demo.mat in markerless_mouse_1.

If you could provide me with information on how to construct and when to use the label3d_dannce.mat file, that would be great! Thank you in advance.

[diegoaldarondo]

You should construct the dannce.mat file once you are done labeling frames. You can do this using the exportDannce class method.

The params and sync variables should be generated before labeling. Params is required for labeling, because you need the camera parameters to triangulate points. Sync is not technically required for Label3D, but you will need that information to load the correct frames for labeling and eventually need it for dannce.

The data frame and data sample ID fields are to help support free running camera acquisition, where the frames across different cameras can be acquired at different times. This might seem silly if you are using hardware triggered acquisition, since the frame and sample number are likely the same, but in the free run case this is not always true.

The length of data frame and data sample ID depends on what field you're referencing. In the case of labelData, they are of length nLabledFrames. In the case of sync, they are of length nSamplesInRecording. For labelData{i}, data_frame(j) is the frame in camera i corresponding to the jth sample. data_sampleID(j) is an identifier for the jth sample id. data_sampleID should be the same across all cameras. The exact definition of a sample depends on your setup, but usually sync{i}.data_sampleID equals something like 0:(timeInSeconds*FPS -1), eg the total number of samples.

The dannce.mat file itself is only used by Dannce, not Label3D.