Export Labeled Frames' H5 Analysis

[sherr-an]

Hi there,

I am trying to export the h5 data of the 1000 frames that I labeled, to use in training autoencoder. However, when I "Export Analysis HDF5", it exports 179681 frames (which is approximately the length of one full video).

After I cleaned up all the empty frames with numpy, only 771 frames left. I am quite confused and don't know how to make it right. Please help me if you can!

I have already updated to 1.1.5, for your information. Thank you.

Sherry An

[talmo]

Hi @sherr-an,

This could be occurring for a couple of different reasons. Either the video is not perfectly seekable, or we're actually failing to find any poses at the start/end of the video.

We can troubleshoot. Do you mind sharing just the .slp file with the predictions that you're trying to export from? You can email it to sleap@princeton.edu if it's too big to attach.

Thanks!

Talmo

[talmo]

Closing this issue due to inactivity but please feel free to comment again if you're still having issues and we'll reopen it.

[aereimer]

Hi @talmo, I'm having similar issues here. I does not matter the length (or total number of frames), SLEAP always exports a fixed number of tracks (48387 - see below). I thought it could be an issue with the video encoder, but I recoded the videos as you suggested and the problem persists:

Video: C:/Users/Adriano Reimer/Documents/Freezing_ufscar-adriano-2021-10-30/Freezing_ufscar-adriano-2021-10-30/videos/96_01_H_122021100000_crop1.mp4 Finished inference at: 2022-02-21 16:12:24.456695 Total runtime: 65.87657165527344 secs Predicted frames: 18059/18059 Saved output: C:/Users/Adriano Reimer/Documents/sleap_alpha/freezing_ufscar\predictions\96_01_H_122021100000_crop1.mp4.220221_161114.predictions.slp

Process return code: 0 track_names: 0 node_names: 7 tracks: (48387, 7, 2, 1) track_occupancy: (1, 48387) point_scores: (48387, 7, 1) instance_scores: (48387, 1) tracking_scores: (48387, 1) Saved as C:/Users/Adriano Reimer/Documents/sleap_alpha/freezing_ufscar/96_01_H_122021100000_crop1.h5

---

Video: C:/Users/Adriano Reimer/Documents/Freezing_ufscar-adriano-2021-10-30/Freezingufscar-adriano-2021-10-30\videos\7.mp4 Finished inference at: 2022-02-21 15:19:06.604072 Total runtime: 15.382797002792358 secs Predicted frames: 2610/2610 Saved output: C:/Users/Adriano Reimer/Documents/sleap_alpha/freezingufscar\predictions\7.mp4.220221_151846.predictions.slp

Process return code: 0 track_names: 0 node_names: 7 tracks: (48387, 7, 2, 1) track_occupancy: (1, 48387) point_scores: (48387, 7, 1) instance_scores: (48387, 1) tracking_scores: (48387, 1) Saved as C:/Users/Adriano Reimer/Documents/sleap_alpha/freezing_ufscar/_7.h5

Thanks, Adriano

[talmo]

Strange that it's outputting with the same number of frames -- how are you doing the inference/export?

Do you have multiple windows of SLEAP open?

[aereimer]

Running only one SLEAP instance. Actually the number of tracks/fps being outputted (48387) correspond to the the longest video added to SLEAP, which was not "inferred" at that point. when I read the h5 file generated for these individual files (96_01_H_122021100000_crop1.h5 and _7.h5) with HDFview, what I see is the correct number of frames (18059 and 2610) followed by NAs until the 48387th row.

The outputs I pasted were generated by 1- cleaning all predictions (from all the files); 2- choosing one of the videos and running the inference only on that video; 3 - exporting the analysis as hdf5. I repeated the same process for the 2nd video

oh, btw, the same issue is happening in SLEAP 1.2.0a2 and a5.

Thanks

[e-velin]

Hi,

A similar issue also occurred in my case: I have a sleap file cointaining multiple video files with different number of frames. The exported h5 file for analysis contains only as many frames as the longest video, yet I have much more labelled frames. (Also, 3 tracks are in the h5 files, while I ran training and inference only for single detection). I also tried with ffmpeg -y -i "input.mp4" -c:v libx264 -pix_fmt yuv420p -preset superfast -crf 23 "output.mp4" from here to make sure the videos are seekable and I deleted all the predictions (with that, the shape of the tracks was a bit more bigger).

Thanks.

[roomrys]

Hi @e-velin,

SLEAP will export one analysis file per video. Are you exporting analysis files for all videos or just the current video? If you don't have the option to export analysis files for all videos, then you will need to install the latest version of SLEAP.

Does that help?

Thanks, Liezl

[e-velin]

Hi @roomrys,

Thanks for the help. I intended to export analysis file for all the videos. I installed the 1.2.9 version of SLEAP and now I can indeed export the annotations for all the videos to separate h5 files. What happens now is that most of the h5 files' tracks are empty yet all of the videos have multiple frames annotated. I'm not sure if I should open a new issue for that, I haven't found anything related. Do you have any suggestion?

Thanks, Evelin

[roomrys]

Hi @e-velin,

There is this issue open #988 for exporting single animal projects. The problem could be that you have instances in each frame but you do not have tracks assigned to most of those instances? The analysis files currently assume that your SLEAP project either has no tracks assigned or all instances have tracks assigned. If this isn't the case, then it might be best to open a new issue and we can follow up there.

Thanks, Liezl