When attempting to process MKV files acquired with a Kinect for Azure device, the code fails to properly retrieve the number of frames in the depth stream in the function moseq2_extract.io.video.get_video_info(), which returns with the key nframes set to None. As a consequence, computing a background image for the video fails, as it tries to sample frames in the range of (0, nframes], which in this case is None. Solution is to force ffprobe to count decoded frames rather than rely on the frame count in metadata (which is not set), by passing the keyword argument count_frames=True to the get_video_info() function.
In the course of debugging the above issue, another minor, but related, issue was discovered which affects windows systems. The function moseq2_extract.io.video.get_stream_names() should strip the carriage return character from each item of the stream names returned by the ffprobe command. Otherwise, the keys of the returned dict will contain the carriage return character, and will fail to properly match in later tests of these values. Note that this was discovered with an MKV that contained both RGB (index 0) and Depth (index 1) data, and the underlying problem can be masked in MKV data with only Depth data or Depth data which happens to be the stream in the zero-th index.
What was done?
Two changes have been made:
1) For the first issue described, explicitly detect for the MKV extension and update parameters to include the count_frames=True keyword argument to be passed to get_video_info().
2) For the second issue described, simply strip carriage return characters from the lines returned by the ffprobe command in get_stream_names()
How Has This Been Tested?
locally
Breaking Changes
no breaking changes
Checklist:
[x] I have performed a self-review of my own code
[] I have commented my code, particularly in hard-to-understand areas
[] I have added or updated relevant unit/integration/functional/e2e tests
[] I have made corresponding changes to the documentation
Issue being fixed or feature implemented
When attempting to process MKV files acquired with a Kinect for Azure device, the code fails to properly retrieve the number of frames in the depth stream in the function
moseq2_extract.io.video.get_video_info(), which returns with the keynframesset toNone. As a consequence, computing a background image for the video fails, as it tries to sample frames in the range of(0, nframes], which in this case isNone. Solution is to forceffprobeto count decoded frames rather than rely on the frame count in metadata (which is not set), by passing the keyword argumentcount_frames=Trueto theget_video_info()function.In the course of debugging the above issue, another minor, but related, issue was discovered which affects windows systems. The function
moseq2_extract.io.video.get_stream_names()should strip the carriage return character from each item of the stream names returned by theffprobecommand. Otherwise, the keys of the returneddictwill contain the carriage return character, and will fail to properly match in later tests of these values. Note that this was discovered with an MKV that contained both RGB (index 0) and Depth (index 1) data, and the underlying problem can be masked in MKV data with only Depth data or Depth data which happens to be the stream in the zero-th index.What was done?
Two changes have been made: 1) For the first issue described, explicitly detect for the MKV extension and update parameters to include the
count_frames=Truekeyword argument to be passed toget_video_info(). 2) For the second issue described, simply strip carriage return characters from the lines returned by the ffprobe command inget_stream_names()How Has This Been Tested?
locally
Breaking Changes
no breaking changes
Checklist: