ChimeraPy / Engine

Distributed computing framework for Multimodal data written in Python
https://chimerapy-engine.readthedocs.io
GNU General Public License v3.0
8 stars 0 forks source link

VideoRecord adding initial frozen frames #243

Closed umesh-timalsina closed 12 months ago

umesh-timalsina commented 1 year ago

It seems like the video writer is adding initial frozen frames, hence the synchronized collection of data is erroneous. Here's a sample run log directory. We are trying to record both audio and video in sync. But the audio's length is almost 20 seconds less than the video length. And the first 20 seconds of the video are just frozen frames.

umesh-timalsina commented 1 year ago

The root of this issue seems to be that the video writer is using elapsed key sent from the data_chunk, which is being added in the save_video function. Now that the operations are triggered based on async Events(hence slower). A node's start time might not be representative of the time when it actually started recording, which I think was the goal of the start_time function anyway. There are a couple of ways to remedy this:

  1. Each FSM's state changed should trigger its own timestamp.
  2. Each recorder should bookkeep it's start time, and account for dropped or extra data.

Thoughts, @edavalosanaya?

edavalosanaya commented 1 year ago

Yes, that makes sense. Let's try that idea

edavalosanaya commented 1 year ago

I am leaning towards option 2. It makes more sense to me.