libavformat branch: player still doing work when paused

[tobiasebsen]

It looks like the new player is still working in the background when it is paused. I discovered this because I migrated a project to the new libavformat-player, where I pre-load 10 video files. Only one file is playing at a time, but every time the previous file is stopped and a new file starts playing, the frame rate drops increasingly. I tried to only call update() on the current player, but not luck.

[bangnoise]

Do the movies have audio?

[tobiasebsen]

No. Only video track.

[bangnoise]

Cheers - can you pull and try now?

[tobiasebsen]

Didn't help. The strange thing is that it's not increasing CPU usage. The frame rate of my entire ofApp just drops increasingly for each time i start the next player. I will try to narrow it down further...

[bangnoise]

I can't reproduce this based on your description. One way to narrow it down would be to start with a new project and make it produce the same effect - you could then attach the project to this issue.

[tobiasebsen]

So here is what I found out....

• The drop in frame rate is due to increased time spent between ofApp::update() and ofApp::draw()
• After some investigation I found out that the time is used by event calls to ofxHapPlayer::update(ofEventArgs & args). As much as over 30ms is used when a video is paused
• Futher investigation revealed the sinner to be the call to _videoPackets.fetch(...) - and more precisely when it returns false (timeout)
• Finally, looking at the ofxHap::LockingPacketCache::fetch(...) shows that it is probably looking for an AVPacket, but gets a timeout (default 30ms)

But, why the PacketCache cannot find the packet, I don't know. Should it even be fetching when it is paused?

[tobiasebsen]

It seems that the PacketCache does not contain any packet for the requested vidPosition. Maybe the Demuxer gets out of alignment with the Clock when pausing the playback? And perhaps my case is particularly prone to this condition because my videos run at 12fps ??

[bangnoise]

Many thanks for the detailed investigation. 0.25 seconds of video should be read after load (which should be fine at 12 fps). I'll see if I can recreate the problem with a 12 fps video, and avoid having a player churn away indefinitely if there isn't a frame for any reason.

[tobiasebsen]

Just to clarify: videos that have been loaded (but is not playing) does not have this problem. The missing frame occurs when the video playback is paused - perhaps at a point in time where no frame is available (in-between frames).

[tobiasebsen]

One more thing: the packet-timeout also occur while playing - at the end of a looping video.

[bangnoise]

Okay - your movies sound a little unusual, as generally one encodes an unbroken stream without gaps between frame times, but I will make changes so players only wait for the timeout when they can reasonably expect a frame may arrive.

[bangnoise]

If you're able to share a small problematic movie, or give instructions to create one, then that would help me test.

[tobiasebsen]

Here is one of the files: https://we.tl/jMYR1tckmN

The problem may be a combination of having multiple players - and running at low frame rate. At 12fps it buffers only 3 frames, which seems to under-run the cache at some frames. But I'm not sure - it's very difficult to debug this asynchronous stuff.

[bangnoise]

As well as the noted problem with the timeout on the main thread, this example eats CPU cycles as the audio thread spins hoping for missing audio samples.

[tobiasebsen]

It looks like the timeout happens on the first few frames right after the video loops. This tells me that there is something wrong with the way the demuxer is told to read future frames at the end of the video. It could be a problem with time ranges or perhaps the order of the demuxers actions (read/seek). Still looking further into this...

[bangnoise]

The original problem is fixed, still to do, related to this issue/test clip:

• [x] don't wait for timeout when no frames are expected (eg if there's a gap in video or the demuxer has an error)
• [ ] the audio track in this sample causes the audio thread to churn at 100% CPU usage

[tobiasebsen]

Wow. Two lines of code was all it took. I've spent hours trying to find the problem. Looking forward to having the last two issues fixed. Thanks!