textured mode - tearing issue

[adrchlik]

Hi jvcleave. Thanks again for a great work. ofxOmxPlayer is getting better and better. I have two issues -

1. I have problem a don't know if there is any solution for it. I use player in texture mode and I see some kind of tearing problem. When I use special video files (in means of content) for example white filled circle moving from left to right on black border .There is visible a huge tearing problem. The OpenGL app is Vsync. I think it comes from the situation that video decoder-render thread is rendering to the texture directly but independently on Vsync signal and eglSwapBuffers command. Do you have any idea how this can be solved ?
2. When movie is looping (texture or non-texture mode) after the first pass it starts playing very quickly. I know that the version from Sep 08, 2013 didn't do that.

[jvcleave]

regarding tearing - there is a now a working isFrameNew() method - depending on what you are doing that may help

the examples and my videos don't have the "playing very quickly" issue. I have seen it happen before though - mostly when I was playing with multiple players. If you could post an app or video that causes this I could take a look.

[adrchlik]

I'm not able to find where is the isFrameNew() method defined. If I would be able to render the final OpenGL scene right after isFrameNew is active and before the video decoder thread starts refreshing the data in texture with next frame it could help. I must somehow exclude concurrence of writting to texture and rendering the scene with that texture. Now I'm testing on the master branch.

This is the link to the problematic video: https://dl.dropboxusercontent.com/u/26953539/vsync_loop_problem.mov

[jvcleave]

sorry - getting my addons mixed up (although I can/need to put isFrameNew in here)

I'll try your video

[jvcleave]

ok - I am seeing the same and also confirmed an earlier version didn't have this issue

A lot has changed so it may take a while to track it down. In the meantime if you want the previous setup this version was playing your file ok

https://github.com/openFrameworks-arm/CMU-RPi-Workshop/tree/master/omxPlayerExample

[jvcleave]

@adrchlik this doesn't solve the tearing issue but adding an audio track seems to fix the "fast playing" after looping

Here is a re-rendered version with an audio track added if you want to try it (also a different size but not sure that matters) https://www.dropbox.com/s/gua2ydasr2slrom/vsync_loop_problem_re-render.mov

@Maigre not sure if this workaround helps any of your stuff at all

[jvcleave]

@adrchlik I just implemented isFrameNew and using that combined with an FBO seems to stop the tearing

you can checkout an example with your file here https://www.dropbox.com/s/dqfr0h9hbofmakj/isFrameNewTest.zip

Looking into maybe using an FBO internally

[adrchlik]

Yes. It really seems that using isFrameNew and FBO together stops the tearing effect. Thanks. I had to delve deeper into the openframe code to understand why it stops the tearing effect. There is one more texture (FBO texture) which is used for normal main OpenGL loop render. When the omxplayer has new frame ready it does one more render to this FBO texture. And it believes(there is no guarantee) that this render(to the FBO) will be done before the omxplayer texture is modified with the next video frame. Pros: No tearing, Cons: more workload. Am I understand it correctly ? Just came my mind different solution without the more workload. But I'm not sure if it is realizable. Use two EGLImage textures and after isFrameNew appears just only switch between them. The main OpenGL loop will use the old EglImage texture and ofxomxplayer will be rendering to the new one. I'm not also sure if this will fit to the main openframe concept.

It seems that using audio track really helps prevent the fast playing after looping. In my projects I normally use several short video sequences playing at the same time without sound track. And even the users can use their own video sequences. There is also no audio output from the system. So I disabled generally processing of audio tracks (enableAudio = false) to conserve more GPU/CPU power.The old omxplayer code works with that configuration OK. But I'm not so skilled to find out which block in omx processing code do the loop problem. So I'm not able to move to the latest omxplayer code to use the latest features and fixes( tearing ). It would be great if you succeed to solve it.

Anyway very thanks for your effort.

[jvcleave]

I don't think I can use 2 different EGLImages as once one is generated it is passed to the OMX egl_render component which "owns" it. I thought about having 2 textures and copying the EGLImage but decided to just use an FBO internally instead as I wouldn't have the copy overhead and I need an FBO for pixel access anyway (ofTexture::readToPixels isn't available with OpenGL ES 2)

So right now this is incorporated in master and was playing your file fine (with an audio track) with the default ofSetVerticalSync(true). I am not really sure why vertical sync matters as it worked with it disabled previously. This combined with the audio helping out looping makes me suspect an issue with the ffmpeg version so I may try and revert that when I get some more time

The problem I have with the CMU version is that it doesn't play back h264 streams correctly which is what the Camera module records

[adrchlik]

Yes I've tested all on the latest master branch. As I understand you well I'm not sure if render to FBO is less CPU/GPU intensive than copy data between textures (glCopyTexImage2D), because even with FBO you still need one reading from texture (EGLImage) and one writting to texture(FBO). I think only way to avoid extra copying is use two instances of EGLImage and just switch between them. I don't know if it is possible but I can imagine create two EGLmages and after each rendered frame try to point the egl_render component to the second EGLImage. I don't have much experience with omx library so I really don't know if you can change EGLImage in egl_render component while the decoding process is running. Maybe your solution is the only best way how to cope with the tearing.

Regard the loop problem.That's new idea to me that the problem could be in a new ffmpeg library. I was all the the time suspect handling the omx stuff.

[adrchlik]

Just for info. I've tested the new master branch with the old ffmpeg library. The ffmeg library was from version which do not suffer from "loop speeding" problem (Sep 08, 2013). But the master branch with old ffmpeg library still have the "loop speeding" problem.

[adrchlik]

Hello. I've time to investigate the loop speeding problem. The commit where it starts speeding after short video loop is 0c4b157b from 2014-01-23 I delved down to the code and the key problem is missing the set of "m_pts" variable in function bool OMXPlayerVideoBase::Decode(OMXPacket *pkt). There is a block in old code in file OMXPlayerVideoBase.cpp in function mentioned above if(pkt->pts != DVD_NOPTS_VALUE) { m_pts = pkt->pts; } where in the latest code is missing. I am not sure what is the missing code about but it seems that it help prevents the loop speeding issue.

[jvcleave]

@adrchlik have you updated this somewhere and tested it to work?

[adrchlik]

I've tried to add it on the latest master branch and it helps. The loop speeding problem disappered.

[jvcleave]

Can you do a PR?

[adrchlik]

Sorry. I was out for several days. But I see you have done this patch already. For confirmation I've tested it on the latest master and it works. Thanks.