demo av. autorestart(false). Video stops playing near end of movie but sound still plays.

[Celestria]

To reproduce: start demo av. Near the end of the movie (can seek, doesnt matter if you do or not) the image will freeze and the sound will continue playing for a second. This only occurs when autorestart is set to false before starting the demo.

I tried to compare the demo av output to what vlc player plays. vlc player will show more video frame than demo av. So to me it looks like demo av's sound is correct but for some reason there are no video frames anymore.

[kspes]

It's most likely that the audio and video duration are not the same. check that please and let me know.

[Celestria]

Are you talking about audio and video stream in the "demo av" sample.ogv file? I have no tools for this. If you know a free tool that I can use to see this please let me know and I will check.

What I did test was if I use vlc to play sample.ogv it will show more video frames than theoraplayer is showing. So where theoraplayer stops adding video frames to the queue vlc goes on for about a second longer. I tried setting the frame queue size to 0 but this did not have any effect.

When comparing the sounds between vlc and theoraplayer I would say they stop playing at the exact same time.

[kspes]

not sure, check vlc stream info, maybe you can see there. we have had these problems and usually the reason is that the streams are of different size. is the audio in a separate file or embedded in the same file?

[Celestria]

The file I am using is the exact same sample.ogv that is in the theoraplayer-master.zip that I downloaded from this repo 30 minutes ago. Literally all I'm doing is opening the project in vs2013, setting the the opengl demos as startup project, setting clip->setAutoRestart(false); in demo_av.cpp and pressing play.

I tried looking in vlc but I cannot find any menu that will tell me the audio and video length in sample.ogv. Do you know where I can find it?

[kspes]

ah, I see. I found the file, but yeah, not sure either, can't be visible from VLC. I'll have to investigate this file when I find the time. It's probably a problem with the file, it was encoded long ago.

you can try another video file, or better yet, use ffmpeg2theora tool to encode one of your videos and check.

[Celestria]

I actually noticed the issue in one of my (by ffmpeg2theora encoded. My source file was uncompressed avi with uncompressed sound) files and then I tried to see if the demo_av was having the same issue or I did something wrong in my integration code. I personally do not believe it is an issue with the sample file. Will explain in my next comment.

[Celestria]

I believe this has something to do with the way the end of the file is detected. But only when there is sound.

The funny thing is that in case I set clip->setAutoRestart(true); it will actually play all videoframe and audio samples untill the file is really at the end. Unfortunately I have not yet been able to find why autorestart has an effect on this since I would expect autorestart to only effect the the audio/video output.

In the image you can see the last frame that is shown when autorestart is false. I tried to do the same for when autostart is true but then I had to press printscreen at the exact time the video loops. So I might be a couple frames off. But compared to the image where autorestart is false there is about a whole second inbetween those images. The image with the frame from vlc is very close to the image of "autorestart true".

I would say for some reason when autostart is false it will not fetch the last frames eventhough we know the frames are in the sample file (vlc is showing them and if autostart is true theoraplayer is showing them too). We also know there is more sound because when autorestart is false the sound keeps on playing while the video image is not doing anything (eventhough we know there are more frames in the files).

[Celestria]

Just verified that this only happens when the video has an audio stream. I removed the sound of my media file and now it is played up until the last frame.

[Celestria]

Think I might have found something.

The issue occurs because the datasource is endOfFile and the decoder stops being called when the datasource is end of file eventhough there is still data left to decode.

But in this situation the there is still some "undecoded" data left for the decoder to decode.

see bool VideoClip_Theora::_decodeNextFrame() Here return false is done as soon as the datasource is endOfFile. Instead when the data source is end of file it should keep decoding until ogg_stream_packetout returns 0.

I did make some changes locally that seem to have solved the issue. Please have a look and see if I'm missing anything. Changes in attached file.

_decodeNextFrame.zip

[kspes]

I took a brief look at the code. seems like it could work. Some minor tweaks will probably be needed. I'll take a look at this more thoroughly when I find the time away from my work deadlines. Thanks for helping investigate this!

[Celestria]

Can you give me any pointers as to what to investigate? I'm kind of in a position where I need to make a decision soon as to which player lib I'm going to use. I kind of liked the open farmat idea of theora and hopefully soon webm with alpha support.

[kspes]

Your code changes seem to work fine, I suggest you use the modified version of the lib for your needs. I will check the code you sent and perform tests to make sure it doesn't break anything else when I find the time. I don't want to half-ass it :)

btw, Theoraplayer has alpha support for theora files too. WebM alpha implementation is native, beter and faster, but theora version in theoraplayer works just fine if you need it.

[Celestria]

I did not know there was alpha support. Movie files can be encoded with alpha like this? Or do i need to have a separate 8 bit file that i need to decode to get the alpha?

One other thing I was not able to find out yet. The way theoraplayer decodes my movie now is that it will output ayuv (not remembering the order at the moment). Then I use my own shader to make rgb of that. What I usually do when I use yuv is to use yuv planes. Is there a way to get theoraplayer decode to yuv planes 4:2:0 or 4:2:2 or something like that? I would like to perform a test as to which method/shader has the best performance.

[kspes]

take a look at these demos for more information: 1) http://libtheoraplayer.cateia.com/wiki/index.php?title=Demo:Composite_Animation 2) http://libtheoraplayer.cateia.com/wiki/index.php?title=Demo:Sprite_Animation

we use this feature A LOT in our games. you just have to encode videos in a specific way to have alpha support in libtheoraplayer and viola!

[kspes]

You mean you'd like to get separate buffers for the YUV components, then use them in your shader? But that would mean you'd have to upload 3 textures instead of one each frame. Not sure if it'll be faster or slower.? Maybe faster because the UV components are half-resolution, but 3 different render calls could slow it down.

in my experience, shaders do speed things up, but we made assembly optimisations for various targets that speed things up a lot, even without using shaders for YUV conversion. but they're not available on 64-bt architectures for theora.

[Celestria]

Kind of what you describe. But in 1 texture. The texture would be 8 bit with a fullsize plane and 2 half size planes.

The old decoder I used would have a big array of data. In that array there were 3 planes that the decoder would decode to.

Then whenever a frame was ready to be drawn I would copy that big array to a single texture. So now i have a texture with 3 planes. Then I use a shader during the drawcall that retrieves the correct values from each plane and create a pixel.

[kspes]

ok, kinda makes sense, that would work good for 4:2:0.

Theoraplayer doesn't support this method though, it outputs YUV formats in 4:4:4 regardless of what the source format is (Theora is mostly 4:2:0)

but it sounds like a good feature request