Performance with H.264

[ghost]

Using the example provided, I was able to compile and play a sample video file encoded with H.264. Using the default ofAVFoundationVideoPlayer on OS X 10.11.3 provides smooth playback at 30fps. Including this line to use Gstreamer

player.setPlayer(ofPtr<ofGstVideoPlayer>(new ofGstVideoPlayer));

works but the playback is very choppy, nowhere close to the performance of ofAVFoundationVideoPlayer. Is this expected? Do I need to do something to enable hardware acceleration, or is this just the current state of Gstreamer?

[arturoc]

it will should be faster if you use the programmable renderer and native pixels by calling:

player.setPixelFormat(OF_NATIVE_PIXELS);

before calling load. that will enable color space conversion in the graphics card but hardware accelerated decoding is still not implemented. gstreamer can do it but it's difficult to extract the final texture to a context created outside of gstreamer. this has possibly changed in latest versions.

what resolution are you using though? unless it's 4k or even bigger, h264 at 30fps should be ok on a modern computer

[ghost]

Yes I have a modern MacBook Pro, using 1920x1080 @ 30fps (H.264) source footage, which works just fine in every other application.

Using OF_PIXELS_NATIVE did speed it up considerably, but now my video is drawn grayscale.

Thanks for your help @arturoc.

[arturoc]

you need to use the programmable renderer so the color space conversion is doen through shaders in the graphics card. in main.cpp:

#include "ofMain.h"
#include "ofApp.h"

//========================================================================
int main( ){
    ofGLWindowSettings settings;
    settings.setGLVersion(3,3);
    settings.windowMode = OF_FULLSCREEN;
    ofCreateWindow(settings);

    // this kicks off the running of my app
    // can be OF_WINDOW or OF_FULLSCREEN
    // pass in width and height too:
    ofRunApp(new ofApp());

}

[CharStiles]

Hi, I am having this issue right now, it has a 3 second lag, I tried: player.setPixelFormat(OF_NATIVE_PIXELS); But it seems that OF_NATIVE_PIXELS is not defined in of 10. Is it called something else now? Thanks, Char

[arturoc]

the correct enum const is OF_PIXELS_NATIVE not OF_NATIVE_PIXELS

[CharStiles]

Thank you, but alas when I use OF_PIXELS_NATIVE the screen is mostly black with some glitchy looking artifacts. I am streaming from an HD ip cam streaming rtsp H264, I have a 1 second delay using this pipeline: gstv.setPipeline("rtspsrc location=rtsp://admin:@192.168.8.192:554/live;stream=0;drop-on-latency=true; latency=0 ! queue2 max-size-buffers=2 ! decodebin ! videoconvert ! videoscale", OF_PIXELS_RGB, true, feedWidth, feedHeight); I have been able to get closer to realtime stream using the windows software ispy (so that gives me hope that I can speed it up). Do you know if it's possible to get the delay/processing time/latency to be less than a second, or is that all I can hope for? I am using OF 10.1 on Majove.

[CharStiles]

I got OF_PIXELS_NATIVE working! But it had no effect on the latency. I suppose there is nothing else I can be doing on the OF side of things to speed it up, and I should keep working on bettering my pipeline? I'll report back here if I get it. Thanks for your responsiveness!

[arturoc]

Yes, OF_PIXELS_NATIVE will make the process less cpu intensive but won't reduce the latency. there's usually a parameter you can pass to the pipeline to lower the latency but can't remember at the moment