search

openFrameworks-RaspberryPi / openFrameworks

This repo has migrated into the openFramworks core! Please go to http://github.com/openFrameworks/openFrameworks for the latest!
http://github.com/openFrameworks/openFrameworks
Other
104 stars 11 forks source link

ofVideoGrabber - slow framerate, needs direct to texture option (via OMX) #21

Closed bakercp closed 11 years ago

bakercp commented 11 years ago

RPI has hardware acceleration for H264 videos.

There is a standalone video player called omxplayer on each RPI.

There is also som demo code here https://github.com/raspberrypi/firmware/tree/master/opt/vc/src/hello_pi/hello_video

For openFrameworks, this might best be implemented as a GST plugin / element, that would be auto-selected, by the current GST player, meaning we won't need to change any core oF code. This should happen once the element is in the core raspbian distro. More on progress here:

http://cgit.freedesktop.org/gstreamer/gst-omx/log/?h=raspberry http://www.raspberrypi.org/phpBB3/viewtopic.php?f=38&t=6852&sid=b679037b35691215df6e1c65ff95fa7b&start=100

It should support the paid codecs offered by RPI (i.e. VC1 and MPEG2 http://www.raspberrypi.com/)

From @arturoc: "openmax is actually a open standard from the kronos group, is also supported on latest versions of android so it would be good to have an ofOMXVideoPlayer."

arturoc commented 11 years ago

btw, this will actually only return textures if i'm correct so anything that needs pixels will be really slow since you need to download the pixels from the graphics card, i've used ffmpeg on android in the past and the performance was ok for small videos, in a 2 year old phone so might be useful to try the gstreamer player as it is now, for getting pixels it might be faster than openmax

danzeeeman commented 11 years ago

I thought we could pull the pixels from the hardware demuxer?

On Wed, Nov 21, 2012 at 3:01 AM, arturo notifications@github.com wrote:

btw, this will actually only return textures if i'm correct so anything that needs pixels will be really slow since you need to download the pixels from the graphics card, i've used ffmpeg on android in the past and the performance was ok for small videos, in a 2 year old phone so might be useful to try the gstreamer player as it is now, for getting pixels it might be faster than openmax

— Reply to this email directly or view it on GitHubhttps://github.com/openFrameworks-RaspberryPi/openFrameworks/issues/21#issuecomment-10588469.

"I believe in science. Unlike mathematical theorems, scientific results can't be proved. They can only be tested again and again, until only a fool would not believe them.

I cannot prove that electrons exist, but I believe fervently in their existence. And if you don't believe in them, I have a high voltage cattle prod I'm willing to apply as an argument on their behalf. Electrons speak for themselves."

-- Seth Lloyd: Quantum Mechanical Engineer, MIT

/.

arturoc commented 11 years ago

yes you should be able to get the pixels but downloading things from the graphics card is usually super slow

danzeeeman commented 11 years ago

that depends on a couple things. the broadcom chip has a separate hardware demuxer for encoded video that is different from the GPU (or at least that is what I understand, please correct me if I'm wrong)/ so we should be able to pull the video frameBuffer. I will have to do some research.

On Wed, Nov 21, 2012 at 3:52 PM, arturo notifications@github.com wrote:

yes you should be able to get the pixels but downloading things from the graphics card is usually super slow

— Reply to this email directly or view it on GitHubhttps://github.com/openFrameworks-RaspberryPi/openFrameworks/issues/21#issuecomment-10613824.

"I believe in science. Unlike mathematical theorems, scientific results can't be proved. They can only be tested again and again, until only a fool would not believe them.

I cannot prove that electrons exist, but I believe fervently in their existence. And if you don't believe in them, I have a high voltage cattle prod I'm willing to apply as an argument on their behalf. Electrons speak for themselves."

-- Seth Lloyd: Quantum Mechanical Engineer, MIT

/.

jvcleave commented 11 years ago

I started an attempt of converting hello_video and have all the components initializing. While doing some troubleshooting I ran across this which looks like it could be ported to OF - going to try this next

https://github.com/esrlabs/AndroidTransporterPlayer/blob/master/RPiPlayer.cpp

jvcleave commented 11 years ago

I got the hello_pi/hello_video sample functionality in an OF app - I am curious if it works for others as I am having issues trying to link ilclient lib through the project.make system

I haven't gotten to the point to where I manipulating pixels yet but you should be able to compile and play video

http://jvcref.com/files/PI/tests/jvc_video.1.zip

jvcleave commented 11 years ago

@danthemellowman @arturoc I think this may be a clue on how to get from the card (been trying to hijack the input without success, going to try this tomorrow) https://github.com/raspberrypi/firmware/blob/master/opt/vc/src/hello_pi/hello_encode/encode.c

arturoc commented 11 years ago

in ubuntu omap4 (will only work with pandaboard and similar) there's a gstreamer plugin that decodes and encodes in hardware and returns pixels, i'm still having crashes with it but it works in totem... it's actually based on openmax so gst-omx should work too on the rpi

arturoc commented 11 years ago

i have the videoplayer working under gstreamer with hw acceleration but it's kind of slow, i think it's related with the colorspace conversion + uploading to the graphics card again but it's weird since i've done this in software in android and the framerate was ok

jvcleave commented 11 years ago

@arturoc do you have your videoplayer published anywhere yet? curious to compare

arturoc commented 11 years ago

it's the usual videoplayer in linux, the gstreamer one, you just need to use ofVideoPlayer, the videoPlayerExample should work. I fixed it for omap but it should work ok with the raspberry, all i did was adding a colorspace conversion that the pipeline wasn't doing automatically

jvcleave commented 11 years ago

I am currently getting this - I've stripped everything out but loadMovie, play, update and draw 

[verbose] ofGstUtils: gstreamer inited
[verbose] loading file:///home/pi/openFrameworks/apps/devApps/videoPlayerExample/bin/data/fingers.mov
[verbose] attaching callbacks
[verbose] Gstreamer: speed change to 1.000000
[warning] received a preroll without allocation

here is the test code - anything I should try?

#include "testApp.h"

bool hasStarted = false;
//--------------------------------------------------------------
void testApp::setup(){
    ofSetLogLevel(OF_LOG_VERBOSE);
}

//--------------------------------------------------------------
void testApp::update(){
    int delay = 4000;
    unsigned long long currentTime = ofGetElapsedTimeMillis();
    if (currentTime > delay) 
    {
        if (!hasStarted) 
        {
            hasStarted = true;
            videoPlayer.loadMovie(ofToDataPath("fingers.mov", true));
            videoPlayer.play();
        }else {
            videoPlayer.update();
        }

    }else 
    {
        ofLogVerbose() << "waiting " << delay - currentTime << "ms";
    }
}

//--------------------------------------------------------------
void testApp::draw(){
    if (!hasStarted) {
        return;
    }
    videoPlayer.draw(0, 0, 320, 240);
}
arturoc commented 11 years ago

it's weird that you are not getting any other messages but seems like the pipeline is not able to get the size of the movie and when you get the fist frame the buffers are not allocated yet.

you can try to deebug gstreamer to see if there's any more errors. before executing the program, run:

export GST_DEBUG=:2

then run the example and you should get a long debug trace from the same gstreamer, if you want to post it use also

export GST_DEBUG_NO_COLOR=1

so it doesn't generate weird characters for the color output

On 11/30/2012 09:19 AM, Jason Van Cleave wrote:

I am currently getting this - I've stripped everything out but loadMovie, play, update and draw

[verbose] ofGstUtils: gstreamer inited [verbose] loading file:///home/pi/openFrameworks/apps/devApps/videoPlayerExample/bin/data/fingers.mov [verbose] attaching callbacks [verbose] Gstreamer: speed change to 1.000000 [warning] received a preroll without allocation

here is the test code - anything I should try?

|#include "testApp.h"

bool hasStarted = false; //-------------------------------------------------------------- void testApp::setup(){ ofSetLogLevel(OF_LOG_VERBOSE); }

//-------------------------------------------------------------- void testApp::update(){ int delay = 4000; unsigned long long currentTime = ofGetElapsedTimeMillis(); if (currentTime > delay) { if (!hasStarted) { hasStarted = true; videoPlayer.loadMovie(ofToDataPath("fingers.mov", true)); videoPlayer.play(); }else { videoPlayer.update(); }

 }else
 {
     ofLogVerbose() << "waiting " << delay - currentTime << "ms";
 }

}

//-------------------------------------------------------------- void testApp::draw(){ if (!hasStarted) { return; } videoPlayer.draw(0, 0, 320, 240); } |

— Reply to this email directly or view it on GitHub https://github.com/openFrameworks-RaspberryPi/openFrameworks/issues/21#issuecomment-10881684.

jvcleave commented 11 years ago

Here is the output from that (did you mean GST_DEBUG=2 ? I tried both...)

[verbose] loading file:///home/pi/openFrameworks/apps/devApps/videoPlayerExample/bin/data/fingers.mov
0:00:00.858212852 30166  0x17f4890 WARN                 qtdemux qtdemux_types.c:191:qtdemux_type_get: unknown QuickTime node type avc1
0:00:00.859994845 30166  0x17f4890 WARN                 qtdemux qtdemux_types.c:191:qtdemux_type_get: unknown QuickTime node type avcC
0:00:00.861679839 30166  0x17f4890 WARN                 qtdemux qtdemux.c:4613:qtdemux_parse_container:<qtdemux0> length too long (720896 > 15)
0:00:00.863300833 30166  0x17f4890 WARN                 qtdemux qtdemux_types.c:191:qtdemux_type_get: unknown QuickTime node type ?aut
0:00:00.865078827 30166  0x17f4890 WARN                 qtdemux qtdemux.c:4613:qtdemux_parse_container:<qtdemux0> length too long (720896 > 15)
0:00:00.866533821 30166  0x17f4890 WARN                 qtdemux qtdemux.c:4613:qtdemux_parse_container:<qtdemux0> length too long (720896 > 15)
0:00:00.868093816 30166  0x17f4890 WARN                 qtdemux qtdemux.c:8854:qtdemux_parse_tree:<qtdemux0> Can't handle datetimes before 1970 yet, please file a bug at http://bugzilla.gnome.org
[verbose] attaching callbacks
0:00:01.636221999 30166  0x17f4ac0 WARN                  ffmpeg gstffmpegdec.c:2299:gst_ffmpegdec_frame:<ffdec_h2640> ffdec_h264: decoding error (len: -1, have_data: 0)
[verbose] Gstreamer: speed change to 1.000000
[warning] received a preroll without allocation
arturoc commented 11 years ago

is difficult to say what's going on but it seems like it couldn't find the size of the video, i would try with some other codec, or even with a different h264 video, i was also receiving some errors from the qt demuxer in the pandaboard but it was working for me. i'll check in the rpi as soon as i get mine ;)

jvcleave commented 11 years ago

cool - just for reference I have been looking at these in troubleshooting this and videograbber (still trying to understand the sink concepts) http://www.aonsquared.co.uk/the_dark_pi_rises http://letsmakerobots.com/node/33589

arturoc commented 11 years ago

the pipelines seem more or less the same i'm using, have you tried the videograbber? it should work according to that. the sink is just the element that receives the final stream after decoding it and converting the color space. usually to show it in the screen, with xvimagesink or similar. in that examples the tcpsink sends the stream through the network, in OF there's an appsink that sends the decoded rgb frames to OF where they are uploaded to a texture and stored in ofPixels

jvcleave commented 11 years ago

yeah - I was playing around trying to pass custom ones before your recent update.

As for videograbber, that seems to have a different issue for me

this is the Ps3eye (which works well under mPlayer)

[notice] ofGstUtils: selected device: USB Camera-B4.04.27.1
[notice] ofGstUtils: selected format: 320x240 video/x-raw-rgb framerate: 125/1
[notice] gstreamer pipeline: v4l2src name=video_source device=/dev/video1 ! video/x-raw-rgb,width=320,height=240  ! ffmpegcolorspace   ! appsink name=ofappsink caps="video/x-raw-rgb, depth=24, bpp=24, endianness=4321, red_mask=0xff0000, green_mask=0x00ff00, blue_mask=0x0000ff, alpha_mask=0x000000ff, width=320, height=240"
[New Thread 0x45e84310 (LWP 30710)]
[verbose] attaching callbacks
jvc_videoGrabberExample_debug: tpp.c:63: __pthread_tpp_change_priority: Assertion `new_prio == -1 || (new_prio >= __sched_fifo_min_prio && new_prio <= __sched_fifo_max_prio)' failed.

Program received signal SIGABRT, Aborted.
0x4087cbfc in raise () from /lib/arm-linux-gnueabihf/libc.so.6

another cheap camera I have just hangs at the callback attachment phase

[notice] ofGstUtils: selected device: USB camera
[notice] ofGstUtils: selected format: 320x240 video/x-raw-rgb framerate: 100/1
[notice] gstreamer pipeline: v4l2src name=video_source device=/dev/video0 ! video/x-raw-rgb,width=320,height=240  ! ffmpegcolorspace   ! appsink name=ofappsink caps="video/x-raw-rgb, depth=24, bpp=24, endianness=4321, red_mask=0xff0000, green_mask=0x00ff00, blue_mask=0x0000ff, alpha_mask=0x000000ff, width=320, height=240"
[verbose] attaching callbacks

I've found overall that video and devices are not very consistent - sometimes the apps crash - cameras work better when re-plugged in. It will be good once you get yours so we can compare

arturoc commented 11 years ago

the first error seems to be something inside gstreamer trying to set an illegal priority to one of the threads, perhaps try running the example with sudo.

i'll also try later today with my beaglebone where i have debian, things should be more or less the same there as in raspbian, will let you know if i find something

arturoc commented 11 years ago

i've been trying in the beaglebone with debian and i think it's just some plugins missing, at least here it solves it and i was getting a similar error message.

be sure to execute install_codecs.sh and then run:

apt-get install gstreamer0.10-nice

if that solves it let me know so i can modify the install scripts

jvcleave commented 11 years ago

hopefully you aren't getting spammed with this message - typing it into the github comments is sending it into the abyss. (edit: I got locked out - had to send an email to github support but I appear to be clear now)

I did these steps

ran install_codecs.sh (nothing needed) ran sudo apt-get install gstreamer0.10-nice (install needed)

here is that output: https://gist.github.com/4200846

I also tried mplayer which played back fingers.mov quite nicely. Here is that output https://gist.github.com/4200675

arturoc commented 11 years ago

that's weird i'm getting the notifications of your posts but don't see them here. anyway, i think it's just some missing plugin since you don't get the debug traces from gstreamer.

i also had to install gstreamer-0.10-tools but if you have gst-launch they should be installed, give it a try anyway

and the correct command to see the debug from gstreamer is:

export GST_DEBUG=*:2

arturoc commented 11 years ago

it seems the problem with this is also related with #61

@bakercp when you say raspberry pi pthreads, what do you mean? is there a special library somewhere for pthreads in rpi?

danzeeeman commented 11 years ago

I'm getting this when trying to run the serialExample. I haven't dug deeper into the code [Thread 1:warning] cannot start, thread already running

bakercp commented 11 years ago

@danthemellowman your issue had to do with thread problems when used with GLES2. That should be fixed with fixed with #63.

Regarding the rest of this -- when I try to run the videoPlayerExample I get:

ofAppEGLWindow constructor 
in ofAppEGLWINDOW: setupOpenGL
succes=0
   REQUESTED SCREEN SIZE w=1024 and  h=768
HARDWARE SCREEN SIZE IS sw=1360 and sh=768
CREATING A SCREEN THAT IS w=1024 and h=768
[ofAppEGLWindow::setupEGL:notice] setting EGL Display
[ofAppEGLWindow::setupEGL:notice] setting default Display
[ofAppEGLWindow::setupEGL:notice] EGL Display correctly set
[ofAppEGLWindow::setupEGL:notice] eglInitialize
[ofAppEGLWindow::setupEGL:notice] No current render selected.
[ofAppEGLWindow::setupEGL:notice] Default Renderer detected.
[ofAppEGLWindow::setupEGL:notice] eglChooseConfig
[ofAppEGLWindow::setupEGL:notice] eglCreateWindowSurface
[ofAppEGLWindow::setupEGL:notice] eglCreateContext
[ofAppEGLWindow::setupEGL:notice] eglMakeCurrent
EGL_VERSION = (null)
GL_RENDERER = VideoCore IV HW
GL_VERSION  = OpenGL ES-CM 1.1
GL_VENDOR   = Broadcom
CREATED SCREEN WITH SIZE 1024 x 768

(videoPlayerExample:10846): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(videoPlayerExample:10846): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(videoPlayerExample:10846): GStreamer-CRITICAL **: gst_bin_add: assertion `GST_IS_ELEMENT (element)' failed

(videoPlayerExample:10846): GStreamer-CRITICAL **: gst_element_get_static_pad: assertion `GST_IS_ELEMENT (element)' failed

(videoPlayerExample:10846): GStreamer-CRITICAL **: gst_ghost_pad_new: assertion `GST_IS_PAD (target)' failed

(videoPlayerExample:10846): GStreamer-CRITICAL **: gst_object_unref: assertion `object != NULL' failed

(videoPlayerExample:10846): GStreamer-CRITICAL **: gst_element_add_pad: assertion `GST_IS_PAD (pad)' failed

(videoPlayerExample:10846): GStreamer-CRITICAL **: gst_element_link_many: assertion `GST_IS_ELEMENT (element_1)' failed

(videoPlayerExample:10846): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(videoPlayerExample:10846): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed
[error] GStreamer: unable to set pipeline to ready

(videoPlayerExample:10846): GStreamer-CRITICAL **: gst_element_query_duration: assertion `GST_IS_ELEMENT (element)' failed
[warning] GStreamer: cannot query time duration

** (videoPlayerExample:10846): WARNING **: gstvideo: failed to get caps of pad app_sink:sink
[error] GStreamer: cannot query width and height

(videoPlayerExample:10846): GStreamer-CRITICAL **: gst_element_query_position: assertion `GST_IS_ELEMENT (element)' failed
[warning] ofGstVideoUtils not loaded
Segmentation fault

I installed all of these system libs: 

sudo apt-get install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev gstreamer0.10-nice gstreamer0.10-tools libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libasound2-dev libxmu-dev libxxf86vm-dev g++ libgl1-mesa-dev libglu1-mesa-dev libraw1394-dev libudev-dev libdrm-dev gstreamer0.10-ffmpeg libglew-dev libopenal-dev libsndfile-dev libfreeimage-dev libcairo2-dev libgtk2.0-dev libjack-jackd2-dev python-lxml python-argparse portaudio19-dev

Am I missing something obvious?

arturoc commented 11 years ago

i've also installed:

gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-ugly gstreamer0.10-alsa gstremaer0.1-tools

you'll need to resize the partition ( raspberry-config has an option for that) since in the default 2g image it won't fit

but anyway the videoplaying is too slow. the videograbber is almost ok but it gets stuck for a couple of frames from time to time, i'll try to optimize it tomorrow

bakercp commented 11 years ago

Works now! Slow indeed, but it's working and pixels are accessible. I'm assuming this isn't using the hardware decoder? Or if it is, the GPU readback is slowing things down a lot?

arturoc commented 11 years ago

no it's just the software decoder, i'll try gst-omx tomorrow

bakercp commented 11 years ago

looking forward to it!

http://christopherbaker.net

On Tue, Dec 4, 2012 at 2:54 PM, arturo notifications@github.com wrote:

no it's just the software decoder, i'll try gst-omx tomorrow

— Reply to this email directly or view it on GitHubhttps://github.com/openFrameworks-RaspberryPi/openFrameworks/issues/21#issuecomment-11015275.

bakercp commented 11 years ago

(updated title)

jvcleave commented 11 years ago

I took a stab at gst-omx. The best reference for installing (that I found ) was here: http://www.raspberrypi.org/phpBB3/viewtopic.php?f=38&t=19606&p=191153#p191153

I didn't use his adds to .profile - I just put gst-omx libs and the .conf into /usr/lib/arm-linux-gnueabihf/gstreamer-0.10/

seemed to work 

pi@raspberrypi$ gst-inspect-0.10 | grep omx
openmax:  omxmpeg4videodec: OpenMAX MPEG4 Video Decoder
openmax:  omxh264dec: OpenMAX H.264 Video Decoder

First error is this: gstvideo: failed to get caps of pad app_sink:sink

https://gist.github.com/4281295#file-gst-omx-of-attempt-L64

arturoc commented 11 years ago

actually i made gst-omx work but it wasn't worth, the colorspace conversion happens in the cpu and that + having to reupload pixels to the card make it as slow as the cpu version. i found that there's a faster colorspace conversion in gstreamer (ffmpegcolorspace is super slow) i still need to test that. another option now that we have gles2 could be retrieving yuv and doing colorspace in a shader but that would require lots of changes to ofPixels, ofVideoGrabber/Player...

arturoc commented 11 years ago
jvcleave commented 11 years ago

interesting - if you still have the version you got to work I would be interested in seeing it - mostly to see what I was doing different.

this guy had some numbers around using shaders using Qt http://www.raspberrypi.org/phpBB3/viewtopic.php?p=192425#p192425

jvcleave commented 11 years ago

I followed the thread above, found and was able to install gst-plugins-gl: http://cgit.freedesktop.org/gstreamer/gst-plugins-gl/log/

It says ES2 support so maybe it is useful? I couldn't get it to display anything (but I couldn't get gst-omx to either)

if you decide to clone it and try it you will have to do this fix in the master branch http://bugzilla-attachments.gnome.org/attachment.cgi?id=216088

danzeeeman commented 11 years ago

you need libv4l-0 and v4l-utils for the pi

bakercp commented 11 years ago

ping? what's the state of the art on this @jvcleave @danthemellowman @arturoc ?

jvcleave commented 11 years ago

I think it all leads back to what @arturoc was talking about - have a specialized OMX Player

Most of the stuff I researched around gst-omx (and what I think arturo experienced) is that it didn't work well yet or work on raspbian.

This seems the most promising lead I have found but it is currently using Wheezy/Qt based and pretty heavy http://thebugfreeblog.blogspot.it/2012/12/decoding-and-rendering-to-texture-h264.html

danzeeeman commented 11 years ago

have you guys looked at https://github.com/huceke/omxplayer it is the test code the XBMC guys wrote for the pi.

On Thu, Jan 3, 2013 at 10:49 PM, Jason Van Cleave notifications@github.comwrote:

I think it all leads back to what @arturoc https://github.com/arturocwas talking about - have a specialized OMX Player

Most of the stuff I researched around gst-omx (and what I think arturo experienced) is that it didn't work well yet or work on raspbian.

This seems the most promising lead I have found but it is currently using Wheezy/Qt based and pretty heavy

http://thebugfreeblog.blogspot.it/2012/12/decoding-and-rendering-to-texture-h264.html

— Reply to this email directly or view it on GitHubhttps://github.com/openFrameworks-RaspberryPi/openFrameworks/issues/21#issuecomment-11874016.

"I believe in science. Unlike mathematical theorems, scientific results can't be proved. They can only be tested again and again, until only a fool would not believe them.

I cannot prove that electrons exist, but I believe fervently in their existence. And if you don't believe in them, I have a high voltage cattle prod I'm willing to apply as an argument on their behalf. Electrons speak for themselves."

-- Seth Lloyd: Quantum Mechanical Engineer, MIT

/.

arturoc commented 11 years ago

i think we should just wait for gstreamer to get fixed, it can take some months but seems like the most universal solution. there's ongoing work for accelerated video playing with gst on desktop, omap and rpi. also no matter which solution we implement direct to texture needs some kind of hardware colorspace conversion (gst also will implement that at some point, there's something like this in the desktop through vaapi) or it'll be useless. implementing it ourselves will mean adding support for lots of pixel formats, like every yuv in ofPixels, what we can do at some point +using shaders for colorspace conversion from the videoplayer/grabber what can be done but would need some radical architectural changes in the core

at some point i talked with @obviousjim about having a usePixels methods in videoPlayers and grabbers so you can disable pixels to get only textures if the video can be hardware accelerated but i think we should do this later for all platforms after merging this into the core

by now i want to try a faster colorspace conversion in gstreamer which is already there, not hardware but still faster than the super slow ffmpegcolorspace element, if that works i think that's enough by now but if someone wants to try to implement omx meanwhile we can have it as an addon

jvcleave commented 11 years ago

As for going the OMX addon route - I've been digging through the code from the last link I sent. It isn't as heavy as I previously thought as much of the code is based off the hello_video example that I had mostly working in OF anyway.

Most of the magic seems to center around this function where you pass in a EGLImage for the video to write to

OMX_UseEGLImage http://maemo.org/api_refs/5.0/beta/libomxil-bellagio/group__buf.html#ga9d093a08220dd91e4a95f7adbaa30dc

arturoc commented 11 years ago

i've just updated both develop in the main repo and develop-raspberrypi here to be compatible with gstreamer-1.0 which incliudes faster colorspace conversion. it's still not present in raspbian but as soon as it is it should work, i still need to change the install scripts to detect if 1.0 can be installed

bakercp commented 11 years ago

@arturoc @kalwalt do you think this issue applies to the Pandaboard? I'm currently classifying some of these issues to get a better sense of what should be prioritized.

arturoc commented 11 years ago

yes there's also hw accelerated decoding in the pandaboard, and you can use it through gstreamer but there's no hw accelerated colorspace conversion yet, but as i said before i would just wait till it fixes itself :) i mean at some point gstreamer gl elements will work on gles properly and we'll get hw colorspace conversion. for the rpi we could have the omx video player as an addon, since doing it in the core will need lots of refactoring and it won't work in any other platform

danzeeeman commented 11 years ago

@bakercp @arturoc @jvcleave can we close this?

jvcleave commented 11 years ago

I think we can close it as it is a broad issue and works as well as it can without gst-omx (ofVideoPlayer-wise). ofVideoGrabber is going to be limited by the USB processing. The only OMX solutions for camera are for specific chipsets that have embedded cameras (like an Android phone's camera)