Patch for native OS X overlay windows, retrieving video width/heigh/rate, pre-roll frames

[GoogleCodeExporter]

Hi,

I've attached a patch file for a few things I've added to gstreamer-java:

(1) Added GstVideo API bindings to be able to easily retrieve the video width, 
height, and frame rate. (See issue #42.)

(2) Added a listener for pre-roll frames so that the "next" frame of the video 
is displayed while the video is paused (instead of just displaying black).

(3) Added support for osxvideosink and Apple's CocoaComponent 
native/heavy-weight component adapter. This significantly improves video 
playback performance on OS X by rendering video frames in GStreamer directly 
onto a native OS X window (instead of passing it through Java/Swing/etc) - the 
OS X equivalent of directdrawsink. I had to bump the JNA version up to 3.2.7 in 
order to implement this. This also requires Apple's java extensions bindings to 
compile on non-OS X platforms - see attached Maven pom package that needs to be 
hosted somewhere.

This patch file is against the SVN repository as of yesterday (12th Oct). 
However, it also includes the patches that I submitted earlier (see issue #51).

Cheers,

Dave

Original issue reported on code.google.com by naskies@gmail.com on 13 Oct 2010 at 2:04

Attachments:

• patch.diff
• applejavaextensions.tgz

[GoogleCodeExporter]

Good work. I will review and add to the trunk if everything is ok.

Original comment by andres.c...@gmail.com on 13 Oct 2010 at 2:51

[GoogleCodeExporter]

it's a rather large diff so it'd be useful to split. one thing i already 
recognize in the previous version is that is you drop "n_prealloced" from 
GValueArray which you can't do since it _must_be the same as the native 
structure!

another not the if you add a new para to a function (like in case of rgbFrame's 
isPrerollFrame why not add as the last parameter and create another function to 
remain compatible with the previous version?
the sam apply to the getValue -> toJava rename.

you should syncronize with the stream-info branch at least StreamInfo.java

Original comment by lfar...@gmail.com on 13 Oct 2010 at 8:12

[GoogleCodeExporter]

Hi,

n_prealloced is not a member of the GValueArray structure, according to the 
official Gnome documentation. This is one of the reasons why getStreamInfo() 
crashes. See:

http://library.gnome.org/devel/gobject/unstable/gobject-Value-arrays.html#GValue
Array

I'm just working on the minimum source changes required to integrate GStreamer 
into a project that I'm working on. Feel free to ignore these patches if 
they're not up to your project standards.

Original comment by naskies@gmail.com on 13 Oct 2010 at 8:27

[GoogleCodeExporter]

The patch refers to "import org.gstreamer.Video". Shouldn't Video.java be part 
of the patch as well? Or am I missing something?

Original comment by andres.c...@gmail.com on 13 Oct 2010 at 1:14

[GoogleCodeExporter]

Sorry - I'd left out a few new files by accident. Here is a new patch against 
gstreamer-java-stream-info branch as requested - it includes the missing files. 
It still needs applejavaextensions and JNA 3.2.7.

Original comment by naskies@gmail.com on 14 Oct 2010 at 5:02

Attachments:

• patch.diff

[GoogleCodeExporter]

I applied the latest patch to the gstreamer-java-stream-info branch. The 
width/height/framerate retrieval for PlayBin and PlayBin2 elements work as 
expected. However, I found a problem when trying to create a new RGBDataSink 
using the name and pipeline, in this case the line (#82 in RGBDataSink.java):

videosink = (FakeSink) pipeline.getElementByName("VideoSink"); 

gives an error when getElementByName returns a BaseSink and not a FakeSink 
(BaseSink cannot be casted to FakeSink), even if the element VideoSink was 
properly created as a fakesink. From reviewing the changes, videosink was 
declared before as BaseSink. Any ideas?

Original comment by andres.c...@gmail.com on 16 Oct 2010 at 9:23

[GoogleCodeExporter]

That line should actually be:

        videosink = (FakeSink) NativeObject.objectFor(pipeline.getElementByName("VideoSink").getNativeAddress(), FakeSink.class);

Btw, I can't figure out off the top of my head how to import 
gst_fake_sink_get_type() from the gstcoreelements plugin so that it can be 
added to the type map in GstTypes.java.

Original comment by naskies@gmail.com on 16 Oct 2010 at 11:32

Attachments:

• patch.diff

[GoogleCodeExporter]

By the way, I think the new files (Video.java, GstVideoAPI.java, 
OSXVideoSink.java, OSXVideoComponent.java, applejavaextensions.jar) are missing 
from your commit to the gstreamer-java-stream-info branch.

Original comment by naskies@gmail.com on 16 Oct 2010 at 11:36

[GoogleCodeExporter]

Hi, i just noticed that Tal created a new branch for the stream-info changes. 
It is not my intention to mix up things by committing the last patch from 
naskies to the stream-info branch, please let me know what would be appropriate 
methodology for applying/testing these patches (separate branch for each patch, 
etc).

Original comment by andres.c...@gmail.com on 17 Oct 2010 at 1:29

[GoogleCodeExporter]

i like one brach and try to check in all new patches and even review them.
imho all addition feature extension are welcome (something broken better than 
nothing), but try to not mess up currently working part.

Original comment by vitorl...@gmail.com on 17 Oct 2010 at 3:45

[GoogleCodeExporter]

Work can continue in the stream-info-branch. Or, it can be renamed, or copied 
into a new branch/activity with an appropriate name. SVN copy operations are 
quite cheap.
Later when the branch is merged into trunk, there will be a couple of conflicts 
- nothing serious.

I am in favour of 'feature branches' which are used only for a particular 
fix/enhancement and then are abandoned once they are merged into trunk.

Original comment by tsha...@gmail.com on 17 Oct 2010 at 6:43

[GoogleCodeExporter]

ok, I made new gstreamer-java-gstvideo branch where I applied the last patch 
from naskies to the current trunk source. 

Actually, question for naskies: Where should I put the apple extensions stuff? 
gsteramer-java/apple/java/applejavaextensions... ?

Original comment by andres.c...@gmail.com on 18 Oct 2010 at 1:47

[GoogleCodeExporter]

It can probably go into the lib/ directory along with the other JARs (swt, jna, 
etc). We'll just have to make sure that it gets picked up on the Java build 
path like those other libraries.

Original comment by naskies@gmail.com on 18 Oct 2010 at 10:52

[GoogleCodeExporter]

I added the applejavaextensions-1.4.jar to the lib folder. There were other 
files in the tgz (applejavaextensions-1.4.pom, etc). Are these needed as well?

Could you check if this branch compiles and works as expected?

Original comment by andres.c...@gmail.com on 19 Oct 2010 at 12:42

[GoogleCodeExporter]

I turned the JAR into a Maven pom bundle - but all we need for gstreamer-java 
is just the applejavaextensions-1.4.jar file. 

Yes - the gstreamer-java-gstvideo branch compiles fine for me. I don't have 
time to test it at the moment though... I'll let you know if I come across any 
problems.

Original comment by naskies@gmail.com on 19 Oct 2010 at 1:15

[GoogleCodeExporter]

when do you plan to merge back these branches to the head?
i applied all other pending issue and patches.

Original comment by lfar...@gmail.com on 19 Oct 2010 at 8:53

[GoogleCodeExporter]

I'll do some more testing on the branch today and if everything is fine I'll 
merge.

Original comment by andres.c...@gmail.com on 19 Oct 2010 at 10:31

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

@naskies: any reason why PREROLL_HANDOFF is defined in FakeSink and not in 
Element as with the other signals?

Original comment by andres.c...@gmail.com on 20 Oct 2010 at 3:02

[GoogleCodeExporter]

The preroll-handoff and handoff signals are actually defined by the FakeSink 
class, not Element. See GStreamer documentation below:

http://www.gstreamer.net/data/doc/gstreamer/head/gstreamer-plugins/html/gstreame
r-plugins-fakesink.html

http://www.gstreamer.net/data/doc/gstreamer/head/gstreamer/html/GstElement.html

Technically, I think that the HANDOFF signal should also be defined in FakeSink 
- but I didn't want to go on a refactoring rampage :-)

Original comment by naskies@gmail.com on 20 Oct 2010 at 3:33

[GoogleCodeExporter]

Yes, you are right. I did some refactoring putting the preroll-handoff and 
handoff signals in BaseSrc. I also added the handoff signal for BaseSink. 

The reason why I put the signals in the base elements instead of the fake 
src/sink is because the Bin.getElementByName(name) method returns a Base 
src/sink even if name identifies a fake src/sink in the pipeline. This leads to 
problems where you want to use a fake source or sink and connect to the handoff 
signals.

I did run various tests (pipelines, playbins, etc) and everything seems to be 
working fine.

Original comment by andres.c...@gmail.com on 20 Oct 2010 at 9:22

• Changed state: Fixed

[GoogleCodeExporter]

Issue 42 has been merged into this issue.

Original comment by andres.c...@gmail.com on 20 Oct 2010 at 9:25

[GoogleCodeExporter]

could you update the ant build file too. currently build failed because can't 
find the osx jar file during build.
thanks.

Original comment by lfar...@gmail.com on 20 Oct 2010 at 1:41

• Changed state: New

[GoogleCodeExporter]

The ant file for eclipse has the reference to the new apple jar (i did a fresh 
checkout from the trunk and it builds ok).

Maybe the ant file for netbeans should be updated as well? Since I'm not 
currently using netbeans, not sure where to make the changes. 
nbproject/project.properties?

Original comment by andres.c...@gmail.com on 21 Oct 2010 at 8:27

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Are we ready for the 1.5 release?

I didn't check/fix the dependencies for netbeans, since I don't want to break 
the config files for it. In Eclipse everything seems to work fine: I deleted by 
local copy of the trunk, checked out again and built from scratch, no errors.

Original comment by andres.c...@gmail.com on 24 Oct 2010 at 10:36

[GoogleCodeExporter]

I just checked out a copy of the trunk r435 and it builds fine for me in 
NetBeans 6.9.1 once my unrelated patch for issue #54 is applied.

Original comment by naskies@gmail.com on 25 Oct 2010 at 2:14

[GoogleCodeExporter]

i like to drop netbeans too, since i don't use it either. but i'd like to build 
from command line and i'd like to build a separate gstreamer-java-swt.jar for 
all swt related class since for those who don't use swt the base 
gstreamer-java.jar (without swt) should have to be enough.

Original comment by vitorl...@gmail.com on 25 Oct 2010 at 10:31

[GoogleCodeExporter]

i close this issue and open a new discussion one the list.

Original comment by lfar...@gmail.com on 25 Oct 2010 at 1:50

• Changed state: Fixed