Feature: improved audio subsystem

[GoogleCodeExporter]

By implementing support for GStreamer [1] via gstreamer-java[2], we could 
a) add support for more audio formats,
b) through its graph design, eventually add support for time scale modification

[1] http://gstreamer.freedesktop.org/
[2] http://code.google.com/p/gstreamer-java/

Original issue reported on code.google.com by markus.g...@gmail.com on 18 Feb 2009 at 9:27

[GoogleCodeExporter]

can u please provide your suggestions about how to go about in implementing 
gstreamer
in emerson? can a player of gstreamer be built into emerson and should we 
replace the
current javazoom as mp3 support can be provided by gstreamer also. 

Original comment by chaturve...@gmail.com on 9 Apr 2009 at 6:04

[GoogleCodeExporter]

Providers of audio rendering services can coexist as separate bundles. They are 
all
required to provide implementations of 
org.daisy.reader.model.audio.IAudioKitFactory
(in bundle: org.daisy.reader.model), defined by the
org.daisy.reader.model.audiokitfactory extension point in plugin.xml of bundle:
org.daisy.reader.model. It is through this extension point that
com.javazoom.javalayer.kit registers itself as a provider of the 
IAudioKitfactory
service.

This means that the current com.javazoom.javalayer.kit bundle can coexist with a
gstreamer bundle. The behavior of 
org.daisy.reader.model.audio.AudioKitFactoryFactory
when several implementations register that they support for example MP3 is 
currently
unspecified; what likely will happen is that the first service provider that 
returns
true when queried on MP3 support is the one that will be returned. 

That would be an interim scenario though: if gstreamer and gstreamer-java 
proves to
work well on all platforms, the javazoom bundle could eventually be disabled.

Hope this helps.

Original comment by markus.g...@gmail.com on 13 Apr 2009 at 6:27

[GoogleCodeExporter]

After understanding javazoom.javalayer.kit bundle, we understood that it is
registering javazoom player as the service provider for audio support. It can 
coexist
with gstreamer bundle but we need to create separate bundle for registering 
gstreamer
as the service provider. Does that mean that we create gstreamer kit bundle for 
this
functionality. Please provide your suggestions.

Original comment by chaturve...@gmail.com on 29 Apr 2009 at 6:46

[GoogleCodeExporter]

Yes, new separate bundles would need to be added.

1) A bundle with the gstreamer library itself. This would basically be a 
wrapper that
turns the gstreamer library into an OSGi bundle. It would be equivalent to the
current com.javazoom.javalayer bundle. Note - I am using source mode in the 
javazoom
case since I had to make some tweaks to the code, but a bundle of this type 
typically
contains a third party jar (the gstreamer jar in this case), and then just OSGi
metadata (in /META-INF/MANIFEST.MF). There is an example of this approach in the
com.ctc.wstx bundle. No code should be needed - just metadata.

The name of this bundle would likely be org.gstreamer.gstreamer-java.

2) A bundle that provides an implementation of the
org.daisy.reader.model.audiokitfactory extension point. It would be equivalent 
to the
current com.javazoom.javalayer.kit. This could declare a hard dependency on the
bundle in 1 above ('Require-bundle' field in MANIFEST.MF). By implementing
org.daisy.reader.model.audiokitfactory (and noting that in its plugin.xml) the
AudioKitFactoryFactory will automatically find it (assuming that the bundle 
also has
been added to the Run Configuration).

The name of this bundle would typically be org.gstreamer.gstreamer-java.kit

To make your kit bundle respond positively to the call from 
AudioKitFactoryFactory,
you will need to provide returns from your implementation of
IAudioKitFactory#supportsContentType(URL). In order to get javazoom to stop 
getting
in the way regarding MP3 playback, you can either temporarily tweak its
implementation of the same method to always return no, or simply remove 
javazoom from
the run confguration.

Original comment by markus.g...@gmail.com on 29 Apr 2009 at 8:58

[GoogleCodeExporter]

I have created the said bundles namely org.gstreamer.gstreamer-java.jar and
org.gstreamer.gstreamerjava.jar. After  I  integrated the bundles while running
Eclipse through Run Configurations...., then it is throwing the following error

org.daisy.reader.model.audio.AudioException: No such Gstreamer factory: fakesrc

    at org.daisy.reader.model.audio.AudioClipPlayer.run(AudioClipPlayer.java:38)
Caused by: java.lang.IllegalArgumentException: No such Gstreamer factory: 
fakesrc
    at org.gstreamer.ElementFactory.makeRawElement(ElementFactory.java:172)
    at org.gstreamer.ElementFactory.make(ElementFactory.java:164)

This error is thrown by the ElementFactory.java file of 
org.gstreamer.gstreamer-java
when it is not able to create the element. I am not able to debug it properly.

Please provide suggestion.

Original comment by chaturve...@gmail.com on 20 May 2009 at 10:42

[GoogleCodeExporter]

It looks like the error is internal to the gstreamer code, or am I missing 
something?

Did you check or ask at the gstreamer discussion group? [1]

(Btw, I saw another person is working on an OSGi bundle there [2].)

Regarding debugging: you can associate source code with a binary osgi jar in 
Eclipse;
I dont have the IDE here so cant tell you the details right now. After that
association, you should be able to step through the gstreamer internals in 
source mode.

[1] http://groups.google.com/group/gstreamer-java 
[2]
http://groups.google.com/group/gstreamer-java/browse_thread/thread/281d2b5773a10
863/2c8d4d71ebd63cbe?lnk=gst&q=osgi#2c8d4d71ebd63cbe

Original comment by markus.g...@gmail.com on 20 May 2009 at 10:19

[GoogleCodeExporter]

I have created the two GStreamer bundles and integrated them with Emerson. 
Emerson is successfully integrated with GStreamer. It is playing the audio file 
using
GStreamer support. But it has certain issues.

It is not navigating the daisy dtb. I need to manually navigate the text of the 
DAISY
book, to play the next audio dtb. Even after manually navigating it starts 
playing
the next audio file but does not stops the audio which it was playing earlier, 
i.e.
it starts playing many audio files.

It is not even highlighting the text, of whose audio it is presently playing. 

I have tried to debug it but it is of no use, neither it is showing any errors.

Please suggest how can I go about fixing these issues.

Original comment by chaturve...@gmail.com on 29 May 2009 at 5:24

[GoogleCodeExporter]

Congratulations! This seems like a good start. 

Regarding your problems, I would start looking in your implementation of
AudioClipFeeder. 

- Does it register the AudioClipCursor correctly at initialization time (compare
JLayerMP3ClipFeeder line 19)

- Does it override the close method, and if so does it set the isDisposed 
property to
true in the close() method? (compare AudioClipFeeder line 44)

Regarding synchronization problems, the design is actually quite simple. All GUI
components (such as the text display) that are interested in knowing about a 
reading
position register themselves with the ModelManager with the 
IPositionChangeListener
interface (see for example BrowserView line 81). The Model in its turn, in the 
case
of audio DTBs, listens to the AudioClipCursor to determine when the position 
changes,
and then notifies all listeners (see AudioClipCursor line 104). It seems its 
this
position change event notification that is not working somehow. 

Feel free to contact me at my gmail address for further discussion. 

Original comment by markus.g...@gmail.com on 29 May 2009 at 10:53

[GoogleCodeExporter]

Hi Markus
I have tried your suggestions,  
-The cursor is initialzed properly
-Also the isDisposed property is set to true

The issue regarding playing of the audio file is resolved, also Emerson now 
supports 
the different audio file formats. I'm also about to complete the volume control 
and 
playback speed enhancement suggested by you.

But, the daisy book is still not navigating.I have also mailed you regarding 
these 
this, but I think it didn't reached you.

Please guide me in how Emerson starts playing the audio file of the next entry 
by 
it's own, as this is not happening using GStreamer.

Thanks

Original comment by chaturve...@gmail.com on 2 Aug 2009 at 2:54

[GoogleCodeExporter]

Congratulations on your progress so far. 

Is my understanding correct that
- sequential playback works (in other words, rendering of several SMIL audio 
elements
that occurs in sequence immediately after opening a book)
- but when explicitly navigating (eg User invoked change of cursor position) the
repositioning does not happen/the playback does not start?

Here is the logic for User invoked navigation:

- The request to relocate is registered by a call to 
Model.setPosition(IPosition).
(See org.daisy.emerson.ui.navigator.handlers for examples of how this is 
invoked from
the ncx/navigation view)

- The default implementation of Model.setPosition appears in 
DtbModel.setPosition. 

Basically, what happens here is that the AudioClipCursor gets repositioned:

spine.getAudioClipCursor().set(targetPhrase);

... followed by a call to  DtbModel.render(AudioMediaObject phrase, Long 
timeOffset) 

I guess it is in this place your debugging needs to start (line 188 of 
DtbModel). Is
the method called with proper parameters? (It should be, since everything that
happens prior to this is not dependent on your changes). Are the feeder.start() 
and
player.start() calls successful? 

Another thing you can do at this point is to add a catch-all clause around the
feeder.start and player.start method calls inside DtbModel.render. If nothing is
thrown here, you need to go into the feeder and player instances to make sure 
things
are set up correctly.  As you can see in DtbModel.render, one thing that 
happens is
that the player and feeder intances are recreated each time render() is called. 
(This
comes as a consequence of SMIL logic.) It would seem that this recreation of 
player
and/or feeder instances is what fails.

Original comment by markus.g...@gmail.com on 4 Aug 2009 at 11:58