AudioSystem.mixerInfo returns 0 mixer (Java 1.3 Service compatibility)

[CeylonMigrationBot]

[@jpragey] When I run the following Ceylon code :

import java.lang { ObjectArray }
import javax.sound.sampled { AudioSystem, Mixer }
hared void run() {
    ObjectArray<Mixer.Info> mixers=AudioSystem.mixerInfo;
    print("Ceylon: ``mixers.size``");
}

module org.audiotest "1.0.0" {
    import java.base "7";
    import java.desktop "7";
}

I get 0 results; however when I run the following Java code (in the same ceylon project):

package org.audiotest;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.Mixer;

public class AudioTest {

    public AudioTest() {}

    public static void main(String[] args) {
        Mixer.Info[] mixers = AudioSystem.getMixerInfo();
        System.out.println("Mixers: " + mixers.length);
    }
}

I get 5 results.

I investigated it with eclipse debugger; it ended up in sun.misc.Service$LazyIterator.hasNext() loading "META-INF/services/javax.sound.sampled.spi.MixerProvider" by CeylonModuleClassLoader.getResources(), which returned an empty enumeration.

In pure java it is loaded by sun.misc.Launcher.AppClassLoader.getResources() returns 2 values. On my PC (linux 64 / openJDK 7) META-INF/services/javax.sound.sampled.spi.MixerProvider is in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/resources.jar.

So the PB is probably a more general Java 1.3 Service SPI and Ceylon class loading compatibility issue than a strictly java audio one.

Full project: https://drive.google.com/file/d/0B09FzhUz_CgbMkhRenpWQXdfQ2c/edit?usp=sharing

[Migrated from ceylon/ceylon-runtime#61]

[CeylonMigrationBot]

[@FroMage] @alesj: how can we make this jar visible to the JDK class loader? What's weird is that I was pretty sure that the JDK used the JDK class loader, so it should just be visible to itself.

[CeylonMigrationBot]

[@quintesse] I know that in JBoss Modules if you use their module.xml descriptor file you have to specifically import or export any services, by default it doesn't do any of that.

So even if the module can be loaded and the classes accessed you won't be able to read any of the service entries unless you import/export them. (The docs are very vague, I think "import" means available to the module itself, "export" means available to everyone).

I don't know how this affects the JDK packages though.

[CeylonMigrationBot]

[@AnandA777] I know that it isn't quite the same situation, but I found a case where someone was trying to get a Java Sound service provider set up as a JBoss module:

https://community.jboss.org/thread/200423?tstart=0&_sscc=t

Does any of the info there help?

[CeylonMigrationBot]

[@alesj] Afaik, we already handle services as they should be:

• https://github.com/ceylon/ceylon-runtime/blob/master/testsuite/src/test/java/org/jboss/ceylon/test/modules/services/test/ServicesTestCase.java

Imo, this here is just a case of proper visibility?

[CeylonMigrationBot]

[@quintesse] What do you mean by "proper visibility"?

[CeylonMigrationBot]

[@alesj]

What do you mean by "proper visibility"?

It depends on what the classloader where you wanna load the services "sees".

Since I'm saying, if all is done right, the services should be loaded; see the test.

[CeylonMigrationBot]

[@quintesse] Well yes, but in this case it's obvious something doesn't work, @jpragey does something very simple, that is supposed to work, using an ordinary Java class that outside of JBoss Modules works just fine. So something is wrong, right? It would be nice to figure out what exactly.

[CeylonMigrationBot]

[@alesj] Try adding a test to the existing ServicesTest, which would mimic what @jpragey does. And we can go from there.

[CeylonMigrationBot]

[@quintesse] @alesj test added

[CeylonMigrationBot]

[@alesj] @quintesse both cases return zero mixers ...

[CeylonMigrationBot]

[@quintesse] Do you have any audio mixers in your system? ;) The Java version should return something non-zero, otherwise there isn't anything for you to test...

[CeylonMigrationBot]

[@quintesse] The test will now also check for file types, to see if that returns something on your system. For me the plain Java version of the AudioSystem returns 5 mixers and 3 file types while the Ceylon runtime versions returns 0 in both cases. As long as your system has working audio you should also get a non-zero result. (PS: you should try some well-known Java app that has sound to see if sound is properly configured on your system for Java, especially on Linux or Mac it can be a problem with access rights)

[CeylonMigrationBot]

[@alesj] @dmlloyd any idea here?

[CeylonMigrationBot]

[@alesj] If I only run ServicesTestCase::testAudioMixerServices, I get

Number of mixers/filetypes using plain Java = 7/3 Command line: -mp /Users/alesj/projects/ceylon/ceylon-runtime/../ceylon-dist/dist/repo:/Users/alesj/projects/ceylon/ceylon-runtime/build/dist/repo ceylon.runtime:1.1.0 +executable ceylon.modules.jboss.runtime.JBossRuntime -rep /var/folders/x0/hk__mkqd78z752spn913k8sr0000gn/T ceylon.audiotest/1.0.0 Number of mixers/filetypes using Ceylon runtime = 7/3 Everything OK

So it looks like previous ServicesTestCase::testLoadServices, changes things somehow ...

[CeylonMigrationBot]

[@quintesse]

So it looks like previous ServicesTestCase::testLoadServices, changes things somehow

But that's in your (and @tombentley 's) case. In my case there's no difference when I run it one way or the other, in both cases I always get correct result in the plain Java version and always a failure in the JBoss Modules / Ceylon runtime version.

[CeylonMigrationBot]

[@FroMage] Note that JBoss Modules fucks up with some global JVM-scope services like for XML providers, which makes it very prone to errors when run in concurrent threads. Perhaps it does something else to other providers?

[CeylonMigrationBot]

[@FroMage] Concurrent threads as in two threads running each an instance of JBoss Modules (even in two separate class loaders).

[CeylonMigrationBot]

[@AnandA777] OK, I did some more looking around, and found a closer case to this one that was solved. Someone was trying to use the Java Sound API in JBoss, and was not seeing any supported file formats on their machine. The solution was:

Darran is right, it requires to edit the modules/sun/jdk/main/modules.xml and add an entry for com.sun.media.

Moreover, copy the following files to modules/sun/jdk/main/service-loader-resources/META-INF/services/ javax.sound.sampled.spi.AudioFileReader javax.sound.sampled.spi.AudioFileWriter javax.sound.sampled.spi.FormatConversionProvider javax.sound.sampled.spi.MixerProvider These files can be found inside /jre/lib/resources.jar of JDK, under /META-INF/services/

Source: https://community.jboss.org/thread/197517

There is a "modules" folder in my Ceylon project, but by default, there is no "sun" subfolder there. I could create a modules/sun/jdk/main path, add an appropriate modules.xml including com.sun.media, and copy the four named SPI files to a META-INF/services subfolder. Would that work? If so, what should I use as a starting point for the modules.xml file? If not, is this something that will have to be fixed or worked around in Ceylon? Thanks!

[CeylonMigrationBot]

[@FroMage] Moving to 1.2, unfortunately.

[CeylonMigrationBot]

[@quintesse] Argh dammit :(

[CeylonMigrationBot]

[@AnandA777] I did some further searching, and it looks like this JBoss Modules issue also affected WildFly, and that it was fixed in their latest release:

https://issues.jboss.org/browse/WFLY-768

I hope that this helps to fix the issue. Aside from that, if anyone can help with a workaround as per my previous post, please let me know.

[CeylonMigrationBot]

[@quintesse] That's nice! If only I had an idea how to apply it to our situation. Add that same list (or at least the ones needed to fix this issue, but I guess they spent some time in getting to that list and it will probably apply to us as well) to the ceylon-runtime module.xml? It would be worth a shot to see what happens.

[CeylonMigrationBot]

[@quintesse] The actual module.xml file they use for WildFly is this one: https://github.com/wildfly/wildfly/blob/84a3f370ce355fb1f390923055d25ad8e7a73eb3/build/src/main/resources/modules/system/layers/base/sun/jdk/main/module.xml

[CeylonMigrationBot]

[@quintesse] I tried applying it to our runtime and even jboss modules but that doesn't help any, so I'm afraid this is something that needs to happen at a much lower level. But I really have no idea. I was hoping @alesj could have some insights.

[jpragey]

Seems to get better with 1.2.0 and 1.2.1: I get mixers with flat classpath option (on shell with --flat-classpath, and in Eclipse with appropriate checkbox).

[FroMage]

So it's really a module visibility issue. We "just" have to find where…

[gavinking]

That was indeed a correct use of "just". :-)

[quintesse]

I think the solution mentioned in https://github.com/ceylon/ceylon/issues/4856#issuecomment-156666517 already pointed in that direction. Somehow JBoss Modules is denying access to some stuff that's available in the JDK resources.jar.

[xkr47]

Same problem occurs with MidiSystem.midiDeviceInfo (returns empty array) from package javax.sound.midi in module java.desktop.. Identical java version returns 4 midi devices.. Example code in initial comment of (unrelated) issue #5868. Also here flat classpath fixes issue.

[xkr47]

I should note that at least with Oracle JDK, it seems that even with no real MIDI devices attached there is always at least two devices returned by MidiSystem.getMidiDeviceInfo() on the Java side - a software sequencer and a software synthesizer. Therefore this could be a better unit test candidate than AudioSystem.getMixerInfo() which may well return an empty list in a system with no mixers and therefore not reveal the problem.

[xkr47]

Not knowing much anything about JBoss and JBoss modules from before, I dug in a bit..

It seems to boil down to usage of ClassLoader.getResources(path) where path is e.g. "META-INF/services/com.example.MyService" (in above cases).

In my current issue I'm trying to get log4j2 (maven) to work; I have manually added log4j-core as a dependency to log4j-api using overrides.xml:

    <artifact groupId="org.apache.logging.log4j" artifactId="log4j-api" version="2.8.2">
        <add groupId="org.apache.logging.log4j" artifactId="log4j-core" version="2.8.2"/>
    </artifact>

Code inside log4j-api calls classLoader.getResources("META-INF/log4j-provider.properties") to find implementations of the api.

ClassLoader[java]
  ConcurrentClassLoader[jboss]
    ModuleClassLoader[jboss]
      CeylonModuleClassLoader[ceylon]

Of these, ConcurrentClassLoader[jboss] implements getResources(String name); this just delegates to `findResources(name, false) when not requesting something in system paths (java/ and sun/reflect/).

ModuleClassLoader[jboss] implements findResources(String name, boolean exportsOnly) by delegating to module.getResources(name).

Module[jboss].getResources(String name) then retrieves a precomputed list of LocalLoaders for the given path ("META-INF"). This list basically contains:

• the CeylonModuleClassLoader for log4j-api
• the ModuleClassLoader for ceylon.language

loadResourceLocal(String name) is then called for each of these, which in both above cases is implemented by ModuleClassLoader[jboss] which calls getResource on the resource loader for the module e.g. JarFileResourceLoader.

So it never looks outside the two jar files of those two modules, when the wanted resource actually resides in the log4j-core module (which I added manually using overrides.xml, see above).

The Module[jboss] class seems to set up things using import and export filters for classes and resources, and I suspect these affect which class loaders end up on the precomputed list of LocalLoaders to traverse. I will continue investigation later..

[FroMage]

Great investigation!

Runtime linking happens in CeylonModuleLoader.java, can you check that log4j-api has the right dep to log4j-core added in findModule?

[xkr47]

At this point I have to admit I'm not really aware of how resources are supposed to be inherited between modules. The same way as classes? All resources of your immediate dependencies become available?

[FroMage]

I am not sure myself, but I suppose that yes.