Audio function setOutputDevice not working properly

[EroticDevelopment]

The setOutputDevice function is not working properly in multiple ways. These issues are all present in the vlcj-player package, and are not present in the official VLC client.

• If a device is set before playback begins, the device setting is lost as soon as playback starts
• If a device is set immediately after playback begins, the setting never takes affect
• If a device is set a few seconds into playback, it works correctly
• If playback is stopped and restarted, the device setting is again lost

After several attempts, I can't find a single way to make a device chosen before playback begins stick. I also can't find a way to make a device chosen during playback stick when a new file is played.

While not in my example code, if a MediaPlayerEventAdapter class is bound to the player the audioDeviceChanged function is called when playback is initially started, and again when the file is restarted. In both cases the audioDevice argument is an empty string and the device is subsequently reset. Attempting to put it back from this function doesn't work. It does work to reset it in the mediaPlayerReady function, however by the time this is called the audio has already been playing for a few hundred ms, and the device change itself takes approximately 2 seconds, so ~2.5 seconds of audio is missing using this workaround.

If there is any additional information I can provide, please let me know.

Example class to demonstrate issue

```java import uk.co.caprica.vlcj.player.base.AudioDevice; import uk.co.caprica.vlcj.player.component.AudioPlayerComponent; import java.util.List; import static java.lang.Thread.sleep; public class MediaTest { public static void main(String[] args) throws InterruptedException { AudioPlayerComponent mpc = new AudioPlayerComponent(); System.out.println("Initial Device: " + mpc.mediaPlayer().audio().outputDevice()); List devices = mpc.mediaPlayer().audio().outputDevices(); for (AudioDevice device : devices) { System.out.println("Available: " + device.getDeviceId()); } // This is the device I want audio output on, pick anything you want (except default...) String deviceToSet = devices.get(5).getDeviceId(); // This won't work System.out.println("\nSetting: " + deviceToSet); mpc.mediaPlayer().audio().setOutputDevice(null, deviceToSet); System.out.println("Updated Device (before playback started): " + mpc.mediaPlayer().audio().outputDevice()); // Start media mpc.mediaPlayer().media().start("test.mp4"); System.out.println("Device immediately after playback starts: " + mpc.mediaPlayer().audio().outputDevice()); // Wait a few seconds sleep(3000); // This one works System.out.println("\nDevice after 3s: " + mpc.mediaPlayer().audio().outputDevice()); System.out.println("Setting: " + deviceToSet); mpc.mediaPlayer().audio().setOutputDevice(null, deviceToSet); System.out.println("Updated Device: " + mpc.mediaPlayer().audio().outputDevice()); // Wait a few seconds. Audio is correct here sleep(5000); // Now, stop and re-start media System.out.println("\nRestarting..."); mpc.mediaPlayer().controls().stop(); mpc.mediaPlayer().media().start("test.mp4"); System.out.println("Device after restart: " + mpc.mediaPlayer().audio().outputDevice()); // Audio is back to the "Default" device... // Setting it here won't work either, the media has to play for a while before you can change the output device for some reason System.out.println("Setting after restart: " + deviceToSet); mpc.mediaPlayer().audio().setOutputDevice(null, deviceToSet); System.out.println("Device after immediately setting: " + mpc.mediaPlayer().audio().outputDevice()); // Wait a few seconds sleep(3000); // This one works again System.out.println("\nDevice after 3s: " + mpc.mediaPlayer().audio().outputDevice()); System.out.println("Setting: " + deviceToSet); mpc.mediaPlayer().audio().setOutputDevice(null, deviceToSet); System.out.println("Updated Device: " + mpc.mediaPlayer().audio().outputDevice()); // Wait a few seconds. Audio is correct here again sleep(5000); } } ```

VLCJ Version 4.7.1 Windows 10, 21H2, Build 19044.1566 openjdk version "11.0.11" 2021-04-20 LTS OpenJDK Runtime Environment Corretto-11.0.11.9.1 (build 11.0.11+9-LTS) OpenJDK 64-Bit Server VM Corretto-11.0.11.9.1 (build 11.0.11+9-LTS, mixed mode)

[caprica]

In this case, vlcj is not really doing anything other than calling the underlying native LibVLC API.

If you stripped out all of the vlcj code and instead invoked the LibVLC API methods directly, you'd see the same behaviour.

There is some API that unfortunately either operates asynchronously, or has no effect until media playback starts.

The official VLC client does not itself use LibVLC.

This might be something that will be improved with the upcoming LibVLC 4.x as the media player code there has changed a lot.

You can maybe try something like start-paused, set device, play, but I do not know if that will help.

[caprica]

(Issue prematurely closed by mis-click).

[caprica]

A mitigation might be to see if you can use the elementary stream selected event instead of media player ready.

[EroticDevelopment]

Thanks for the additional information and potential mitigation. I'm going to test that next and see if that changes anything.

The main reason I opened this issue was because this behavior appears to have changed between vlcj 3.x and 4.x. In the 3.x versions (tested with 3.12.1), this works as I would expect and you can set an audio output device prior to starting playback. It does get cleared when calling stop(), but that can be avoided by using pause() instead so it wasn't a huge deal. On 4.x, no such workaround seems to exist that I've found.

Was this change between 3.x and 4.x just caused by changes in the upstream LibVLC? If so, and since others here likely have more experience developing against LibVLC directly, does this seem like something worthy of a bug report to LibVLC? I know there may not be anything possible here if this is a LibVLC limitation, but it really seems like setting an output device prior to starting playback would be a fairly basic and desirable functionality.

[EroticDevelopment]

After some additional testing, the elementaryStreamSelected event is raised but attempting to set the audio output from there does not work, directly or by submitting a Runnable.

startPaused does not seem to help either, nor does start followed by a pause.

Another odd thing I noticed, after calling component.mediaPlayer().controls().stop(), the component.mediaPlayer().audio().outputDevices() function returns a completely empty list. Prior to starting playback, and during playback, this list is populated.

I'm going to try the 5.0.0-SNAPSHOT version next and see if there are any differences.

[caprica]

Yes, the upstream LibVLC has changed a lot from 3.x to 4.x with, from what I can tell, a complete re-implementation of some things like the media player.

This issue is related, but not exactly on-topic: https://code.videolan.org/videolan/vlc/-/issues/26440

From that you can see that previously there was API on the libvlc instance itself to get the audio devices - but now, that has been removed and the only audio output device API is associated with the media player.

Would the VLC developers consider it a bug in LibVLC or just how it works now, I don't know. Also VLC 4.x is not finished yet, probably still a long way from being finished, so maybe this is planned to be address anyway. I can maybe post a question in the LibVLC discord and see if other native library bindings developers have any comment on this.

But anyway, what I can say, this is not something that can be fixed vlcj.

If you wrote a C program calling the same API methods that vlcj does, in the same sequence or at the same time you're doing now, it will operate in the same way.

Switching to the vlcj-5.0.0-SNAPSHOT jar won't help you, that dependency is a reasonably well up-to-date (like last week or something) snapshot of the master branch here, and this issue is not on the Java side.

For me, using vlcj-5.x and VLC 4.x, while pre-release works perfectly for what I need it to, but to be on the "safe" side people generally should stick with vlcj-4.x and VLC 3.x until VLC 4.x is actually released.

[caprica]

This actually works for me with the vlcj-5.0.0-SNAPSHOT and the latest VLC 4.x built from source.

This is the code (note that you will not be able to use this version of the code yourself, because it relies on some non-public release of vlcj bindings, I will try and convert this to a current-style binding at some point):

public class DevTest {

    public static void main(String[] args) throws Exception {
        // Remember there is no vlcj native discovery here, you may need to set jna.library.path and VLC_PLUGIN_PATH manually

        libvlc_instance_t libvlc = libvlc_new();
        libvlc_media_player_t mediaPlayer = libvlc_media_player_new(libvlc);
        libvlc_event_manager_t eventManager = libvlc_media_player_event_manager(mediaPlayer);

        libvlc_event_cb eventCallback = libvlc_event_cb.eventCallbackFor((event, userData) -> {
            System.out.printf("MediaPlayerAudioDevice: %s%n", event.u.media_player_audio_device().device);
        });
        libvlc_event_attach(eventManager, libvlc_event_e.libvlc_MediaPlayerAudioDevice, eventCallback, null);

        libvlc_media_t media = libvlc_media_new_path(libvlc, "some-movie.mp4");
        libvlc_media_player_set_media(mediaPlayer, media);

        libvlc_audio_output_device_t device = libvlc_audio_output_device_enum(mediaPlayer);
        while (device != null) {
            System.out.printf("device enum: %s (%s)%n", device.psz_device, device.psz_description);
            device = device.p_next;
        }

        String wantedDevice = "alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo";

        System.out.println("Before play...");

        System.out.printf("default audio device is %s%n", libvlc_audio_output_device_get(mediaPlayer));
        libvlc_audio_output_device_set(mediaPlayer, wantedDevice);
        System.out.printf("after set audio device is %s, should be %s%n", libvlc_audio_output_device_get(mediaPlayer), wantedDevice);

        System.out.println("Playing...");

        libvlc_media_player_play(mediaPlayer);

        Thread.sleep(10000);

        libvlc_event_detach(eventManager, libvlc_event_e.libvlc_MediaPlayerAudioDevice, eventCallback, null);
        libvlc_media_release(media);
        libvlc_media_player_release(mediaPlayer);
        libvlc_release(libvlc);
    }
}

This is the test output:

device enum: alsa_output.pci-0000_01_00.1.hdmi-stereo-extra1 (GP107GL High Definition Audio Controller Digital Stereo (HDMI 2))
device enum: alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo (SteelSeries Arctis 7 Analogue Stereo)
device enum: alsa_output.pci-0000_00_1f.3.iec958-stereo (Built-in Audio Digital Stereo (IEC958))
Before play...
default audio device is null
MediaPlayerAudioDevice: alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo
after set audio device is alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo, should be alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo
Playing...
[00007f1db40f2390] vdpau_instance decoder device: Using NVIDIA VDPAU Driver Shared Library  390.144  Wed Jun  2 22:57:50 UTC 2021

Process finished with exit code 0

You can see from the output:

• get audio device returns null initially
• after calling set audio device you see the audio device event callback first with the correct device
• then you see the result of the get returning the correct device
• then the media starts playing

So there is at least some encouragement here that this will work.

(I did not try with vlcj 4.x and LibVLC 3.x, as my entire focus right now is vlcj 5.x and LibVLC 4.x, and switching tack at this point is not convenient).

[caprica]

Here's a version of the previous test code works with vlcj-5.0.0-SNAPSHOT and LibVLC 4.x pre-release.

It might work with vlcj-4.x and LibVLC 3.x, I'm not sure.

It should match what vlcj does behind the scenes.

package uk.co.caprica.vlcj;

import uk.co.caprica.vlcj.binding.NativeString;
import uk.co.caprica.vlcj.binding.internal.*;
import uk.co.caprica.vlcj.factory.MediaPlayerFactory;

import java.util.concurrent.CountDownLatch;

import static uk.co.caprica.vlcj.binding.LibVlc.*;

public class DevTest {

    public static void main(String[] args) throws Exception {
        new MediaPlayerFactory(); // Just to handle the native library discovery

        CountDownLatch latch = new CountDownLatch(1);

        libvlc_instance_t libvlc = libvlc_new(0, null);
        libvlc_media_player_t mediaPlayer = libvlc_media_player_new(libvlc);
        libvlc_event_manager_t eventManager = libvlc_media_player_event_manager(mediaPlayer);

        libvlc_callback_t eventCallback = (event, userData) -> {
            if (event.type == libvlc_event_e.libvlc_MediaPlayerAudioDevice.intValue()) {
                event.u.setType(media_player_audio_device.class);
                event.read();
                System.out.printf("MediaPlayerAudioDevice: %s%n", event.u.media_player_audio_device.device);
                return;
            }
            if (event.type == libvlc_event_e.libvlc_MediaPlayerStopped.intValue()) {
                System.out.printf("Stopped: %s%n", NativeString.copyNativeString(libvlc_audio_output_device_get(mediaPlayer)));
                latch.countDown();
                return;
            }
        };
        libvlc_event_attach(eventManager, libvlc_event_e.libvlc_MediaPlayerAudioDevice.intValue(), eventCallback, null);
        libvlc_event_attach(eventManager, libvlc_event_e.libvlc_MediaPlayerStopped.intValue(), eventCallback, null);

        libvlc_media_t media = libvlc_media_new_path(libvlc, "some-movie.mp4");
        libvlc_media_player_set_media(mediaPlayer, media);

        libvlc_audio_output_device_t device = libvlc_audio_output_device_enum(mediaPlayer);
        while (device != null) {
            System.out.printf("device enum: %s (%s)%n", NativeString.copyNativeString(device.psz_device), NativeString.copyNativeString(device.psz_description));
            device = device.p_next;
        }

        String wantedDevice = "alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo";

        System.out.println("Before play...");

        System.out.printf("default audio device is %s%n", NativeString.copyNativeString(libvlc_audio_output_device_get(mediaPlayer)));
        libvlc_audio_output_device_set(mediaPlayer, wantedDevice);
        System.out.printf("after set audio device is %s, should be %s%n", NativeString.copyNativeString(libvlc_audio_output_device_get(mediaPlayer)), wantedDevice);

        System.out.println("Playing...");

        libvlc_media_player_play(mediaPlayer);

        Thread.sleep(5000);

        libvlc_media_player_set_position(mediaPlayer, 0.98f, 1);

        latch.await();

        System.out.println("After the media player stopped");
        System.out.printf("audio device is %s%n", NativeString.copyNativeString(libvlc_audio_output_device_get(mediaPlayer)));

        libvlc_event_detach(eventManager, libvlc_event_e.libvlc_MediaPlayerAudioDevice.intValue(), eventCallback, null);
        libvlc_event_detach(eventManager, libvlc_event_e.libvlc_MediaPlayerStopped.intValue(), eventCallback, null);
        libvlc_media_release(media);
        libvlc_media_player_release(mediaPlayer);
        libvlc_release(libvlc);
    }
}

The output is:

device enum: alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo (SteelSeries Arctis 7 Analogue Stereo)
device enum: alsa_output.pci-0000_00_1f.3.iec958-stereo (Built-in Audio Digital Stereo (IEC958))
device enum: alsa_output.pci-0000_01_00.1.hdmi-stereo-extra1 (GP107GL High Definition Audio Controller Digital Stereo (HDMI 2))
Before play...
default audio device is null
MediaPlayerAudioDevice: alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo
after set audio device is alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo, should be alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo
Playing...
[00007f17740f2d50] vdpau_instance decoder device: Using NVIDIA VDPAU Driver Shared Library  390.144  Wed Jun  2 22:57:50 UTC 2021
Stopped: alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo
After the media player stopped
audio device is alsa_output.usb-SteelSeries_SteelSeries_Arctis_7-00.analog-stereo

From this output, it looks like it works correctly.

If you could try that and report your results, it would be interesting.

[caprica]

I also just tested this using the vlcj bindings rather than the raw code I posted above, and it is working as expected for me.

The audio device can be set before the media player starts, and the audio device changed event reports the expected device name.

This is on Linux of course, it's possible there's some issue on Windows.

[caprica]

Not sure there's anything more can be usefully said on this issue, it can be re-opened if anything new comes up.