Capture (loopback) desktop audio

nurdism commented 2 years ago

Summary

I'd like to be able to capture desktop audio (or an output device)

Motivation

The idea (for me) would to be able to record and stream the desktops audio/video and do a screen cast with that.

Describe alternatives you've considered

The other solution would to find (or create) another lib to capture desktop audio

nurdism commented 2 years ago

it looks like this could be done with https://github.com/gen2brain/malgo (used in the microphone driver) I was doing my own research on this and this https://github.com/gen2brain/malgo/issues/34 lead me to https://github.com/mackron/miniaudio/blob/master/examples/simple_loopback.c I am not competent enough in c to understand any of that. But anyone else looking into this feature, this might save you some time.

nurdism commented 2 years ago

After further reading, it seems miniaudio only supports loopback via WASAPI (windows) but for Linux/Unix systems you could look into https://github.com/gordonklaus/portaudio or https://github.com/sqp/pulseaudio

miniaudio supports the pulse audio backend so this should be possible

ZeoWorks commented 2 years ago

Did you ever get desktop audio working?

nurdism commented 2 years ago

Did you ever get desktop audio working?

I have not, I need to give it another shot. But have been unable to dedicate the time to figuring it out.

Meonardo commented 2 years ago

Hi @nurdism @ZeoWorks , I have managed to make capture the speaker sound working on Windows, https://github.com/Meonardo/mediadevices/blob/master/pkg/driver/speaker/speaker.go

func getAudioTrack() (*mediadevices.AudioTrack, error) {
        // Currently I use the default speaker from the system, so the device filter will not work!
    // var deviceID = ""
    devices := mediadevices.EnumerateDevices()

    for _, device := range devices {
        if device.Kind == mediadevices.AudioOutput {
            log.Printf("Found Speaker: %s", device.Name)
            //deviceID = device.DeviceID
                        //break
        }
    }

    s, err := mediadevices.GetUserMedia(mediadevices.MediaStreamConstraints{
        Audio: func(c *mediadevices.MediaTrackConstraints) {
            //c.DeviceID = prop.String(deviceID)
        },
        Codec: element.audioCodecSelector,
    })

    if err != nil {
        log.Println("Audio track create failed", err)
        return nil, err
    } else {
        audioTrack := s.GetAudioTracks()[0].(*mediadevices.AudioTrack)
        return audioTrack, err
    }
}

Notice: I also made some changes of the malgo repo in my fork. Hope this will help you.

pion / mediadevices