Support for audio loopback in Windows?

[janleskovec]

I am making a cross-platform audio-based application and I need to record the audio output of the speakers. I have successfully done this in linux by using the exclude_monitors=False option. More details on the topic: https://stackoverflow.com/questions/26573556/record-speakers-output-with-pyaudio

Can this be done in your library? If not do you plan to support this in the future?

[bastibe]

No, this is not supported by the underlying API.

But I have used this with some success in the past: https://www.vb-audio.com/Cable/index.htm

[janleskovec]

Doesn't WASAPI support this? I mean I've even used this kind of loopback recording when I was first trying out the feasibility of my idea (I used NAudio: https://github.com/naudio/NAudio).

[bastibe]

You seem to be right! I didn't know that!

https://docs.microsoft.com/en-us/windows/desktop/CoreAudio/loopback-recording

This is definitely something I would want to include in SoundCard. But it will probably take a few months until I get around to doing it. Would you like to try yourself, and create a pull request?

[janleskovec]

I will definitely take a look at it and try if I can do it myself and later create a pull request. I will follow up on this issue If I'll be able to do it.

[bastibe]

Cool! Let me know if you need help!

[janleskovec]

I got it working already!!! I just added a parameter (isloopback) to the constructor of the _AudioClient class that affects the streamflags. I also added two functions (all_loopback_devices() and default_loopback_device()) that return the speaker devices as _Microphone. I tested my code with:

import soundcard

import time
import numpy

length = 5 #seconds
mysamplerate=48000

input = soundcard.default_loopback_device()

output = soundcard.default_speaker()

print("----------------------\n Devices:")
print("input: " + str(input))
print("output: " + str(output))

print("\n Recording...")
data = input.record(samplerate=mysamplerate, numframes=(mysamplerate*length))
print("\n Wait")
time.sleep(length)
print("\n Playing...")
output.play(data/numpy.max(data), samplerate=mysamplerate)
print("\n Done\n ----------------------")

Question 1: To create a pull request I create a fork , commit, push and then create a pull request or...? (I don't have much experience with GitHub outside the online version)

Question 2: Is there a way for me to record and analyse the audio stream live? Maybe a callback kind of system or...?

EDIT: Nevermind question 1, I already created a pull request.

[janleskovec]

Also: The next step would probably be to also implement the get_loopback_device(id) method?

[bastibe]

Very cool! Thank you very much!

Would it be possible to conform to the same function signature as in the Linux case, i.e. exclude_monitors in all_microphones?

Or maybe that is just not a good idea. It might be better to add two functions loopbacks, and default_loopback to the existing speakers/default_speaker and microphones/default_microphone.

What do you think?

[janleskovec]

I was thinking in this direction, however it would be a little counter-intuitive due to the naming being a bit backwards, because "loopback devices" are actually speakers and not microphones. Additionally I think it wouldn't make it simpler, because you would have to differentiate between platforms in the application code.

I think it isn't that bad to just have separate functions for windows-specific functionality. However I think that the optimal solution would be to isolate the "loopback devices" and "monitors" in a different class (much like _Microphone and _Speaker, there would be a separate class named _Loopback). This might complicate the code a bit, however it would result in much better consistency across all platforms (not sure if macOS supports this kind of recording). A solution to prevent having too much code would be to have the theoretical _Loopback class inherit from the _Microphone class and just change the constructor to enable loopback recording (at least in case of the Windows implementation).

tl;dr: The best solution would be to implement a new _Loopback class that inherits from _Microphone and implement the according default_loopback and all_loopbacks methods.

[bastibe]

"loopback devices" are actually speakers and not microphones

I can see that this is confusing, since loopback devices are both, speakers and microphones. From the point of view of SoundCard, however, they are clearly microphones, since you use them to record audio data. Or do we have a misunderstanding here?

I think it isn't that bad to just have separate functions for windows-specific functionality.

I do though, and strongly so. The maintenance burden of maintaining several different APIs is horrendous, and not something I want to do. The only reason I included exclude_monitors at all, is because it was so tremendously useful for testing.

The problem is, however, that as far as I can tell, Core Audio on macOS does not support loopback devices at all. Because of that, and the above cross-platform compatibility, I don't want to add all_loopbacks/default_loopback after all.

I must conclude that exclude_monitors is currently the best way to go. The name is terrible, though. Maybe include_loopbacks would be much better? "Monitors" is the pulseaudio terminology, but terminology was never pulse's strong suit.

Could you change your pull request to include the loopback devices in all_microphones with the exclude_monitors or include_loopbacks flag? It might require appending "Loopback" to the device name to differentiate them from the microphones.

[janleskovec]

Ok, will do. I also think it would be a good Idea to make the naming of the flag at least consistent across Windows and Linux. What should I rename the flag to? include_speakers, include_loopback?

Also: I think it would be a good idea to also add the flag to the macOS code and just alert the user that such a thing is not supported in case someone tries to run the same code on macOS and to provide consistency across all three platforms. What do you think?

[bastibe]

I think I'd go with include_loopback.

Good idea about the macOS code. We should probably raise a NotImplementedError if it is used.

[janleskovec]

Ok, everything is done. I also fixed the comments to be consistent with the new flag. If there is anything that should be changed before the merge, let me know.

[bastibe]

Wow, thank you very much! I created a code review, with a few small change requests.

[janleskovec]

Done. One last thing I noticed: if there is no audio data when recording on loopback, Windows doesn't even send any frames, therefore the application hangs, while on Linux you get a clicking sound, which is a minor inconsistency caused by the underlying APIs.

[janleskovec]

Another thought: maybe we shouldn't raise an error in the macOS code when using the include_loopback, but instead only throw a warning (like I did in the Linux code with the DeprecationWarning)?

[bastibe]

Great work! I like the isloopback property and simply calling it <Loopback ...> instead of <Microphone ..>.

And I agree with you, we don't need to raise an error, a warning would be enough.

Many many thanks for your contributions!

[janleskovec]

I needed this to be done so it wasn't really a problem for me to do this especially because you have already written such good code. After including this I think this could be one of the most feature-rich implementations of cross-platform audio in python.

I'll update the macOS code to warn instead of raising an error.

[janleskovec]

Just to let you know: I changed it from raise NotImplementedError to warnings.warn()

[bastibe]

Thank you very very much for your contribution, and our discussion!

By the way, I have long wanted to allow the creation of exclusive-mode streams on Windows. This would allow low-latency audio recording on Windows. As far as I can tell, all this would require is setting sharemode to AUDCLNT_SHAREMODE_EXCLUSIVE. After that, it should honor low block sizes.

However, I currently don't have a Windows machine to test this on. Would you be willing to implement and test this?

[janleskovec]

I'll do that as soon as I find the time.

[Thiago-marcondes]

Hello there i am looking your posts and i have a question: can you show a simple way to record the audio going to the speakers with soundcard lib? how can i put the speakers as a input signal device? Is it with include_loopback = True as param inside ? Thanks for any help!

[janleskovec]

When you call all_microphones and add the include_loopback flag, all speaker devices will be included in the returned list. Usualy the device at index 0 will be the default speaker.

[Thiago-marcondes]

Ok but how i assign this 0 ? as id number? If i have a script that plays wav files when i press a key and then i want to record in a audio file what i just played pressing that key. Can i send you my script? thank you very much for your help. I am searching for this the last 6 months....

[janleskovec]

all_microphones returns a list. You get the device with a simple "result[0]". Sure you can send the code.

[Thiago-marcondes]

Put the code here or email it? I am new to github sorry for the basic questions...

[janleskovec]

You can do it here.

[Thiago-marcondes]

Here is my script it seems that is not recording the sounds played.

import pygame from tkinter import * from tkinter import filedialog import matplotlib.pyplot as plt import numpy as np import soundcard as sc

pygame.mixer.init(frequency=44100, size=-16, channels=2, buffer=512) pygame.init()

def record_mic(): mics = sc.all_microphones(include_loopback=True) m1 = sc.default_microphone() data = m1.record(samplerate=48000, numframes=48000)

def key1(event):

global audio_file_name1
if audio_file_name1:  # play sound if just not an empty string
    sound1 = pygame.mixer.Sound(audio_file_name1)
    sound1.play()

Browser button 1

def browse1():

global audio_file_name1
audio_file_name1 = filedialog.askopenfilename(filetypes=(("Audio Files", ".wav .ogg"), ("All Files", "*.*")))

playing button 1

def playsound1():

global audio_file_name1
if audio_file_name1:  # play sound if just not an empty string
  sound1 = pygame.mixer.Sound(audio_file_name1)
  sound1.play()

2

keypress event 'a'

def key2(event):

global audio_file_name2
if audio_file_name2:  # play sound if just not an empty string
    sound2 = pygame.mixer.Sound(audio_file_name2)
    sound2.play()

Browser button 2

def browse2():

global audio_file_name2
audio_file_name2 = filedialog.askopenfilename(filetypes=(("Audio Files", ".wav .ogg"), ("All Files", "*.*")))

playing button 2

def playsound2():

we will also use the audio_file_name global variable

global audio_file_name2
if audio_file_name2:  # play sound if just not an empty string
    sound2 = pygame.mixer.Sound(audio_file_name2)
    sound2.play()

record sound from mic

root = Tk() frame = Frame(root) audio_file_name1 = ''

browse button 1

b1 = Button(root, text='open file', bg="yellow", command=browse1) # browser button 1 b1.pack(anchor=CENTER)

playing sound button 1

p1 = Button(root, text='Som1', command=playsound1) # playsound1

p1.pack(anchor=W)

browse button 2

b2 = Button(root, text='open file', bg="light green", command=browse2) # browser button 2 b2.pack(anchor=CENTER)

playing sound button 2

p2 = Button(root, text='Som2', command=playsound2) # playsound2

p2.pack(anchor=W)

recb = Button(root, text="REC", command=record_mic) # rec button recb.pack(anchor='s')

root.bind('d', key1) root.bind('a', key2)

root.mainloop()

[janleskovec]

m1 should be set to mics[0]. What you are doing now is just ignoring the result of the sc.all_microphones(include_loopback=True) call.

[Thiago-marcondes]

I did this

def record_mic(): mics = sc.all_microphones(include_loopback=True) m1 = mics[0] audio = m1.record(samplerate=44100, numframes=(44100))

But it is not recording. It takes a second and then i can play the sound pressing the d key but no recording.

[janleskovec]

In the README you have:

# alternatively, get a `Recorder` and `Player` object
# and play or record continuously:
with default_mic.recorder(samplerate=48000) as mic, \
      default_speaker.player(samplerate=48000) as sp:
    for _ in range(100):
        data = mic.record(numframes=1024)
        sp.play(data)

[Thiago-marcondes]

I tryied that too and did not work. It seems even with the 0 index that the recording is getting the mic input instead the speakers. When i print mics[0] it says that is loopback phones and speakers(2 channels) Maybe we need to discriminate between phones and speakers? I am using a lap top. thanks again for the help.

[bastibe]

Please write a short script that only does the recording. No graphical tkinter or pygame interaction. Then, we'll be able to help you.

We are doing this for free, in our spare time. Please be respectful of our time and work by being as helpful as possible, and by reading the documentation thoroughly. We are not here to help you with your homework, but to work out issues with our library. Thank you.