Can you capture the sound of headphones ?

[sanpangzi]

Dear muaz, Thank you very much. Because by your help, I realized microphone and canvas recording. Now we are using in production, but our users have a new requirement that capture the sound of headphones. Do you have any suggestions? Looking forward to your reply, best wishes.

[muaz-khan]

You can capture all audios being played on a tab using tabCapture or desktopCapture.

desktopCapture additionally allows you capture all audios from your speakers.

For testing purpose, you can install this chrome extension and choose "Full Screen + System Audio" option to record entire screen a well as all audios from system speakers.

You can construct a new MediaStream object, that can be used to record only audios, and ignore all video tracks:

onlySpeakersAudio = new MediaStream();

desktopCaptureStream.getAudioTracks().forEach(function(track) {
    onlySpeakersAudio.addTrack(track);
});

recorder = RecordRTC(onlySpeakersAudio, {
    type: 'audio'
});

recorder.startRecording();

Here is the complete usage:

chrome.desktopCapture.chooseDesktopMedia(['window', 'screen', 'audio'], function(chromeMediaSourceId, opts) {
    if (!chromeMediaSourceId) {
        alert('User denied this action.');
        return;
    }

    var constraints = {
        audio: false,
        video: {
            mandatory: {
                chromeMediaSource: 'desktop',
                chromeMediaSourceId: chromeMediaSourceId
            },
            optional: []
        }
    };

    if (opts.canRequestAudioTrack === true) {
        constraints.audio = {
            mandatory: {
                chromeMediaSource: 'desktop',
                chromeMediaSourceId: chromeMediaSourceId,
                echoCancellation: true
            },
            optional: []
        };
    }

    navigator.webkitGetUserMedia(constraints, function(desktopCaptureStream) {
        var onlySpeakersAudio = new MediaStream();

        desktopCaptureStream.getAudioTracks().forEach(function(track) {
            onlySpeakersAudio.addTrack(track);
        });

        recorder = RecordRTC(onlySpeakersAudio, {
            type: 'audio'
        });

        recorder.startRecording();
    }, function() {});
});

[sanpangzi]

Thank you for your reply, Muaz, I will try in my project.

[sanpangzi]

Dear Muaz, I try your code. But I meet a error Cannot read property 'chooseDesktopMedia' of undefined. I search the answer from net. It seems that chrome.* APIs can only be used in extensions (extensions API) or apps (apps API). But I don't want to write a extension. The links is as follow: https://stackoverflow.com/questions/33061677/chrome-desktopcapture-choosedesktopmedia-can-only-be-used-in-extention . Can you help me ?

[muaz-khan]

You can either use

• https://github.com/muaz-khan/getScreenId

Or

• https://github.com/muaz-khan/Chrome-Extensions/tree/master/Screen-Capturing.js

i.e. your webpage will contact with chrome extension internally using postMessage methods. Your webpage will ask to capture full-screen including system audio.

[sanpangzi]

OK. Thanks for your quick reply. I will try your method.

[sanpangzi]

Dear Muaz, According to your method, I study your extension of desktopCapture and screen-recording. Thank you very much. I have gained a lot of knowledge. By the desktopCapture extension and getScreenId-master, I realized get system audio. I modified the following files: getScreenId-master/index.html ` ......

<script src="./RecordRTC.js"> </script>

........

getScreenId(function(error, sourceId, screen_constraints) { ..... navigator.getUserMedia = navigator.mozGetUserMedia || navigator.webkitGetUserMedia;

                    navigator.getUserMedia(screen_constraints, function(stream) {

                               navigator.mediaDevices.getUserMedia({
                                audio: true
                                }).then(function(microphone) {

                                    microphone.getAudioTracks().forEach(function(audioTrack) {
                                        stream.addTrack(audioTrack);
                                    });

                                    recorder = RecordRTC(stream, {
                                        type: 'video'
                                    });
                                    recorder.startRecording();
                                });  

                    }

...... `

[sanpangzi]

getScreenId-master/getScreenId.html ` .... function getScreenConstraints(error, sourceId) { var screen_constraints = {

        video: {
            mandatory: {
                chromeMediaSource: error ? 'screen' : 'desktop',
                maxWidth: window.screen.width > 1920 ? window.screen.width : 1920,
                maxHeight: window.screen.height > 1080 ? window.screen.height : 1080
            },
            optional: []
        }
    };

    if (sourceId) {
        screen_constraints.video.mandatory.chromeMediaSourceId = sourceId;

        screen_constraints.audio = {
            mandatory: {
                chromeMediaSource: 'desktop',
                chromeMediaSourceId: sourceId,
                echoCancellation: true
            },
            optional: []
        };
    }

    return screen_constraints;
}

..... desktopCapture/background-script.js ...... function onAccessApproved(sourceId,options) {

    // if "cancel" button is clicked
    if(!sourceId || !sourceId.length) {

        return port.postMessage('PermissionDeniedError');
    }

    // "ok" button is clicked; share "sourceId" with the
    // content-script which will forward it to the webpage
    if(options.canRequestAudioTrack===true){
        port.postMessage({
            sourceId: sourceId

        });
    }
}

...... ` That's all I have done. I can record the screen and audio.

[sanpangzi]

But I can't get the microphone's audio, I don't know the reason. I install your chrome extension and choose "Full Screen + System Audio" option to record entire screen and audios. When I play the file ,It still doesn't include the microphone's audio by your RecordRTC extension. Dear Muaz, can you give me some help? thank you very much.

[sanpangzi]

Muaz, It seems Microphone+Screen can realize Full Screen + System Audio+ Microphone Audio's recording in your RecordRTC extension, today I will study it.Thank you very much for your selfless contribution, Muaz.

[sanpangzi]

Dear Muaz, RecordRTC extension include five parts such as Full Screen + Systme Audio and so on. They Mixed together. I only need Microphone+ Screen. Can you help me to separate out into a single extension and include only Microphone+ Screen ? Thank you.

[muaz-khan]

Please use this instead of getScreenId:

• https://github.com/muaz-khan/Chrome-Extensions/tree/master/desktopCapture

manifest.json#L17 must link localhost or your own HTTPs domain.

Change this line and replace with:

var screenOptions = ['screen', 'window', 'audio'];

// or
var screenOptions = ['screen', 'audio'];

// or
var screenOptions = ['window', 'audio'];

Now you must link Screen-Capturing.js. Or CDN.

Now capture screen (full-screen+speakers), along with microphone:

getScreenConstraints(function(error, screen_constraints) {
    navigator.mediaDevices.getUserMedia({
        // video captures screen
        video: {
            mandatory: {
                chromeMediaSource: screen_constraints.mandatory.chromeMediaSource,
                chromeMediaSourceId: screen_constraints.mandatory.chromeMediaSourceId
            },
            optional: []
        },

        // audio is important here to capture speakers
        audio: {
            mandatory: {
                chromeMediaSource: screen_constraints.mandatory.chromeMediaSource,
                chromeMediaSourceId: screen_constraints.mandatory.chromeMediaSourceId
            },
            optional: []
        }
    }).then(function(screenPlusSpeakers) {
        navigator.mediaDevices.getUserMedia({
            audio: true // microphone
        }).then(function(microphone) {
            var finalStreamToBeRecorded = new MediaStream();

            // important: we must convert multiple audio tracks into single audio track
            var mixedAudioStream = getMixedAudioStreamXYZ(microphone, screenPlusSpeakers) || microphone;

            mixedAudioStream.getAudioTracks().forEach(function(audioTrack) {
                finalStreamToBeRecorded.addTrack(audioTrack);
            });

            screenPlusSpeakers.getVideoTracks().forEach(function(videoTrack) {
                finalStreamToBeRecorded.addTrack(videoTrack);
            });

            // "finalStreamToBeRecorded" will have
            // only one audio track
            // and only one video track
            recorder = RecordRTC(finalStreamToBeRecorded, {
                type: 'video'
            });
            recorder.startRecording();
        });
    });
});

function getMixedAudioStreamXYZ(arrayOfAudioStreams) {
    var audioContext = new AudioContext();

    var audioSources = [];

    var gainNode = audioContext.createGain();
    var gainNode.connect(audioContext.destination);
    var gainNode.gain.value = 0; // don't hear self

    var audioTracksLength = 0;
    arrayOfAudioStreams.forEach(function(stream) {
        if (!stream.getAudioTracks().length) {
            return;
        }

        audioTracksLength++;

        var audioSource = audioContext.createMediaStreamSource(stream);
        audioSource.connect(gainNode);
        audioSources.push(audioSource);
    });

    if (!audioTracksLength) {
        return;
    }

    audioDestination = audioContext.createMediaStreamDestination();
    audioSources.forEach(function(audioSource) {
        audioSource.connect(audioDestination);
    });
    return audioDestination.stream;
}

[sanpangzi]

Thank you very much, Muaz. I will try your method. Thanks again.

[sanpangzi]

Dear Muaz, I am sorry that I try your code. I meet a error is 'arrayOfAudioStreams.forEach is not a function'. I think it's wrong to go down here, but I don't know how to correct it. It is as follow in getScreenConstraints(function(error, screen_constraints):

var mixedAudioStream = getMixedAudioStreamXYZ(microphone, screenPlusSpeakers) || microphone;

I know I've brought you a lot of trouble, but I still need your help. Thank you very much.

[muaz-khan]

Here is the fix:

// pass array
var mixedAudioStream = getMixedAudioStreamXYZ([microphone, screenPlusSpeakers]) || microphone;

[sanpangzi]

Thank you very much, Muaz. By your help, I realized my project needs. I realized my project needs. Now I change your code and realized to record canvas + Speaker audio + microphone audio +Screen. The changes I made are as follows: in getScreenConstraints(function(error, screen_constraints) {........}), I change the code ......... ` then(function(screenPlusSpeakers) { navigator.mediaDevices.getUserMedia({ audio: true // microphone }).then(function(microphone) { var finalStreamToBeRecorded = new MediaStream();

        // important: we must convert multiple audio tracks into single audio track
        var mixedAudioStream = getMixedAudioStreamXYZ(microphone, screenPlusSpeakers) || microphone;

        mixedAudioStream.getAudioTracks().forEach(function(audioTrack) {
            finalStreamToBeRecorded.addTrack(audioTrack);
        });

        screenPlusSpeakers.getVideoTracks().forEach(function(videoTrack) {
            finalStreamToBeRecorded.addTrack(videoTrack);
        });

        // "finalStreamToBeRecorded" will have
        // only one audio track
        // and only one video track
        recorder = RecordRTC(finalStreamToBeRecorded, {
            type: 'video'
        });
        recorder.startRecording();
    });
});

` ..........

to ............ ` then(function(screenPlusSpeakers) { navigator.mediaDevices.getUserMedia({ audio: true // microphone }).then(function(microphone) {

                  //  var finalStreamToBeRecorded = new MediaStream();
                    var canvasStream = window.canvasElementToBeRecorded.captureStream();
                    var mixedAudioStream = getMixedAudioStreamXYZ([microphone, screenPlusSpeakers]) || microphone;

                    mixedAudioStream.getAudioTracks().forEach(function(audioTrack) {
                        canvasStream.addTrack(audioTrack);
                    });

                    screenPlusSpeakers.getVideoTracks().forEach(function(videoTrack) {
                        canvasStream.addTrack(videoTrack);
                    });

                    // now you can record "canvasStream" which include "microphone" tracks as well
                    recorder = RecordRTC(canvasStream, {
                        type: 'video'
                    });
                    recorder.startRecording();

                    });
                });

` ............

[sanpangzi]

Above, I change the file is /RecordRTC-master/Canvas-Recording/record-canvas-drawings.html. Now in fact, I don't need the screen, it only need canvas+ Speaker audio + microphone audio. Even though I realized what I needed, I had to choose once screen every time when I record. I don't want to choose the screen every time when I record it, because I don't need it, do you have a good idea?

[sanpangzi]

Dear Muaz, I know the choose screen is from the api " chrome.desktopCapture.chooseDesktopMedia", Muaz, Do you have a way to block it? Because even though I chose the screen, I'm still recording canvas.

[sanpangzi]

Dear Muaz, I upload some code to github include a your changed chrome extension and some record canvas code. I want to realize to record canvas+audio(speaker and microphone) without choose screen. If you have time, please help me to see. Thank you very much. Muaz.

[houcy]

Dear Muaz, I upload some code to github include a your changed chrome extension and some record canvas code. I want to realize to record canvas+audio(speaker and microphone) without choose screen. If you have time, please help me to see. Thank you very much. Muaz.

Hi Sanpangzi, Have you realized to record canvas+audio(speaker+microphone) without choose screen? Thanks!

[suhaskurade8]

Hi @muaz-khan, I have tried your code in my angular project. In video recording, in some cases, it happens that audio is not getting captured . The only video is captured. Can you please help me to solve this issue? Is it based on some microphone settings from a laptop. Do I need to do some additional settings on the laptop?

[saloni483]

Hi mauz, Can you help me in audio recording using headphones? Actually I am facing issue in audio recording while using headphones. Thanks in advance

[vishnupriyan-inbasekaran]

Hi @muaz-khan

Can you please let me know if speakers/headphone streams can be recorded without using the chrome-extension? If so, can you point me in that direction on how to implement that? Thank you.

[old-zoomer]

Hey @muaz-khan , Please take a look at this. Because it seems like this functionality is currently broken.

When I record screen + speakers. No audio is present in the resulting video file. Browser: Chromium Version 87.0.4280.88 (Official Build) snap (64-bit) Extension version: 9.0

[old-zoomer]

https://github.com/muaz-khan/RecordRTC/issues/709

[usmnoor]

@sanpangzi I've got some issue with the code presented by you. Are you available for a quick fix?

[usmnoor]

@sanpangzi Please share your update if your availability is possible for a quick fix.

[shiveshsingh87]

@sanpangzi did you get it(canvas + headphone input and output) successfully implemented?

[louismorgner]

Just leaving what I found here for future reference: it appears that there is simply no way to record system-level audio output with any web browser on macOs/linux. It works on windows. I've been trying RecordRTC, all sorts of MediaStream options, and WebAudio API. No way to battle the OS-induced barriers tho. Happy to be proven wrong tho.

[hareshitechnotion]

I recorded tab audio and screen using offscreen. Now I need to include microphone sound with the captured video/tabAudio. Does anyone have any ideas?