search

Unity-Technologies / com.unity.webrtc

WebRTC package for Unity
Other
738 stars 185 forks source link

feat: Add AudioOptions class #978

Open karasusan opened 9 months ago

karasusan commented 9 months ago

This PR adds the AudioOptions class to set audio options to tracks. The options are defined by the AudioOptions class.

class AudioOptions
{
        public bool? echoCancellation;
        public bool? autoGainControl;
        public bool? noiseSuppression;
        public bool? highpassFilter;
}

This PR makes AudioStreamTrack constructor changing.

flamacore commented 9 months ago

So hey, I've followed the issues that are mentioned in this PR and I also would like to give a try to the audio options.

Sadly I have problems with getting this specific pr to build as I'm not very well versed with CMake and the like. It throws an error about not finding presets and shuts down.

Anyhow, is there any progress on making this a reality? Any sort of easier way I can add this and experiment with it in my own project yet?

karasusan commented 9 months ago

@flamacore Hi, if you can help us to test the audio option, I'll be glad to take you a package.

Here you are. https://drive.google.com/file/d/1Qf4ZLTqNxT-hoLfgJj3JZIpFu3hEPLNI/view?usp=drive_link

Please install the tgz file following this manual. https://docs.unity3d.com/Manual/upm-ui-tarball.html

flamacore commented 9 months ago

@karasusan would be delighted to. Thanks a lot for sharing the package. I'll give it a check soon and will inform if it works or not.

MaximKurbanov commented 9 months ago

I tested and issue #964 remains. I tried this with android-android and windows-android:

  #...
    {
  #...
        var track = new AudioStreamTrack(sendAudioSource, GetAudioOptions());
        mediaStream.AddTrack(track);
    }

    private AudioOptions GetAudioOptions()
    {
        return new AudioOptions()
        {
            echoCancellation = true,
            noiseSuppression = true
        };
    }
flamacore commented 9 months ago

Can confirm the echo issue persists. Not sure if it's related to us enabling/disabling the options or if the internal echo cancellation works as intended. EDIT: I tried it multiple ways:

karasusan commented 9 months ago

@MaximKurbanov @flamacore

Thank you for checking. I'd like to clarify which this issue is a specific by mobile platforms or not. For example, windows and windows, or windows or macOS.

It looks there are different implementations for mobile and desktop platforms. https://stackoverflow.com/questions/62479789/webrtc-android-echo-cancellation

MaximKurbanov commented 9 months ago

@karasusan I tested Windows-Mac, AEC does not work.

karasusan commented 8 months ago

@MaximKurbanov I understood the cause of this issue. Sorry but we need more time to fix it.

I can't share you many information about that, but at least I can say that the "audio processing" in native code is not processed.

The "audio processing" in webrtc is implemented here. https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_processing/audio_processing_impl.cc

pixlhero commented 6 months ago

@karasusan I tested the package from Oct 6th on Mac-Editor <-> iPad. It also didn't cancel the echo.

This is a show-stopper for us. So we really, really need this fix in the next release (pre-8). 🙏

Also building the plugin doesn't work for us. So if 02724602838a48b95bac2e1389d03c10c39b3c7e would fix anything (we're targeting iOS) it would be great if you could provide us with a .tgz build.

がんばってよ!

pixlhero commented 6 months ago

@karasusan I tested the package from Oct 6th on Mac-Editor <-> iPad. It also didn't cancel the echo.

This is a show-stopper for us. So we really, really need this fix in the next release (pre-8). 🙏

Also building the plugin doesn't work for us. So if 0272460 would fix anything (we're targeting iOS) it would be great if you could provide us with a .tgz build.

がんばってよ!

We found a workaround on iOS involving AVAudioSession. So for our project this issue is resolved. https://gist.github.com/pixlhero/3d370dcdbcfaefa4df12138161307da4