millicast / millicast-player-unreal-engine-plugin

Millicast Player plugin for Unreal Engine
Other
19 stars 15 forks source link

External audio consumer support #15

Closed improbable-andreaskrugersen closed 2 years ago

improbable-andreaskrugersen commented 2 years ago

The Millicast plugin uses the default Unreal audio device. Unfortunately that doesn't work if you use a different audio solution. We use Wwise on our project for example, so we had to do additional work to integrate this. So I prepared this PR which allows you to implement your own audio consumer (whichever you want) or fall back to the default solution that was already implemented.

Feel free to reject if this doesn't fit your ideas. However, it would make future updates easier for us to manage and I think it's a useful addition to the plugin.

One caveat though: Since we don't use the default Unreal audio device, I had no way of testing that the default flow still works. It should, since I just copied the same code into a different file, but you'd have to verify that in your own test setup

murillo128 commented 2 years ago

Thank you very much for the PR, it is something that we definitely what to support.

@dbaldassi could you take a look?

dbaldassi commented 2 years ago

Yes that's great, I'll take a look at it.

dbaldassi commented 2 years ago

The default audio consumer is not working properly. We actually need to do some modification to make it work : in Peerconnection.cpp :

@@ -69,8 +69,12 @@ FWebRTCPeerConnection* FWebRTCPeerConnection::Create(const FRTCConfig& Config, T
     {
         UMillicastDefaultAudioConsumer* DefaultAudioConsumer = NewObject<UMillicastDefaultAudioConsumer>();
         DefaultAudioConsumer->AddToRoot();
+               AudioDeviceModule->SetAudioConsumer(DefaultAudioConsumer);
+       }
+       else
+       {
+               AudioDeviceModule->SetAudioConsumer(ExternalAudioConsumer);
        }
-    AudioDeviceModule->SetAudioConsumer(ExternalAudioConsumer);

        FWebRTCPeerConnection * PeerConnectionInstance = new FWebRTCPeerConnection();
        webrtc::PeerConnectionDependencies deps(PeerConnectionInstance);

In the MillicastExternalAudioConsummer.h :

struct FMillicastAudioParameters
 {
-    int32 SampleSize = 16;
+    int32 SampleSize = sizeof(int16);
     int32 SamplesPerSecond = 48000;
     int32 NumberOfChannels = 2;
     int32 TimePerFrameMs = 10;
improbable-andreaskrugersen commented 2 years ago

Thanks for checking @dbaldassi! Should be fixed now