Closed levs42 closed 3 months ago
- Is it correct to understand that ReplayKit is the one used from the Control Center?
Yes, from the Control Center
- The reason is that it becomes 3 channels. ðŸ˜
Indeed, it's unfortunate! I believe audio mixer still should handle it if the proper channel mapping is provided, but somehow it doesn't. Will take a look too, just wanted to report the issue.
Currently, channels passing through this line fail to convert and are treated as muted audio. https://github.com/shogo4405/HaishinKit.swift/blob/main/Sources/Util/AVAudioUtil.swift#L45-L47
Modify it as follows: https://github.com/shogo4405/HaishinKit.swift/compare/main...feature/replaykit-multicam-issue#diff-7d19719e55e081126ea8a413c2b66323cb0a7881098ef295294650da226b0922R37-R38
rtmpStream.audioMixerSettings = .init(sampleRate: 0, channels: 2)
rtmpStream.audioMixerSettings.tracks[0]?.downmix = true
In an environment with iOS 17.5 + iPhone 15 Pro Max, you can hear the mixed audio. I am struggling with whether kAudioChannelLayoutTag_DVD_2 is the appropriate value.
While trying kAudioChannelLayoutTag_DVD_2
the right ear sounded significantly more quiet on the stream while listening it through headphones. I also tried without downmix
with no difference; I tried kAudioChannelLayoutTag_AAC_3_0
and it worked without downmix and sounded fine on the stream. The layout is a bit different: Center, Left, Right instead of Left, Right, Center surround for DVD2. Maybe "surround" channel is treated differently by CoreAudio; I'm not too familiar and there's barely any doc. AAC_3_0
worked with 1 output channel too.
Edit: CoreAudioTypes.h sheds some light how the types correspond between each other.
I changed it to MPEG_3_0_B because when I look at it later, I might mistakenly think "Why is it AAC when it's PCM audio?"
kAudioChannelLayoutTag_AAC_3_0 = kAudioChannelLayoutTag_MPEG_3_0_B, // C L R
It was ReplayKit audio, CLR was appropriate. However, on some devices, it might come as LRC. I thought it might be a good idea to allow the library user to set the LayoutTag in the future.
Thank you for the feedback.
It's also worth updating output channels count in the screencast example, otherwise setupAudioNodes
throws an error.
Describe the bug
There's some interference between
stream.isMultiCamSessionEnabled = true
and ReplayKit. Somehow enabling multi cam session changes audio format of.audioMic
audio in SampleHandler. Without multicam:With multicam:
Cannot hear audio on the stream. I tried to use channel map for the main track but it doesn't work. Audio buffer has data.
To Reproduce
Expected behavior
To head audio on the stream
Version
main branch, 52a34fd261ff46ad01a8d2e64a883cfa39082b80
Smartphone info.
iPhone 15 Pro Max, iOS 15.5
Additional context
No response
Screenshots
No response
Relevant log output
No response