Problem with Audio / channels / layout

skrew commented 1 year ago

Hi,

Users complaining about the audio support:

I have a 5.1 hardware, the lib always send 5.1 sound, even if i'm watching a simple stereo TV channel
And if i'm watching a 5.1 movie, it play as stereo (2.0) mode

My app have MPVLib too, and with MPV the same user tell it's working fine.

I have watched ao_audiounit.m and it look like they use LPCM mode to handle all situations.

What do you think ?

kingslay commented 1 year ago

设置了KSOptions.channelLayout 也不行吗

kingslay commented 1 year ago

我试着根据mpv的代码改了下。也不知道这样对不对，因为我这边没有5.1声道设备。也没有找到在macbook模拟5.1声道的方法。所以只能麻烦你们进行测试下

skrew commented 1 year ago

Sorry i can't test right now, because you have changed KSPlayerLayer behavior recently (it needs URL at init), and i need to init the view at start (because i'm supporting multi player libs).

As i don't have an url to play at this moment, i'm init it with a fake url: KSPlayerLayer(url: URL(string: "http://xyzfakedomain.com")!, options: MEOptions())

This make the app crash here: if var layout = setupLayout(audioUnit: audioUnit) {

Any luck to have a way to init a KSPlayerLayer without parameters like before ?

kingslay commented 1 year ago

你可以参考PlayerView。把playerLayer设置成可选。

open class PlayerView: UIView, KSPlayerLayerDelegate, KSSliderDelegate {
    public typealias ControllerDelegate = PlayerControllerDelegate
    public var playerLayer: KSPlayerLayer? {
        didSet {
            playerLayer?.delegate = self
        }
    }
}

你的crash是在什么设备上发生的

skrew commented 1 year ago

on apple TV 4K 2021

Note i think it crash because of my fake url

skrew commented 1 year ago

I still have a crash (my testers too) when you are playing a file on a real Apple TV (it works on simulator)

skrew commented 1 year ago

Sorry but still not good:

On my configuration (Apple TV / normal sound bar) i miss some sounds, like voices (looks like i'm in 5.1 instead of stereo, which is my config)
For the users with 5.1 / 7.1 sound system, "Result at home with 2 systems in 5.1.2, the central one remains mute, the channels according to the sound tracks differ, either inaudible, or they arrive on my left front speaker"

skrew commented 1 year ago

Tested to works on any system, this fork:

https://github.com/plexinc/mpv/blob/v3.0/audio/out/ao_audiounit.m

With this patch for tvOS 16 maybe ?

https://github.com/plexinc/mpv/commit/b9e116c08719966221b1c270ef446ad9b6131f01

kingslay commented 1 year ago

Sorry but still not good:

On my configuration (Apple TV / normal sound bar) i miss some sounds, like voices (looks like i'm in 5.1 instead of stereo, which is my config)

For the users with 5.1 / 7.1 sound system, "Result at home with 2 systems in 5.1.2, the central one remains mute, the channels according to the sound tracks differ, either inaudible, or they arrive on my left front speaker"

在你的tv上，执行这个代码之后， if let audioUnit = engine.outputNode.audioUnit { KSOptions.channelLayout = AVAudioChannelLayout(layout: setupLayout(audioUnit: audioUnit)) } KSOptions.channelLayout.channelCount 和 AVAudioSession.sharedInstance().outputNumberOfChannels 的值分别是多少呢？

skrew commented 1 year ago

if let audioUnit = engine.outputNode.audioUnit { KSOptions.channelLayout = AVAudioChannelLayout(layout: setupLayout(audioUnit: audioUnit)) }

(lldb) po KSOptions.channelLayout.channelCount
2

(lldb) po AVAudioSession.sharedInstance().outputNumberOfChannels
8

skrew commented 1 year ago

With the last patch, i got a warning:

Output channel layout '8 channels (FL+FL+FL+FL+FL+FL+FL+FL)' is not supported

Then a crash here:

let nbSamples = swr_get_out_samples(swrContext, numberOfSamples)

kingslay commented 1 year ago

我增加了log。麻烦你把日志发给我下，可以加一下channelLayout 这个关键词进行过滤

skrew commented 1 year ago

The log from the play to the crash

It don't crash anymore while watching a stereo live channel, but crash when the file contain more than 2 audio channels.

2022-11-30 11:17:08.375 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:240 init(codecpar:) | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 0, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 8, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 1, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)))
🔹 2022-11-30 11:17:08.378 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:249 init(codecpar:) | channelLayout map: [__C.AVChannelCustom(id: __C.AVChannel(rawValue: 0), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: 0), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: 0), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil)]
🔹 2022-11-30 11:17:08.380 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:255 init(codecpar:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 2), nb_channels: 8, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)
🔹 2022-11-30 11:17:08.380 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: MEPlayerItem.swift:127 init(url:options:) | Output channel layout '8 channels (FL+FL+FL+FL+FL+FL+FL+FL)' is not supported

When it doesn't crash:

🔹 2022-11-30 11:27:30.203 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:240 init(codecpar:) | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 0, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 8, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 1, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)))
🔹 2022-11-30 11:27:30.206 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:249 init(codecpar:) | channelLayout map: [__C.AVChannelCustom(id: __C.AVChannel(rawValue: 0), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: 0), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: 0), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil), __C.AVChannelCustom(id: __C.AVChannel(rawValue: -1), name: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), opaque: nil)]
🔹 2022-11-30 11:27:30.208 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:255 init(codecpar:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 2), nb_channels: 8, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)
🔹 2022-11-30 11:27:30.208 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: MEPlayerItem.swift:127 init(url:options:) | Output channel layout '8 channels (FL+FL+FL+FL+FL+FL+FL+FL)' is not supported

🔹 2022-11-30 11:27:30.209 [Debug] [KSPlayer_soun] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: MEPlayerItem.swift:127 init(url:options:) | Output channel layout "8 channels (USR1514465088@+FL+FL+FL@† +NONE@ˇˇˇˇß˘#™˛ˇˇˇZTUMZTUMDNOC+FL+FL+USR-2082936224@)" is invalid or unsupported.
(...)
🔹 2022-11-30 11:27:30.839 [Debug] [main] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: KSPlayerLayer.swift:367 changeLoadState(player:) | ["decodeVideoTime": 0.000293124932795763, "readAudioTime": 0.00020533334463834763, "firstTime": 1.1720472499728203, "decodeAudioTime": 0.0006334586068987846, "readVideoTime": 0.00023895828053355217, "findTime": 0.1651997920125723, "openTime": 0.3654407914727926, "readyTime": 0.010520749725401402]

skrew commented 1 year ago

No more crash, we are near something. BTW:

If i'm watching a TV channel, the sound is OK
If i'm watching an episode of a serie with surround sound, i'm not hearing voices. Looks like i'm in a format which is not in stereo (it's my config)

🔹 Resample.swift:245 setup(descriptor:) | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 0, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 8, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 1, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)))
🔹 Resample.swift:251 setup(descriptor:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 8, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)
🔹 Resample.swift:260 setup(descriptor:) | channelLayout mapping: [0, 1, 3, 2, 6, 7, -22, 6]

skrew commented 1 year ago

Same problem, new logs:

🔹 Resample.swift:283 channel | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 0, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 8, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 1, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)))
🔹 Resample.swift:240 init(codecpar:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 7, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil

kingslay commented 1 year ago

我增加了日志了。你在试下，然后把日志发给我。你是有7.1的设备吗？还是可以用软件进行模拟7.1吗？

skrew commented 1 year ago

I don't have a 7.1 device, just a "classical" sound bar (stereo, 2.1), but i have a testflight user who have 5.1 and 7.1 devices

logs:

🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:283 channel | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 0, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 8, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 1, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)))
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 1
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 0
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 2
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 1
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 4
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 3
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 3
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 2
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 5
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 9
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 6
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 10
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 7
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 8
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 8
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 9
🔹 2022-12-01 14:52:56.769 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:315 channel | out channelLayout mask: 1807
🔹 2022-12-01 14:52:56.771 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:240 init(codecpar:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 7, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)

skrew commented 1 year ago

Same, new logs:

🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:283 channel | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 0, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 8, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 1, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)))
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 1
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 0
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 2
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 1
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 4
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 3
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 3
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 2
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 5
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 4
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 6
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 5
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 7
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 6
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:305 channel | KSOptions channelLayout label: 8
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:307 channel | KSOptions channelLayout avChannel: 7
🔹 2022-12-01 16:46:06.344 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:315 channel | out channelLayout mask: 255
🔹 2022-12-01 16:46:06.346 [Debug] [KSPlayer_MEPlayerItem_open] [PlayerCore.swift:214] viewPlayerInitialized() > KSPlayer: Resample.swift:240 init(codecpar:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 8, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)

kingslay commented 1 year ago

应该少了一条打印tag 的值的日志。但是看了下目前的日志应该是对的。只是他的音响摆放的位置好象不对。目前用户那边反应的问题是什么呢。是发出声音的音响的位置错乱吗还是没有声音

kingslay commented 1 year ago

是所有的音响都没有发出声音吗

skrew commented 1 year ago

In this log, it's my test. I can't send more than 3 or 4 testflight version / day. The actual problem (for me) is, when i play a multi channel file, i only hear some sounds (not the voice of actors), because the lib send multi channel (5.1 ?) to my 2.1 sound bar. I hope i'm clear enough.

If you want, i can open a teamviewer session so you can test directly on a device ?

skrew commented 1 year ago

TV channels which "only" have stereo sound works fine, the problem is only when file contain 5.1 / 7.1 audio, the lib have to resample it the 2.0 (stereo), like before, or like any other app based on libs like MPV, VLC (...)

kingslay commented 1 year ago

但是我看这个日志输出的音响个数是8个啊

kingslay commented 1 year ago

我在swiftui的demo里面增加了一个7.1的视频。你用你的设备看那个视频有问题吗

skrew commented 1 year ago

Yep but it's wrong. When i trace your code, before audio init, i have the good number of voices (2) via AVAudioSession.sharedInstance().outputNumberOfChannels but after audio init (KSOptions.setAudioSession()) outputNumberOfChannels is 8

kingslay commented 1 year ago

32c946113956877c5998af59b77e00f5db8f4b4d 这个commit会把tag给打印出来。你看下你的设备这个tag的值是什么

kingslay commented 1 year ago

那你可以把enableMaxOutputChannels 这个设置为false 试试看

kingslay commented 1 year ago

或是把AVAudioSession.sharedInstance().setSupportsMultichannelContent(true) 改成设置成false

skrew commented 1 year ago

AudioEnginePlayer.swift:179 setupLayout(audioUnit:) | audio unit channelLayout tag: 0 I'm testing your 2 latests posts and will post the result in a few minutes

skrew commented 1 year ago

enableMaxOutputChannels to false: OK with stereo / 5.1 files (in my case, stereo sound bar) setSupportsMultichannelContent(false) don't fix the missing voices problem

kingslay commented 1 year ago

@fedalcode 麻烦你试下当 enableMaxOutputChannels = true的时候。 5.1 的音响能否工作。谢谢

kingslay commented 1 year ago

@fedalcode 那如果enableMaxOutputChannels = false呢会怎么样呢麻烦你测试下

kingslay commented 1 year ago

@fedalcode 我把enableMaxOutputChannels去掉了。改成根据视频的音轨个数来设置setPreferredOutputNumberOfChannels。这样应该不会有问题了。如果还有问题的话，麻烦你把日志发给我下，日志的过滤关键字是 channelLayout

kingslay commented 1 year ago

player does it automatically

kingslay commented 1 year ago

@fedalcode 不好意思，我写错了代码了。我写成max。应该是min。 min(UInt32(AVAudioSession.sharedInstance().maximumOutputNumberOfChannels), channels) 我提交了新的代码了。麻烦你在看下outputNumberOfChannels的值是不是对的。谢谢你

skrew commented 1 year ago

For me it's look like OK on my stereo sound bar. Stereo channel or 5.1 channel seems to play fine for me 👍🏻

skrew commented 1 year ago

@fedalcode yes i know, it's just to say to @kingslay that's it's ok for my config. I will send a test version to an user which have 5.1 / 7.1 sound system, tonight or tomorrow

kingslay commented 1 year ago

@fedalcode 你可以把KSLog输出的日志发给我吗？我看下为什么CENTER不能工作，日志可以通过channelLayout进行过滤下。谢谢

skrew commented 1 year ago

Hi, Testing today patchs, i have the same problems: -> Missing voices on my stereo system (but music / sounds effects working) -> Got crashs in connect(nodes[i], to: nodes[i + 1], format: format):

extension AVAudioEngine {
    func connect(nodes: [AVAudioNode], format: AVAudioFormat?) {
        if nodes.count < 2 {
            return
        }
        for i in 0 ..< nodes.count - 1 {
            connect(nodes[i], to: nodes[i + 1], format: format)
        }
    }
}

The log contain, just before the crash:

KSPlayer: MEPlayerItem.swift:127 init(url:options:) | Impossible to convert between the formats supported by the filter 'in' and the filter 'auto_scale_0'

skrew commented 1 year ago

logs for the missing actors voices:

🔹 2022-12-10 11:16:28.878 [Debug] KSPlayer: AudioEnginePlayer.swift:274 channelLayout | audio unit channelLayout tag: 0
🔹 2022-12-10 11:16:29.377 [Debug] KSPlayer: Resample.swift:285 channelLayout(channelCount:) | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 0, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 6, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 1, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)))
🔹 2022-12-10 11:16:29.377 [Debug] KSPlayer: Resample.swift:307 channelLayout(channelCount:) | KSOptions channelLayout label: 1
🔹 2022-12-10 11:16:29.378 [Debug] KSPlayer: Resample.swift:309 channelLayout(channelCount:) | KSOptions channelLayout avChannel: 0
🔹 2022-12-10 11:16:29.378 [Debug] KSPlayer: Resample.swift:307 channelLayout(channelCount:) | KSOptions channelLayout label: 2
🔹 2022-12-10 11:16:29.378 [Debug] KSPlayer: Resample.swift:309 channelLayout(channelCount:) | KSOptions channelLayout avChannel: 1
🔹 2022-12-10 11:16:29.378 [Debug] KSPlayer: Resample.swift:307 channelLayout(channelCount:) | KSOptions channelLayout label: 4
🔹 2022-12-10 11:16:29.378 [Debug] KSPlayer: Resample.swift:309 channelLayout(channelCount:) | KSOptions channelLayout avChannel: 3
🔹 2022-12-10 11:16:29.378 [Debug] KSPlayer: Resample.swift:307 channelLayout(channelCount:) | KSOptions channelLayout label: 3
🔹 2022-12-10 11:16:29.379 [Debug] KSPlayer: Resample.swift:309 channelLayout(channelCount:) | KSOptions channelLayout avChannel: 2
🔹 2022-12-10 11:16:29.379 [Debug] KSPlayer: Resample.swift:307 channelLayout(channelCount:) | KSOptions channelLayout label: 5
🔹 2022-12-10 11:16:29.379 [Debug] KSPlayer: Resample.swift:309 channelLayout(channelCount:) | KSOptions channelLayout avChannel: 4
🔹 2022-12-10 11:16:29.379 [Debug] KSPlayer: Resample.swift:307 channelLayout(channelCount:) | KSOptions channelLayout label: 6
🔹 2022-12-10 11:16:29.380 [Debug] KSPlayer: Resample.swift:309 channelLayout(channelCount:) | KSOptions channelLayout avChannel: 5
🔹 2022-12-10 11:16:29.380 [Debug] KSPlayer: Resample.swift:317 channelLayout(channelCount:) | out channelLayout mask: 63
🔹 2022-12-10 11:16:29.387 [Debug] KSPlayer: Resample.swift:243 init(codecpar:outChannel:outSampleRate:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 6, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)
🔹 2022-12-10 11:16:29.521 [Debug] KSPlayer: KSPlayerLayer.swift:364 changeLoadState(player:) | ["readAudioTime": 0.9719416252337396, "firstTime": 1.296391666866839, "openTime": 0.14269899995997548, "decodeVideoTime": 0.04653045814484358, "findTime": 0.004936666693538427, "readVideoTime": 0.9715368752367795, "readyTime": 0.001279124990105629, "decodeAudioTime": 0.0423096246086061]

kingslay commented 1 year ago

@skrew 你试下用tvos的模拟器是不是不会有这个问题。我用demo是不会有问题的，你用demo试下，看下是设置了options的那个参数导致的。

skrew commented 1 year ago

@kingslay if you play the file i have sent to you for the #344 you can see there are missing voices on your Apple TV (stereo)

kingslay commented 1 year ago

@skrew 你可以看下AVAudioSession.sharedInstance().maximumOutputNumberOfChannels 这个方法返回的是值是不是大于2.你的设备其实只是支持2音轨。如果是大于2的话，那这就是问题的产生的原因了。那你就要尝试下用其他方法来获取设备的真实音轨

kingslay commented 1 year ago

@skrew 你可以尝试下用AVAudioSession.sharedInstance().currentRoute.outputs.first.channels. 看下能不能拿到真实的音轨，如果可以的话，那我就换用这个方法来取

skrew commented 1 year ago

(lldb) po AVAudioSession.sharedInstance().maximumOutputNumberOfChannels
8

(lldb) po AVAudioSession.sharedInstance().currentRoute.outputs.first!.channels
▿ Optional<Array<AVAudioSessionChannelDescription>>
  ▿ some : 6 elements
    - 0 : <AVAudioSessionChannelDescription: 0x28349cb00, name = Salon 1; label = 1 (0x1); number = 1; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 1 : <AVAudioSessionChannelDescription: 0x28349cab0, name = Salon 2; label = 2 (0x2); number = 2; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 2 : <AVAudioSessionChannelDescription: 0x28349cac0, name = Salon 3; label = 4 (0x4); number = 3; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 3 : <AVAudioSessionChannelDescription: 0x28349caa0, name = Salon 4; label = 3 (0x3); number = 4; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 4 : <AVAudioSessionChannelDescription: 0x28349ce70, name = Salon 5; label = 5 (0x5); number = 5; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 5 : <AVAudioSessionChannelDescription: 0x28349ced0, name = Salon 6; label = 6 (0x6); number = 6; port UID = 4C2DB571-0000-0000-011F-010380794478>

It's wrong, even with my soundbar off (so using the TV sound system).

On your Apple TV, with this video, you are hearing voices ? What are your numbers ?

skrew commented 1 year ago

Also setting options.channels = at KSOptions() initialization is not working because the number i set is overrided by

options.channels = channels

in func sourceDidOpened()

kingslay commented 1 year ago

(lldb) po AVAudioSession.sharedInstance().maximumOutputNumberOfChannels
8

(lldb) po AVAudioSession.sharedInstance().currentRoute.outputs.first!.channels
▿ Optional<Array<AVAudioSessionChannelDescription>>
  ▿ some : 6 elements
    - 0 : <AVAudioSessionChannelDescription: 0x28349cb00, name = Salon 1; label = 1 (0x1); number = 1; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 1 : <AVAudioSessionChannelDescription: 0x28349cab0, name = Salon 2; label = 2 (0x2); number = 2; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 2 : <AVAudioSessionChannelDescription: 0x28349cac0, name = Salon 3; label = 4 (0x4); number = 3; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 3 : <AVAudioSessionChannelDescription: 0x28349caa0, name = Salon 4; label = 3 (0x3); number = 4; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 4 : <AVAudioSessionChannelDescription: 0x28349ce70, name = Salon 5; label = 5 (0x5); number = 5; port UID = 4C2DB571-0000-0000-011F-010380794478>
    - 5 : <AVAudioSessionChannelDescription: 0x28349ced0, name = Salon 6; label = 6 (0x6); number = 6; port UID = 4C2DB571-0000-0000-011F-010380794478>

It's wrong, even with my soundbar off (so using the TV sound system).

On your Apple TV, with this video, you are hearing voices ? What are your numbers ?

在我的设备上 (lldb) po AVAudioSession.sharedInstance().maximumOutputNumberOfChannels 2

kingslay commented 1 year ago

options.channels

options.channels一开始会用视频的音轨个数，然后进入AudioEnginePlayer就会根据当前设备的音轨数进行更新了，取两者的最小值，所以问题在于你的设备的音轨数怎么是8

po AVAudioSession.sharedInstance().currentRoute.outputs.first?.channels ▿ Optional<Array> ▿ some : 2 elements

0 : <AVAudioSessionChannelDescription: 0x283f4a800, name = 客厅 1; label = 1 (0x1); number = 1; port UID = 61A44A00-0000-0000-051F-0103805F3678>
1 : <AVAudioSessionChannelDescription: 0x283f4a810, name = 客厅 2; label = 2 (0x2); number = 2; port UID = 61A44A00-0000-0000-051F-0103805F3678>

kingslay commented 1 year ago

apple tv上的设置的音频格式你是不是设置成自动呢？还是设置为杜比5.1呢？

skrew commented 1 year ago

Yes i'm in AUTO. If i force it (from AUTO to STEREO), i have 2 channels, instead of 6.

kingslay / KSPlayer

Problem with Audio / channels / layout #331