search

kingslay / KSPlayer

A video player for iOS、macOS、tvOS、visionOS , based on AVPlayer and FFmpeg, support the horizontal, vertical screen. support adjust volume, brightness and seek by slide, SwiftUI, support subtitles.
https://apps.apple.com/app/tracyplayer/id6450770064
GNU General Public License v3.0
1.02k stars 201 forks source link

Problem with Audio / channels / layout #331

Closed skrew closed 1 year ago

skrew commented 1 year ago

Hi,

Users complaining about the audio support:

I have a 5.1 hardware, the lib always send 5.1 sound, even if i'm watching a simple stereo TV channel
And if i'm watching a 5.1 movie, it play as stereo (2.0) mode

My app have MPVLib too, and with MPV the same user tell it's working fine.

I have watched ao_audiounit.m and it look like they use LPCM mode to handle all situations.

What do you think ?

UnknownCoder807 commented 1 year ago

Do you have Dolby Atmos displayed on compatible content (Atmos dolby digital+), as with infuse?

I have just tried with Netflix and I get Dolby Atmos flashing up on my soundbar when the film starts.

JeremTM93 commented 1 year ago

What happens if any user has any one of the following Dolby Atmos system?

IMG_7449

Infuse is OK with Dolby Atmos DD+, it uses the Apple player for this. Otherwise, it's sent in LPCM and we lose the pitch effects like with MPV or VLC 😕

UnknownCoder807 commented 1 year ago

What happens if any user has any one of the following Dolby Atmos system?

@kingslay can ksplayer define all those atmos sound output? Or can you implement it? also, with logic_7_1_A, 5.1 videos do not output centre sound.

IMG_7449

I made some changes above, in case you missed it @kingslay

kingslay commented 1 year ago

@UnknownCoder807 我不知道这些atmos可不可以实现，但是我猜测是不行的，因为ffmepg只支持下面几种标准的音频

static const struct channel_layout_name channel_layout_map[] = { { "mono", AV_CHANNEL_LAYOUT_MONO }, { "stereo", AV_CHANNEL_LAYOUT_STEREO }, { "2.1", AV_CHANNEL_LAYOUT_2POINT1 }, { "3.0", AV_CHANNEL_LAYOUT_SURROUND }, { "3.0(back)", AV_CHANNEL_LAYOUT_2_1 }, { "4.0", AV_CHANNEL_LAYOUT_4POINT0 }, { "quad", AV_CHANNEL_LAYOUT_QUAD }, { "quad(side)", AV_CHANNEL_LAYOUT_2_2 }, { "3.1", AV_CHANNEL_LAYOUT_3POINT1 }, { "5.0", AV_CHANNEL_LAYOUT_5POINT0_BACK }, { "5.0(side)", AV_CHANNEL_LAYOUT_5POINT0 }, { "4.1", AV_CHANNEL_LAYOUT_4POINT1 }, { "5.1", AV_CHANNEL_LAYOUT_5POINT1_BACK }, { "5.1(side)", AV_CHANNEL_LAYOUT_5POINT1 }, { "6.0", AV_CHANNEL_LAYOUT_6POINT0 }, { "6.0(front)", AV_CHANNEL_LAYOUT_6POINT0_FRONT }, { "hexagonal", AV_CHANNEL_LAYOUT_HEXAGONAL }, { "6.1", AV_CHANNEL_LAYOUT_6POINT1 }, { "6.1(back)", AV_CHANNEL_LAYOUT_6POINT1_BACK }, { "6.1(front)", AV_CHANNEL_LAYOUT_6POINT1_FRONT }, { "7.0", AV_CHANNEL_LAYOUT_7POINT0 }, { "7.0(front)", AV_CHANNEL_LAYOUT_7POINT0_FRONT }, { "7.1", AV_CHANNEL_LAYOUT_7POINT1 }, { "7.1(wide)", AV_CHANNEL_LAYOUT_7POINT1_WIDE_BACK }, { "7.1(wide-side)", AV_CHANNEL_LAYOUT_7POINT1_WIDE }, { "octagonal", AV_CHANNEL_LAYOUT_OCTAGONAL }, { "hexadecagonal", AV_CHANNEL_LAYOUT_HEXADECAGONAL }, { "downmix", AV_CHANNEL_LAYOUT_STEREO_DOWNMIX, }, { "22.2", AV_CHANNEL_LAYOUT_22POINT2, }, };

UnknownCoder807 commented 1 year ago

What about when using AVAudioChannelLayout(layoutTag: kAudioChannelLayoutTag_Logic_7_1_A)! ? there is no centre sound for 5.1 video.

kingslay commented 1 year ago

@UnknownCoder807 我又重新改下代码了。麻烦你看下是否可以。这次不用你在设置kAudioChannelLayoutTag_Logic_7_1_A和kAudioChannelLayoutTag_Logic_5_1_A了

UnknownCoder807 commented 1 year ago

@UnknownCoder807 我又重新改下代码了。麻烦你看下是否可以。这次不用你在设置kAudioChannelLayoutTag_Logic_7_1_A和kAudioChannelLayoutTag_Logic_5_1_A了

Centre doesn't work on 5.1. Centre doesn't work on 7.1 either.

kingslay commented 1 year ago

@UnknownCoder807 能够把日志发给我吗

UnknownCoder807 commented 1 year ago

5.1 log

KSPlayer: AudioEnginePlayer.swift:169 prepare(options:) | audioStreamBasicDescription channelLayout tag: 4294901760
KSPlayer: AudioEnginePlayer.swift:170 prepare(options:) | audioStreamBasicDescription channelLayout channelDescriptions: []
KSPlayer: AudioEnginePlayer.swift:174 prepare(options:) | outputFormat channelLayout tag: 0
KSPlayer: AudioEnginePlayer.swift:175 prepare(options:) | outputFormat channelLayout channelDescriptions: [__C.AudioChannelDescription(mChannelLabel: 1, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 2, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 4, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 3, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 5, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 6, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0))]
KSPlayer: AVFoundationExtension.swift:134 channelLayout(channelCount:) | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 4294901760, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 0, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 0, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)))
KSPlayer: KSPlayerLayer.swift:149 state | playerStateDidChange - readyToPlay
KSPlayer: Resample.swift:190 init(audioDescriptor:outChannel:outSampleRate:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 6, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | Using fltp internally between filters

KSPlayer: MEPlayerItem.swift:127 init(url:options:) | Matrix coefficients:

KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FL: 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FL:1.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FC:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | LFE:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | 

KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FR: 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FR:1.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FC:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | LFE:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | 

KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FC: 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FC:1.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | LFE:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | 

KSPlayer: MEPlayerItem.swift:127 init(url:options:) | LFE: 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FC:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | LFE:1.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | 

KSPlayer: MEPlayerItem.swift:127 init(url:options:) | BL: 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FC:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | LFE:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SL:1.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | 

KSPlayer: MEPlayerItem.swift:127 init(url:options:) | BR: 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FR:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | FC:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | LFE:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SL:0.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) | SR:1.000000 
KSPlayer: MEPlayerItem.swift:127 init(url:options:) |

7.1 log

KSPlayer: AudioEnginePlayer.swift:169 prepare(options:) | audioStreamBasicDescription channelLayout tag: 4294901760
KSPlayer: AudioEnginePlayer.swift:170 prepare(options:) | audioStreamBasicDescription channelLayout channelDescriptions: []
KSPlayer: AudioEnginePlayer.swift:174 prepare(options:) | outputFormat channelLayout tag: 0
KSPlayer: AudioEnginePlayer.swift:175 prepare(options:) | outputFormat channelLayout channelDescriptions: [__C.AudioChannelDescription(mChannelLabel: 1, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 2, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 4, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 3, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 5, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 6, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 33, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0)), __C.AudioChannelDescription(mChannelLabel: 34, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (0.0, 0.0, 0.0))]
KSPlayer: KSPlayerLayer.swift:149 state | playerStateDidChange - readyToPlay
KSPlayer: AVFoundationExtension.swift:134 channelLayout(channelCount:) | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 4294901760, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 0, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 0, mChannelFlags: __C.AudioChannelFlags(rawValue: 4), mCoordinates: (0.0, 3.1365813e-38, 1e-45)))
KSPlayer: Resample.swift:190 init(audioDescriptor:outChannel:outSampleRate:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 8, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)
kingslay commented 1 year ago

@UnknownCoder807 我更新了下代码。麻烦你在试下

UnknownCoder807 commented 1 year ago

Sorry @kingslay but there is no centre on 5.1 (logs below)

KSPlayer: AVFFmpegExtension.swift:413 layoutTag | FFmepg channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 6, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)
KSPlayer: AudioEnginePlayer.swift:151 prepare(options:audioDescriptor:) | outputFormat channelLayout tag: 7929862
KSPlayer: AudioEnginePlayer.swift:152 prepare(options:audioDescriptor:) | outputFormat channelLayout channelDescriptions: []
KSPlayer: AVFoundationExtension.swift:134 channelLayout() | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 7929862, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 0, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 0, mChannelFlags: __C.AudioChannelFlags(rawValue: 99835424), mCoordinates: (1e-45, 0.0, 0.0)))
KSPlayer: AVFoundationExtension.swift:160 channelLayout() | out channelLayout mask: 1551 nb_channels: 6
KSPlayer: Resample.swift:190 init(audioDescriptor:outChannel:outSampleRate:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 6, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)

and no centre on 7.1 (logs below)

KSPlayer: AVFFmpegExtension.swift:413 layoutTag | FFmepg channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 8, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)
KSPlayer: AudioEnginePlayer.swift:151 prepare(options:audioDescriptor:) | outputFormat channelLayout tag: 8388616
KSPlayer: AudioEnginePlayer.swift:152 prepare(options:audioDescriptor:) | outputFormat channelLayout channelDescriptions: []
KSPlayer: KSPlayerLayer.swift:149 state | playerStateDidChange - readyToPlay
KSPlayer: AVFoundationExtension.swift:134 channelLayout() | KSOptions channelLayout: AudioChannelLayout(mChannelLayoutTag: 8388616, mChannelBitmap: __C.AudioChannelBitmap(rawValue: 0), mNumberChannelDescriptions: 0, mChannelDescriptions: __C.AudioChannelDescription(mChannelLabel: 0, mChannelFlags: __C.AudioChannelFlags(rawValue: 0), mCoordinates: (9.1835e-41, 0.0, 0.0)))
KSPlayer: AVFoundationExtension.swift:160 channelLayout() | out channelLayout mask: 1599 nb_channels: 8
KSPlayer: Resample.swift:190 init(audioDescriptor:outChannel:outSampleRate:) | out channelLayout: AVChannelLayout(order: __C.AVChannelOrder(rawValue: 1), nb_channels: 8, u: __C.AVChannelLayout.__Unnamed_union_u(), opaque: nil)
kingslay commented 1 year ago

@UnknownCoder807 我又更新下代码了。你可以先试下5.1看下行不行，如果5.1不行的话，那7.1应该也是不行的

UnknownCoder807 commented 1 year ago

Centre and rears working on 5.1 and also on 7.1

kingslay commented 1 year ago

太好了。终于可以了。 Dolby Atmos 你也可以试下5.1.2 看可不可以

UnknownCoder807 commented 1 year ago

太好了。终于可以了。 Dolby Atmos 你也可以试下5.1.2 看可不可以

I need to source a dolby atmos test video to be able to try.

JeremTM93 commented 1 year ago

Centre and rears working on 5.1 and also on 7.1

Great job ! So, 2.0 stereo track = sound only comes from the 2 front speakers and centre turned off?

5.1 / 7.1 track = dialogue on centre?

The KS player adapts to the track it receives and sends in LPCM to the audio receiver?

@kingslay do you know if your library is ok with Dolby MAT like the Apple player?

UnknownCoder807 commented 1 year ago

Centre and rears working on 5.1 and also on 7.1

Great job ! So, 2.0 stereo track = sound only comes from the 2 front speakers and centre turned off?

5.1 / 7.1 track = dialogue on centre?

The KS player adapts to the track it receives and sends in LPCM to the audio receiver?

@kingslay do you know if your library is ok with Dolby MAT like the Apple player?

I used a video from this website to check all speakers: https://www.demolandia.net/cinema/dolby-demo-trailers-hd/page-10.html

All speakers seemed to work but my sound bar did not show the ATMOS tag. I guess this is because ffmpeg sends the sound LPCM format

UnknownCoder807 commented 1 year ago

太好了。终于可以了。 Dolby Atmos 你也可以试下5.1.2 看可不可以

I tried an atmos video and no atmos flag on my soundbar.

But, thank you for your awesome work on getting this far.

kingslay commented 1 year ago

apple对外只支持 case otherFormat = 0

case pcmFormatFloat32 = 1

case pcmFormatFloat64 = 2

case pcmFormatInt16 = 3

case pcmFormatInt32 = 4

然后我试着用pcmFormatInt32，会crash。reason: '[[busArray objectAtIndexedSubscript:(NSUInteger)element] setFormat:format error:&nsErr]: returned false, 所以还需要看下是什么原因