shogo4405 / HaishinKit.swift

Camera and Microphone streaming library via RTMP and SRT for iOS, macOS, tvOS and visionOS.
BSD 3-Clause "New" or "Revised" License
2.76k stars 612 forks source link

Appending audio sample buffer crashes when unwrapping _converter variable #889

Closed alexisPonce98 closed 3 years ago

alexisPonce98 commented 3 years ago

Describe the bug A clear and concise description of what the bug is. When i am appending audio sample buffers from a RPScreenRecorder, the app is crashing with HaishinKit/AudioCodec.swift:164: Fatal error: Unexpectedly found nil while unwrapping an Optional value. It seems to be when the _converter variable from the AudioCodec class. If i attach the app audio it doesnt seem to crash but there is also no audio being passed to the stream, it only happens with mic audio. To Reproduce Steps to reproduce the behavior:

  1. append an audio sampleBuffer to the stream
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior A clear and concise description of what you expected to happen. Not to crash and append the audio to the stream.

Screen Shot 2021-07-18 at 3 20 01 PM

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Smartphone (please complete the following information):

Additional context Add any other context about the problem here.

shogo4405 commented 3 years ago

It caused to crash with an unsupported input format for CMSampleBuffer. I see mChanneIsPerFrame: 3. It value is mChanneIsPerFrame: 2(stereo) or 1(mono) normal.

Please provide more info print(RPScreenRecorder.CMSampleBuffer).

alexisPonce98 commented 3 years ago

Thank you for your reply!, yea i thought that was weird that i am getting mChannelsIsPerFram: 3. Here is a screenshot of the sample all the info from the sample buffer that is being captured by rpScreenRecorder. Screen Shot 2021-07-30 at 12 34 57 PM

SergiyZhyrov commented 3 years ago

Hello. There are related bug with audio, also crash. To fix just update one enum

    enum ChannelConfiguration: UInt8 {
        case definedInAOTSpecificConfig = 0
        case frontCenter = 1
        case frontLeftAndFrontRight = 2
        case frontCenterAndFrontLeftAndFrontRight = 3
        case frontCenterAndFrontLeftAndFrontRightAndBackCenter = 4
        case frontCenterAndFrontLeftAndFrontRightAndBackLeftAndBackRight = 5
        case frontCenterAndFrontLeftAndFrontRightAndBackLeftAndBackRightLFE = 6
        case frontCenterAndFrontLeftAndFrontRightAndSideLeftAndSideRightAndBackLeftAndBackRightLFE = 7

        case ch8 = 8
        case ch9  = 9
        case ch10 = 10
        case ch11 = 11
        case ch12 = 12
        case ch13 = 13
        case ch14 = 14
        case ch15 = 15
        case ch16 = 16
    }

and also change defaultChannels in public class AudioCodec. Converter can't convert to 16 output channels /// 0 means according to a input source public static let defaultChannels: UInt32 = 2

To reproduce bug, just try BlackHole with 16 channels https://github.com/ExistentialAudio/BlackHole

shogo4405 commented 3 years ago

HaishinKit can support 1 or 2 channels input audio. I thought iOS can't support 3 channels an AudioConverter. I will not fix it.