search

kasimok / AECAudioStream

Automatic Echo Cancellation implemented with VoiceProcessingIO audio unit
20 stars 4 forks source link

Recording silence #1

Open rebryk opened 1 year ago

rebryk commented 1 year ago

I am trying to use your code to record audio on my macOS with echo cancellation. It seems like the code is running, but there is silence in the file. Could you please help me to fix it?

Here is how i do it:

audioRecorder = AECAudioStream(sampleRate: 44100)
let format = audioRecorder!.streamFormat
let outputFile = try! AVAudioFile(
    forWriting: file,
    settings: format.settings,
    commonFormat: format.commonFormat,
    interleaved: format.isInterleaved
)

audioRecorder!.startAudioStream(enableAEC: false, enableRendererCallback: true ) { (ioData, inNumberFrames) in
    let audioBufferList = ioData.pointee
    let frameLength = audioBufferList.mBuffers.mDataByteSize / UInt32(MemoryLayout<Int16>.size)

    let pcmBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(frameLength))!

    pcmBuffer.frameLength = AVAudioFrameCount(frameLength)

    let dst = pcmBuffer.int16ChannelData![0]
    let src = audioBufferList.mBuffers.mData!.assumingMemoryBound(to: Int16.self)

    dst.initialize(from: src, count: Int(frameLength))

    do {
        try outputFile.write(from: pcmBuffer)
    } catch {
        print("Error writing to file: \(error)")
    }
}
kasimok commented 1 year ago

Hey Yurii, sorry for the project, I was trying an audio component that can filter the sound output from the microphone' s input. Are you trying to do the same thing?

Later I found implementing the AEC with this repo's approach has a drawback: you have to provide audio data to the rendererClosure in the class, which basically means we need to write a code to fill some kind of ring buffer with the audio data we need to play. otherwise the output sound's volume will be small.

And later I found apple's provides a new solution implemented with new, higher level API called: AVAudioNode

Here is the sample project: AVEchoTouch https://developer.apple.com/documentation/avfaudio/audio_engine/audio_units/using_voice_processing?language=objc

You can port it to macOS, i think, apart from that we do not need to set up avaudiosession on macOS.

Cheers Evilisn

On Thu, Jul 13, 2023 at 9:54 PM Yurii Rebryk @.***> wrote:

I am trying to use your code to record audio on my macOS with echo cancellation. It seems like the code is running, but there is silence in the file. Could you please help me to fix it?

Here is how i do it:

audioRecorder = AECAudioStream(sampleRate: 44100)let format = audioRecorder!.streamFormatlet outputFile = try! AVAudioFile( forWriting: file, settings: format.settings, commonFormat: format.commonFormat, interleaved: format.isInterleaved )

audioRecorder!.startAudioStream(enableAEC: false, enableRendererCallback: true ) { (ioData, inNumberFrames) in let audioBufferList = ioData.pointee let frameLength = audioBufferList.mBuffers.mDataByteSize / UInt32(MemoryLayout.size)

let pcmBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(frameLength))!

pcmBuffer.frameLength = AVAudioFrameCount(frameLength)

let dst = pcmBuffer.int16ChannelData![0]
let src = audioBufferList.mBuffers.mData!.assumingMemoryBound(to: Int16.self)

dst.initialize(from: src, count: Int(frameLength))

do {
    try outputFile.write(from: pcmBuffer)
} catch {
    print("Error writing to file: \(error)")
}

}

— Reply to this email directly, view it on GitHub https://github.com/kasimok/AECAudioStream/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHUG263V4OBVSSF335CGTLXP74YBANCNFSM6AAAAAA2I7QMQQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

rebryk commented 1 year ago

Yeah, I am. I've tried to use AudioEngine for recording, but I didn't succeed in making it work.

kasimok commented 1 year ago

Ah, I see. GO for the new solution implemented with AVEchoTouch. Any Questions, let me know.

On Fri, Jul 14, 2023 at 6:39 PM Yurii Rebryk @.***> wrote:

Yeah, I am. I've tried to use AudioEngine for recording, but I didn't succeed in making it work.

— Reply to this email directly, view it on GitHub https://github.com/kasimok/AECAudioStream/issues/1#issuecomment-1635668253, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHUG26MF2HEYHQYSMLNSCDXQEOUHANCNFSM6AAAAAA2I7QMQQ . You are receiving this because you commented.Message ID: @.***>

rebryk commented 1 year ago

I cannot see that they use echo cancellation in their example. I'm not sure how easy it is to add it there. Maybe smth like that will work: https://wwdcnotes.com/notes/wwdc23/10235/

Best Regards, Yurii Rebryk

On Fri, Jul 14, 2023 at 1:17 PM, kakaiikaka @.***> wrote:

Ah, I see. GO for the new solution implemented with AVEchoTouch. Any Questions, let me know.

On Fri, Jul 14, 2023 at 6:39 PM Yurii Rebryk @.***> wrote:

Yeah, I am. I've tried to use AudioEngine for recording, but I didn't succeed in making it work.

— Reply to this email directly, view it on GitHub https://github.com/kasimok/AECAudioStream/issues/ 1#issuecomment-1635668253, or unsubscribe https://github.com/notifications/unsubscribe-auth/ ABHUG26MF2HEYHQYSMLNSCDXQEOUHANCNFSM6AAAAAA2I7QMQQ . You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/kasimok/AECAudioStream/issues/1#issuecomment-1635780704, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAJD4PKC2SIAY6EPMOHS43XQE2FXANCNFSM6AAAAAA2I7QMQQ . You are receiving this because you authored the thread.Message ID: @.***>

kasimok commented 1 year ago

Look for [image: image.png]

On Fri, Jul 14, 2023 at 8:42 PM Yurii Rebryk @.***> wrote:

I cannot see that they use echo cancellation in their example. I'm not sure how easy it is to add it there. Maybe smth like that will work: https://wwdcnotes.com/notes/wwdc23/10235/

Best Regards, Yurii Rebryk

On Fri, Jul 14, 2023 at 1:17 PM, kakaiikaka @.***> wrote:

Ah, I see. GO for the new solution implemented with AVEchoTouch. Any Questions, let me know.

On Fri, Jul 14, 2023 at 6:39 PM Yurii Rebryk @.***> wrote:

Yeah, I am. I've tried to use AudioEngine for recording, but I didn't succeed in making it work.

— Reply to this email directly, view it on GitHub https://github.com/kasimok/AECAudioStream/issues/ 1#issuecomment-1635668253, or unsubscribe https://github.com/notifications/unsubscribe-auth/ ABHUG26MF2HEYHQYSMLNSCDXQEOUHANCNFSM6AAAAAA2I7QMQQ . You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/kasimok/AECAudioStream/issues/1#issuecomment-1635780704>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABAJD4PKC2SIAY6EPMOHS43XQE2FXANCNFSM6AAAAAA2I7QMQQ>

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/kasimok/AECAudioStream/issues/1#issuecomment-1635808626, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHUG25RYWQLFBGQ7DTG6YTXQE5B5ANCNFSM6AAAAAA2I7QMQQ . You are receiving this because you commented.Message ID: @.***>

rebryk commented 1 year ago

Look for [image: image.png]

I didn't get it :D

kasimok commented 1 year ago

the avechotouch demo project is a simple demo app, that can filter output signal from input signal, give it a try!Sent from my iPhoneOn Jul 14, 2023, at 11:47 PM, Yurii Rebryk @.***> wrote:

Look for [image: image.png]

I didn't get it :D

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

rebryk commented 1 year ago

I've tried AVEchoTouch project.

So, when speech is playing inside the app and I am recording, it provides echo cancellation. However, when I record and play back speech audio using the default player, its volume is very low and echo cancellation does not work.

I need echo cancellation to only record my voice during Hangouts calls. Any ideas how to solve it?

kasimok commented 1 year ago

Bro, that's a little hard, basically you need to manipulate inputs from each audio unit and merge them. But I have tried, and I have a POC project. I'll try to find it and please leave a email.

rebryk commented 1 year ago

Actually, with the new AVAudioEngine API on macOS 14, it is easy to solve by disabling ducking. However, I have no idea what to do with earlier versions.

Rostyk commented 10 months ago

@rebryk Hey Yurii. Facing exactly the same issue with voice processing unit making everything very quiet. Did you find the solution on macOS < 13.

rebryk commented 10 months ago

Nope

Best Regards, Yurii Rebryk https://www.linkedin.com/in/yrebryk

On Fri, Dec 1 2023 at 9:49 PM, Rostyk < @.*** > wrote:

@rebryk ( https://github.com/rebryk ) Hey Yurii. Facing exactly the same issue with voice processing unit ducking making everything very quiet. Did you find the solution on macOS < 13.

— Reply to this email directly, view it on GitHub ( https://github.com/kasimok/AECAudioStream/issues/1#issuecomment-1836533428 ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/ABAJD4O5LYV5N4EMIVCOEVLYHIKCDAVCNFSM6AAAAAA2I7QMQSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZWGUZTGNBSHA ). You are receiving this because you were mentioned. Message ID: <kasimok/AECAudioStream/issues/1/1836533428 @ github. com>