baresip / baresip-ios

Baresip for iOS
46 stars 33 forks source link

AudioUnit hell #28

Closed LonestarX91 closed 3 years ago

LonestarX91 commented 4 years ago

Hello,

So i have a big issue. The story starts like this: I need audio only (no video), a few days ago I tried using https://github.com/miche-atucha/taresip which was lovely managed to call, could hear the other client (even tho it had a small issue of unintentional audio passthrough, it was pretty much ok). ALSO, IT USED COREAUDIO surprised pikachu. As far as I looked, that was built with baresip-0.4.20, re-0.4.17 and rem-0.4.7.

Now here comes problem#1: I needed TLS. So i found this https://github.com/rtcexpert/baresip-ios-bitcode (bless his soul), which is also a pending PR here and it connects fine. So i took the mk, i grabbed baresip 0.6.6, re 0.6.1 and rem 0.6.0, openssl&co, built it just fine. problem is if i wanna add coreaudio module in

BARESIP_BUILD_FLAGS_X86_64 := \
    $(BARESIP_BUILD_FLAGS) \
    EXTRA_MODULES='g711 coreaudio srtp audiounit avcapture opengles'

it will fail with a metric ton of errors like here (https://github.com/baresip/baresip/issues/593). So I did what @eriksundin suggested and patched coreaudio.c with

CFStringRef coreaudio_get_device_uid(const char *name)
{
    #if TARGET_OS_IPHONE
        return NULL;
    # else
        ....
       #endif
}

and then i got some more errors like

CC [m]  /Users/dani/Downloads/baresip-ios-bitcode-master/build/aarch64/baresip/modules/coreaudio/recorder.o
modules/coreaudio/recorder.c:78:51: error: too many arguments to function call, expected 2, have 3
        rh(inQB->mAudioData, inQB->mAudioDataByteSize/2, arg);

So I just gave up...

Now you ask what happens if i just use the default audiounit ? I get the following logs

2020-07-15 19:34:47.336010+0300 baresiptest[1728:295101] uaInitErr: 0
aucodec: PCMU/8000/1
aucodec: PCMA/8000/1
auplay: audiounit
ausrc: audiounit
medianat: stun
medianat: turn
medianat: ice
medianat: ice-lite
Populated 2 audio codecs
Populated 0 audio filters
Populated 0 video codecs
Populated 0 video filters
2020-07-15 19:34:47.378361+0300 baresiptest[1728:295101] confModulesErr: 0
2020-07-15 19:34:47.378498+0300 baresiptest[1728:295101] uaAllocErr: 0
2020-07-15 19:34:47.378883+0300 baresiptest[1728:295101] REGISTERED: YESSSSS
ua: using best effort AF: af=AF_INET
...........connecting.................
call: answering call from sip:2000@xxx with 200
stream: update 'audio'
audio: Set audio decoder: PCMU 8000Hz 1ch
audiounit: AudioRouteChange - reason 3
audio: player started with sample format S16LE
audio: Set audio encoder: PCMU 8000Hz 1ch
audiounit: record: enable resampler 8000.0 -> 8000 Hz
2020-07-15 19:34:49.054507+0300 baresiptest[1728:295140] [aid] AudioIssueDetectorNode.cpp:166:Initialize: Caught analyzer graph exception 1718775073 !mrf slice duration must be set before configure. in /Library/Caches/com.apple.xbs/Sources/AudioDSP/AudioDSP-366.38/CoreAudioUtility/Source/CADSP/DSPGraph/DSPGraph_Graph.cpp:628
audio: source started with sample format S16LE
audio tx pipeline:   audiounit ---> PCMU
audio rx pipeline:   audiounit <--- PCMU
main: long async blocking: 656>100 ms (h=0x1046e3f48 arg=0x28152cbf0)
call: SIP Progress: 180 Ringing (/)
call: SIP Progress: 180 Ringing (/)
2020-07-15 19:34:49.089235+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.112152+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.135369+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.158808+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.182018+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2000@xxx: Call established: sip:2000@xxx
2020-07-15 19:34:49.205226+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.228837+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.252263+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.275400+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.299691+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
stream: update 'audio'
audio: Set audio decoder: PCMU 8000Hz 1ch
2020-07-15 19:34:49.322460+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.344792+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
audio: player started with sample format S16LE
audio: Set audio encoder: PCMU 8000Hz 1ch
2020-07-15 19:34:49.682740+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.817741+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.821564+0300 baresiptest[1728:295140] [aurioc] AURemoteIO.cpp:1086:Initialize: failed: -66635 (enable 3, outf< 1 ch,      0 Hz, Float32> inf< 1 ch,   8000 Hz, Int16>)
audiounit: record failed: -66635 (\377\376\373\265)
audio: start_source failed (audiounit.default): Operation not supported by device
sip:2000@xxx:5061;transport=tls: session closed: Operation not supported by device
2020-07-15 19:34:49.840826+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
main: long async blocking: 531>100 ms (h=0x1046e3f48 arg=0x28152cbf0)
2020-07-15 19:34:49.864077+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.887167+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.910383+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.933624+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.956830+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:49.980111+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.003410+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.026485+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.049776+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.073054+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.096461+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.119868+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.143138+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.166310+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.189574+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.212550+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-07-15 19:34:50.236056+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
sip:2000@xxx: session closed: Connection reset by peer
2020-07-15 19:34:50.258984+0300 baresiptest[1728:295168] AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
sip:2000-0x12de04f80@192.168.0.144:57128;transport=tls: Call with sip:2000@xxx terminated (duration: 1 sec)

I tried multiple configs such as: https://gist.github.com/LonestarX91/ff4922fd03b9aa88a1e8dbbb79d78a32 https://gist.github.com/LonestarX91/483bfeb7327af18e5776dcf084d47770 same result And obviously... no audio.

If anyone can please help me get either (coreaudio or audiounit) working, i'll raise them a statue

THANKS

LonestarX91 commented 4 years ago

Switched from that fork to baresip-ios repo, patched contrib.mk for TLS (https://gist.github.com/LonestarX91/e5aca98ab051999499e7b59ec18892b7) and built for 0.6.0 all (which was HEAD). still getting those pesky errors you see, but I also discovered this which seems to encounter my issue https://github.com/baresip/baresip/issues/784... You said you pushed it to master but I don't understand how it still happens... :/

LonestarX91 commented 4 years ago

switched to 0.6.6 baresip, same problem log: https://gist.github.com/LonestarX91/b2974ae949ebb131964ee8ec651533ba config: https://gist.github.com/LonestarX91/447b3e9ff0883ab98027ec6e9a68628b hopes and dreams: image

LonestarX91 commented 4 years ago

Well, finally ended my story with patching baresip 0.6.1 coreaudio with this https://github.com/ylevytskyy/baresip/commit/66b147373b8deab802e2e76e56309ea849c5c2bc

Now audio works fine and I have TLS. Would've been nice to use audiounit but beggars can't be pickers

hope this thread helps anyone

joaoduartemariucio commented 4 years ago

@LonestarX91

Can you describe better how you solved the audio problem? I have the same problem:

2020-08-20 10: 16: 09.913201-0300 APP [652: 138895] [aid] AudioIssueDetectorNode.cpp: 166: Initialize: Caught analyzer graph exception 1718775073! Mrf slice duration must be set before configure. in /Library/Caches/com.apple.xbs/Sources/AudioDSP/AudioDSP-366.38/CoreAudioUtility/Source/CADSP/DSPGraph/DSPGraph_Graph.cpp:628
2020-08-20 10: 16: 09.934585-0300 APP [652: 138895] [aurioc] AURemoteIO.cpp: 1590: Start: AUIOClient_StartIO failed (-66637)
Koshub commented 3 years ago

Got the same error: AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""

with current master version with audiounit configured, without coreaudio.

[0:00:16] audio=63467/63467 (bit/s)    
2020-11-06 14:47:40.231683+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-11-06 14:47:40.256133+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-11-06 14:47:40.279915+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-11-06 14:47:40.302482+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
2020-11-06 14:47:40.325733+0200 AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""
Koshub commented 3 years ago

And probably with open ssl:

AudioIssueDetectorNode.cpp:166:Initialize: Caught analyzer graph exception 1718775073 !mrf slice duration must be set before configure. in /Library/Caches/com.apple.xbs/Sources/AudioDSP/AudioDSP-366.38/CoreAudioUtility/Source/CADSP/DSPGraph/DSPGraph_Graph.cpp:628
Koshub commented 3 years ago

Confirm the errors like:

AUBuffer.h:61:GetBufferList: EXCEPTION (-1) [mPtrState == kPtrsInvalid is false]: ""

but can hear the audio on iOS. Stack:

samirmagnates commented 3 years ago

@Koshub did you find any solution? I am using Xcode11 and Xcode10 with ios 13 and ios 14, I am not getting audio...

Koshub commented 3 years ago

@samirmagnates answered here

alfredh commented 3 years ago

please create a new ticket for this, use latest versions from git HEAD. please provide exact info about software/hardware etc.

tatuanfpt commented 3 years ago

https://github.com/robovm/apple-ios-samples/blob/master/CoreAudioUtilityClasses/CoreAudio/AudioUnits/AUPublic/Utility/AUBuffer.h I tracked down the source which logging it but cant turn it off. For my case, it work normally except annoying message.