songbird::driver::tasks::udp_rx: Failed to decode received packet: Opus(BufferTooSmall)

[bmwalters]

Hello. I'm attempting to use this library to receive voice data. However, all the VoicePackets I receive are either empty (when talking) or silent (as expected after I stop talking).

I enabled tracing and also added some logging to audiopus in order to debug this issue. It looks like the failure is caused by a BufferTooSmall error from Opus. Indeed when I replace the buffer size of STEREO_FRAME_SIZE with 9600 in udp_rx.rs the packets are successfully decoded.

I have included a sample Opus packet that fails to decode. Hopefully this can help get to the root cause of the issue.

Speaking state update: user Some(UserId(XXX)) has SSRC XXX, using MICROPHONE
asked to decode Opus packet Packet([123, 131, 47, 46, 11, 32, 223, 10, 132, 144, 147, 61, 250, 196, 24, 255, 86, 235, 228, 71, 243, 149, 78, 24, 128, 150, 72, 7, 206, 93, 149, 61, 231, 176, 14, 4, 251, 236, 76, 161, 243, 240, 254, 63, 150, 186, 156, 239, 217, 149, 78, 10, 136, 24, 76, 222, 60, 244, 212, 213, 126, 89, 217, 60, 149, 38, 195, 197, 112, 44, 116, 98, 82, 143, 103, 252, 247, 122, 206, 191, 213, 191, 197, 172, 61, 140, 89, 87, 177, 77, 11, 180, 232, 51, 126, 101, 251, 10, 136, 17, 103, 209, 212, 138, 70, 9, 157, 122, 0, 3, 213, 115, 44, 134, 54, 104, 224, 211, 152, 139, 227, 72, 166, 93, 244, 174, 159, 241, 41, 150, 96, 44, 85, 230, 71, 166, 249, 186, 168, 77, 83, 206])
Nov 29 00:02:56.230 ERROR songbird::driver::tasks::udp_rx: Failed to decode received packet: Opus(BufferTooSmall).

Thanks for your work on this library.

[FelixMcFelix]

That's surprising, and suggests that the packet is larger than 20ms. I'll have a look later; was the sender a user, or another bot?

On 29 Nov 2020, at 07:19, Bradley Walters notifications@github.com wrote:

 Hello. I'm attempting to use this library to receive voice data. However, all the VoicePackets I receive are either empty (when talking) or silent (as expected after I stop talking).

I enabled tracing and also added some logging to audiopus in order to debug this issue. It looks like the failure is caused by a BufferTooSmall error from Opus. Indeed when I replace the buffer size of STEREO_FRAME_SIZE with 9600 in udp_rx.rs the packets are successfully decoded.

I have included a sample Opus packet that fails to decode. Hopefully this can help get to the root cause of the issue.

Speaking state update: user Some(UserId(XXX)) has SSRC XXX, using MICROPHONE asked to decode Opus packet Packet([123, 131, 47, 46, 11, 32, 223, 10, 132, 144, 147, 61, 250, 196, 24, 255, 86, 235, 228, 71, 243, 149, 78, 24, 128, 150, 72, 7, 206, 93, 149, 61, 231, 176, 14, 4, 251, 236, 76, 161, 243, 240, 254, 63, 150, 186, 156, 239, 217, 149, 78, 10, 136, 24, 76, 222, 60, 244, 212, 213, 126, 89, 217, 60, 149, 38, 195, 197, 112, 44, 116, 98, 82, 143, 103, 252, 247, 122, 206, 191, 213, 191, 197, 172, 61, 140, 89, 87, 177, 77, 11, 180, 232, 51, 126, 101, 251, 10, 136, 17, 103, 209, 212, 138, 70, 9, 157, 122, 0, 3, 213, 115, 44, 134, 54, 104, 224, 211, 152, 139, 227, 72, 166, 93, 244, 174, 159, 241, 41, 150, 96, 44, 85, 230, 71, 166, 249, 186, 168, 77, 83, 206]) Nov 29 00:02:56.230 ERROR songbird::driver::tasks::udp_rx: Failed to decode received packet: Opus(BufferTooSmall). Thanks for your work on this library.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[FelixMcFelix]

It's a 30ms packet (2880 samples when decoding to stereo), which I don't think should ever be sent if documentation is to be believed. I can modify the maximum decoded packet size to include 60ms frames/120ms packets (the largest allowed by the opus spec), but I think I'd really like to know more of the specifics you were testing with.

Can you tell me:

• Whether any users in the voice call were bots?
• Any users were on custom clients?
• Whether you're using this as part of a true bot, selfbot, or a custom client?
• Were any users on the mobile client?
• What voice server location you were using?

I know that Discord does some weird custom re-encoding and mixing for mobile clients, but I'm curious as to whether they've started some experiment somewhere. Voice receive is undocumented, after all.

[bmwalters]

• Two users in the call (human speaker and my bot).
• Human speaker is connected through the web client.
• Songbird is used via a true bot.
• Voice server location: I can reproduce this 100% of the time with the original buffer size. Today I checked and this occurred using us-west359.discord.media:443

[FelixMcFelix]

Took me a while to repro, but the root cause is in fact Discord Web Client on Firefox. I tested with nightly.

Source is naturally minified and unreadable, but this seems to be some incredibly platform-specific hack. I'm not sure why this exists or is needed, since SDP/ICE logs suggest that both Chrome and Firefox are using WebRTC for audio and signalling.

I think that the solution might be to store a known-good size for each SSRC so that most clients pre-allocate 20ms, and only heavy-packet clients need to scan for the correct size once.