Cannot properly add tsclientlib as a dependency

[snaiperskaya96]

Hello! In first place i'm sorry if i'm gonna say anything weird. I'm a rust super-noob, in fact this is the first time i'm approaching it.

I'm in need to use this library with the audio feature on in my project, so my cargo.toml looks like this

[dependencies.tsclientlib]
git = "https://github.com/ReSpeak/tsclientlib.git"
branch = "master"
features = ["audio"]

Now cargo is crying about tsclientlib having no feature called audio, however i can run cargo build --features audio just fine so i'm super confused.

error: failed to select a version for `tsclientlib`.
    ... required by package `myprj v0.1.0 (/home/jury/Documents/dev/myprj)`
versions that meet the requirements `= 0.1.0` are: 0.1.0

the package `myprj` depends on `tsclientlib`, with features: `audio` but `tsclientlib` does not have these features.

So what is the right way of including this project into my project? Also if i'd manage to get it to work, what format/encoding does the audio client expects the data to be?

Many many thanks, Jury :)

[Flakebi]

Hi! Hm, that’s right, the audio feature does not work at the moment. Sorry for that, I should update the readme. I planned to use gstreamer for all the audio stuff, unfortunately that produces stuttering audio sometimes and I’m not able to figure out why this happens. Thus I plan to ditch gstreamer and directly use an opus decoder and probably SDL to capture and play audio.

So, without the audio feature, you have to decode the audio packets without gstreamer. E.g. audiopus works for that. In the PacketHandler, you get a stream of audio packets and you create an opus decoder for each different stream (so one for every client that sends audio).

Something along these lines:

// The OpusMusic code uses stereo instead
let decoder = Decoder::new(audiopus::SampleRate::Hz48000, audiopus::Channels::Mono)?;

// ...

// data are the packet data
if data.len() == 0 {
    // Resetting decoder
    voice.decoder.reset_state()?;
    return;
}

// 48 kHz and 20 ms = 50 packets per second
let mut output = vec![0f32; 48000 / 50];
let len = voice.decoder.decode_float(data, &mut output, false)?;
// I think this does nothing as the returned length is meaningless, but I don't remember this exactly
output.truncate(len);
// output contains the raw audio data now

[snaiperskaya96]

Hey! Thanks for the speedy answer. That's fine, i was just thinking of ignoring the audio helper and do it myself the manual way, however i need to send audio rather than receive it, i'm assuming it's going to be the other way round, encode using opus and send it away somehow.

Thank you again

[Flakebi]

Right, you encode the audio, craft an audio packet and send it.

// Create a client to server packet
let packet = OutAudio::new(&AudioData::C2S {
    id: 0,
    codec: CodecType::OpusMusic,
    data: data,
});
let future = connection.as_packet_sink().send(packet);

[snaiperskaya96]

Am i wrong in saying that a bunch of those types, like OutAudio, are private and not accessible from outside the package?

[snaiperskaya96]

Yes i was, not sure where i was attempting to get them from, but tsproto_packets::packets::OutAudio is definitely public and accessible :)

[snaiperskaya96]

Hey... Hello again sorry for being such huge PITA. I've got finally something like this

                let packet = tsproto_packets::packets::OutAudio::new(
                    &tsproto_packets::packets::AudioData::C2S {
                        id: 0,
                        codec: tsproto_packets::packets::CodecType::OpusVoice,
                        data: &encoded_data,
                    },
                );

                tokio::run(
                    self.connection
                        .get_packet_sink()
                        .send(packet)
                        .map(|_| ())
                        .map_err(|e| (println!("Failed to send voice packet."))),
                );

And packets are being sent or at least this is what i see

    Sep 19 13:05:00.648 DEBG UdpPacket, content:encoded 116 bytes
 Packet { header: { mac: [0x89, 0x8a, 0xc9, 0x17, 0xf0, 0x7, 0x9f, 0x10], p_id: 670, c_id: 2, type: Voice, flags: ----}, content: [0xa3, 0xdb, 0x7d, 0x4e, 0x9e, 0x69, 0xef, 0xac, 0, 0x84, 0x98, 0x96, 0xb6, 0xa7, 0xef, 0xca, 0x90, 0x2, 0xc4, 0x4a, 0xe9, 0xfa, 0x44, 0xe6, 0xd8, 0x8e, 0x78, 0x10, 0x1, 0x77, 0x77, 0x39, 0x72, 0x73, 0x16, 0x48, 0x61, 0xf5, 0xd5, 0xc9, 0x62, 0x5b, 0xb4, 0xd, 0x14, 0xc9, 0x5e, 0xac, 0x4f, 0x7a, 0x93, 0x7c, 0xce, 0x7d, 0xc5, 0x7a, 0x12, 0xc9, 0xe3, 0x7f, 0x81, 0xf8, 0x60, 0x4c, 0x32, 0x5c, 0x85, 0x50, 0x30, 0x4a, 0x88, 0x52, 0xe8, 0xd, 0x50, 0x5f, 0xa5, 0x48, 0x35, 0x5f, 0x1b, 0x26, 0xd6, 0x64, 0x92, 0xa6, 0x74, 0xa0, 0x74, 0x86, 0x4d, 0x13, 0xd, 0x8c, 0x9b, 0x73, 0x29, 0x63, 0x62, 0x61, 0xf0, 0xb4, 0x69, 0x89, 0x13, 0x3e, 0xfd, 0x7, 0x28, 0x15, 0x5, 0xe7, 0xff, 0x19, 0x8d, 0xe2, 0xf7, 0xdc, 0x21, 0xd8, 0xa4, 0xf4, 0x4f, 0x41, 0x73, 0xa8, 0x40, 0x8a, 0x35, 0xa1, 0x69, 0x14, 0x48, 0xc4, 0xb1, 0xc2, 0x99, 0xff, 0x6c, 0x65, 0x3e, 0x5, 0xf8, 0x69, 0x88, 0xce, 0xd2, 0xba, 0x8, 0xc9, 0x3, 0x79, 0x40, 0x8f, 0x84, 0x46, 0x6a, 0x51, 0x65, 0x75, 0x58, 0x62, 0x67, 0xcf, 0x40, 0x78, 0x48, 0xb6, 0x65, 0x63, 0xfc, 0x9e, 0x6, 0x54, 0xee, 0x88, 0x84, 0x3f, 0xa9, 0xab, 0xae, 0x84, 0x6e, 0xba, 0x86, 0xfb, 0xf6, 0xb2, 0xc3, 0x7c, 0x4f, 0xb7, 0x9, 0x13, 0x65, 0x8a, 0x15, 0x35, 0x59, 0x83, 0, 0x2c, 0x17, 0x80, 0xc1, 0xcc, 0x65, 0xed, 0x1c, 0x99, 0xc8, 0xdb, 0x40, 0x51, 0x22, 0xfe, 0xb8, 0x92, 0x8, 0xc7, 0x5c, 0xad, 0xb9, 0x66, 0x2f, 0xa2, 0x32, 0xe5, 0x55, 0xb3, 0xb5, 0x8c, 0x4f, 0x25, 0x5d, 0xa5, 0xd2, 0xc, 0xc5, 0x23, 0xe5, 0xb5, 0xf6, 0x9e, 0x4d, 0xf3, 0xf9, 0xa1, 0xca, 0x75, 0x2a, 0xd4, 0x20, 0xb6, 0x83, 0xd3, 0x8, 0x65, 0x7, 0xa6, 0xff, 0x9a, 0xda, 0x43, 0xd4, 0xe9, 0xb, 0xdb, 0x43, 0xc2, 0x18, 0, 0x43, 0x1f, 0xde, 0xca, 0x35, 0xec, 0xbc, 0x73, 0xe9, 0x50, 0xf0, 0xe8, 0xa9, 0xde, 0x2, 0xd7, 0x58, 0xb3, 0xcb, 0x37, 0x72, 0x70, 0xef, 0xa3, 0xe2, 0xd6, 0x26, 0xa3, 0x23, 0xa5, 0x12, 0xff, 0x7, 0xa6, 0x81, 0xf1, 0x8d, 0xed, 0xdf, 0x42, 0x84, 0xf3, 0xc2, 0xff, 0x3f, 0x1c, 0xde, 0xe9, 0xd1, 0x5c, 0xab, 0x67, 0x8, 0x66, 0xe0, 0xd7, 0xbd, 0x49, 0x1a, 0xfa, 0x96, 0x73, 0x8e, 0x67, 0x94, 0x9b, 0x1, 0x69, 0x58, 0xd2, 0xd2, 0x4c, 0xfc, 0xf4, 0x92, 0x58, 0xe0, 0x24, 0x8f, 0x8c, 0xa7, 0x29, 0xa2, 0x2c, 0xf0, 0x3a, 0x6, 0x3a, 0xfd, 0xda, 0xa3, 0xf, 0x79, 0xdd, 0x88, 0x6f, 0xea, 0xc5, 0x28, 0xbc, 0x46, 0xcb, 0xb2, 0x41, 0x7a, 0xf3, 0x2f, 0x7e, 0x1b, 0xd0, 0xa6, 0x74, 0x4c, 0x29, 0x81, 0xf3, 0xb1, 0x8a, 0x7a, 0x74, 0x9a, 0x2a, 0x70, 0x80, 0xa6, 0x59, 0xbb, 0xff, 0x4f, 0x5f, 0x9d, 0xf, 0xbb, 0xa9, 0xe0, 0xbb, 0x6c, 0x98, 0x7a, 0xf2, 0x7, 0x90, 0xb2, 0xb5, 0xfa, 0xf7, 0xd6, 0xb9, 0xb2, 0xbe, 0xf, 0x7b, 0xa7, 0xde, 0x57, 0x26, 0xc5, 0xc4, 0xd5, 0x3, 0x29, 0x3f, 0x64, 0x9c, 0xf9, 0xe6, 0xcf, 0x67, 0xd8, 0x53, 0x32, 0xc9, 0x87, 0x10, 0, 0x43, 0x85, 0xd8, 0xcf, 0x7b, 0xf0, 0x6e, 0x17, 0x9, 0xa5, 0xb8, 0x13, 0x17, 0xc, 0x40, 0xe3, 0xc, 0xb9, 0x2d, 0x60, 0x85, 0x90, 0xb1, 0x1d, 0x2f, 0x36, 0x1d, 0x80, 0x74, 0xcc, 0x5, 0x33, 0xa8, 0x86, 0x8a, 0xb5, 0x42, 0x29, 0xa3, 0x57, 0x26, 0x69, 0x8, 0xc7, 0x2f, 0x7a, 0xa3, 0x44, 0xa7, 0x26, 0xc5, 0x60, 0x7e, 0x58, 0x57, 0x8f, 0x73, 0xf, 0x8d, 0x43, 0x5b, 0xb2, 0xbc, 0x9b, 0x54, 0xc2, 0xef, 0x8f, 0x27, 0xbd, 0x7a, 0xce, 0x3f, 0x3e, 0xae, 0xf1, 0x79, 0xec, 0xf4, 0x91, 0xd4, 0x4e, 0x7a, 0xac, 0x4b, 0x12, 0xfa, 0x21, 0x10, 0x54, 0xd8, 0x69, 0x58, 0x4, 0xb5, 0x1e, 0x77, 0x72, 0x23, 0x3, 0xeb, 0x2b, 0x4b, 0x51, 0xdd, 0x68, 0xaa, 0xde, 0xae, 0xce, 0x97, 0x25, 0xe2, 0x7b, 0xd6, 0x61, 0x6f, 0x88, 0xab, 0xbb, 0x9e, 0x35, 0x63, 0x22, 0x88, 0xe3, 0x35, 0xc5, 0xeb, 0xc7, 0x44, 0xc6, 0xbd, 0x14, 0x17, 0x19, 0x4a, 0x4, 0x9c, 0xbc, 0xde, 0x4e, 0x51, 0x5b, 0x15, 0x2d, 0xd3, 0xe7, 0x8, 0x3d, 0x71, 0x5a, 0x3f, 0x6b, 0x41, 0x76, 0x7d, 0x34, 0xe8, 0x5d, 0x6d, 0xf9, 0x34, 0x3e, 0x3, 0xd0, 0xa1, 0xd3, 0xa7, 0x66, 0x40, 0xb5, 0x20, 0x68, 0x39, 0x22, 0xab, 0x46, 0xbb, 0xca, 0xea, 0xbc, 0x17, 0x99, 0xf9, 0xb3, 0xc1, 0xac, 0xc4, 0x8a, 0xf7, 0xfa, 0x4, 0x92, 0x37, 0x3d, 0x19, 0x95, 0x26, 0xba, 0x81, 0xef, 0x4b, 0x26, 0x8d, 0x46, 0xed, 0xc, 0x70, 0x9, 0xd2, 0x4f, 0xa5, 0xfa, 0xc0, 0x53, 0xf2, 0x47, 0x7e, 0x7c, 0xf0, 0xec, 0xb9, 0x75, 0xa0, 0xbd, 0x68, 0x8d, 0xe9, 0x87, 0x24, 0xcd, 0x14, 0xcb, 0x76, 0x1b, 0x66, 0xb2, 0x28, 0x77, 0xe2, 0x21, 0x7e, 0x6e, 0xa8, 0x13, 0x3, 0x66, 0xdb, 0x47, 0x18, 0xb9, 0x2d, 0xd4, 0xaf, 0x12, 0xd4, 0xa9, 0x28, 0xf8, 0xa8, 0x64, 0xbc, 0x1b, 0x43, 0x61, 0x9a, 0x2f, 0xde, 0x9c, 0x68, 0x98, 0xc8, 0x45, 0x7d, 0xba, 0x85, 0x66, 0xd2, 0x8c, 0x96, 0xfe, 0x34, 0x42, 0x76, 0x60, 0x85, 0x66, 0x88, 0x87, 0x8d, 0x5a, 0xf3, 0x92, 0x71, 0x4b, 0x21, 0x59, 0x26, 0xc2, 0x16, 0xe4, 0x8d, 0xef, 0x5f, 0xfc, 0xf4, 0xc, 0xa5, 0x87, 0x1d, 0x60, 0x3f, 0xea, 0xe7, 0xda, 0xd9, 0xf7, 0x28, 0xae, 0x8c, 0xba, 0xb, 0x5b, 0xc0, 0xc8, 0xe5, 0x3d, 0xf7, 0xd, 0x58, 0xe1, 0x28, 0x96, 0xbe, 0xa6, 0xe4, 0x9d, 0xb2, 0x4f, 0x73, 0x89, 0xdb, 0x9, 0xe3, 0x6e, 0xf1, 0x2a, 0xbf, 0x23, 0x37, 0x12, 0x9f, 0xb6, 0xea, 0xec, 0xe2, 0xf0, 0x4e, 0x4, 0x36, 0xbb, 0x95, 0x4f, 0xbc, 0x88, 0x58, 0xe0, 0xa4, 0x14, 0xba, 0x6c, 0xae, 0xdf, 0x86, 0xfd, 0x51, 0x50, 0x4f, 0xdd, 0x73, 0x4f, 0x6e, 0x15, 0x3, 0xdd, 0x14, 0xfa, 0x9, 0xe, 0xf8, 0xc2, 0x3b, 0x21, 0x6a, 0x67, 0x1b, 0x3d, 0xd1, 0xf0, 0x6b, 0x8, 0x20, 0x46, 0xeb, 0x40, 0x50, 0x9d, 0xb7, 0x16, 0x3c, 0xd, 0x2b, 0x4f, 0xf4, 0xbd, 0x8c, 0x52, 0xfc, 0x6c, 0x72, 0xaa, 0x78, 0xce, 0xbb, 0x1d, 0x38, 0xd3, 0x4a, 0x5f, 0xe2, 0x22, 0xa2, 0xe9, 0x1a, 0xe4, 0x89, 0x6d, 0x73, 0x3d, 0xd4, 0x90, 0xf5, 0x81, 0x85, 0x6d, 0x1b, 0xf7, 0xb4, 0x3b, 0xe, 0x6f, 0x98, 0x5f, 0x6b, 0x27, 0x92, 0x32, 0x8a, 0xcb, 0x70, 0xc5, 0x6a, 0x44, 0x82, 0x66, 0xe6, 0x29, 0x7f, 0x5f, 0x22, 0x3c, 0x37, 0x42, 0x4e, 0xf3, 0x23, 0xf6, 0xa2, 0xd9, 0xc5, 0xa3, 0xbe, 0xb6, 0x7c, 0xd5, 0x1f, 0xf7, 0x55, 0xbd, 0x87, 0xdf, 0x67, 0x72, 0x59, 0xfe, 0xe2, 0xfa, 0xa0, 0x5b, 0xa1, 0x37, 0x4f, 0x8c, 0x1f, 0xf2, 0x52, 0xff, 0x63, 0x5c, 0x5f, 0x11, 0x43, 0x2f, 0x43, 0x55] }

However i can't hear or see the bot light up. Is there anything you think i should take care off as well?

[Flakebi]

Hi, no problem. The code looks fine but the packet seems a bit big, usually the opus data is a bit below 200 bytes. How do you encode the packets? You should use a 48 kHz sample rate and 20 ms frames.

[snaiperskaya96]

You made me realise i was sending the whole buffer i was giving to opus to store the data in, which was big by design, so i've shorten that down to the size opus returns and that makes the icon in teamspeak lights up however i have no audio yet.

    Sep 19 15:03:17.753 DEBG Packet, content: Packet { header: { mac: [0, 0, 0, 0, 0, 0, 0, 0], p_id: 0, c_id: 0, type: Voice, flags: ----}, content: [0, 0, 0x4, 0xf4, 0xef, 0xf5, 0x10, 0xab, 0x56, 0xb3, 0x43, 0x76, 0x65, 0x99, 0xdb, 0x21, 0x6c, 0x9f, 0x11, 0x8, 0xdb, 0xad, 0x35, 0xb9, 0x37, 0xef, 0x1, 0x40, 0xe5, 0xdb, 0x1b, 0xbf, 0xeb, 0x34, 0x2d, 0x75, 0xf3, 0xa7, 0x9a, 0xfa, 0xf7, 0x71, 0x8, 0x13, 0x45, 0x2f, 0x83, 0xdb, 0xbc, 0x6a, 0x54, 0x8d, 0x33, 0xc0, 0x30, 0xf7, 0x9e, 0x6f, 0x81, 0xb4, 0x75, 0xb9, 0xa1, 0xc9, 0xd2, 0x11, 0xe0, 0x42, 0x59, 0x9d, 0x15, 0xc7, 0xd5, 0xfd, 0x82, 0xc0, 0x56, 0xcd, 0x4d, 0x16, 0xee, 0x50, 0x92, 0xa, 0xfc, 0xe1, 0x72, 0xb1, 0xf6, 0x3b, 0xcf, 0x30, 0x55, 0x6d, 0x44, 0x36, 0x8b, 0x8d, 0xc3, 0xd9, 0x6a, 0x45, 0x93, 0x9f, 0xf1, 0, 0xac, 0xac, 0x31, 0xa0, 0x9f, 0xfa, 0x11, 0xe2, 0xc1, 0x36, 0xbe, 0x7f, 0xab, 0xad, 0xda, 0x7b, 0x89, 0x95, 0x3a, 0x2f, 0x48, 0x73, 0xf0, 0x29, 0x7f] }
    Sep 19 15:03:17.755 DEBG UdpPacket, content: Packet { header: { mac: [0x2, 0x76, 0x18, 0x15, 0x85, 0xc1, 0x17, 0xc2], p_id: 3049, c_id: 2, type: Voice, flags: ----}, content: [0x45, 0xa3, 0x37, 0xb4, 0x1a, 0xea, 0x93, 0x69, 0x7a, 0x6f, 0xdc, 0x2, 0x48, 0xcd, 0x3f, 0x36, 0x62, 0x83, 0x26, 0x95, 0x1c, 0x87, 0x8e, 0xc3, 0x16, 0xd0, 0x8f, 0x2d, 0xd4, 0xd5, 0xb0, 0xba, 0xc8, 0xa7, 0x47, 0x77, 0x4a, 0x46, 0xdd, 0x34, 0x26, 0x38, 0x1f, 0x4f, 0x9, 0x6, 0x57, 0x19, 0x7f, 0x4c, 0xc4, 0x8c, 0x7, 0xc8, 0x87, 0xc4, 0x5c, 0xef, 0x3b, 0xa4, 0x4, 0x1a, 0xc7, 0xa6, 0xd6, 0x27, 0xd3, 0xad, 0x15, 0x13, 0xeb, 0x5a, 0x31, 0xde, 0xf5, 0xea, 0x43, 0xd7, 0xb, 0x72, 0x3c, 0xf8, 0xb9, 0xc7, 0x12, 0xc8, 0x9c, 0x33, 0x75, 0xc1, 0x8f, 0x11, 0x81, 0xdf, 0xe1, 0x1d, 0x3, 0xf4, 0xd6, 0xcb, 0x3c, 0x26, 0xfc, 0x2c, 0xcf, 0x85, 0x27, 0x6d, 0x89, 0x23, 0xc3, 0xf6, 0x68, 0x81, 0x77, 0x97, 0x64, 0xd7, 0x14, 0xfb, 0x21, 0x22, 0x13, 0x20, 0x27, 0x58, 0x9b, 0x4f, 0x49, 0x49, 0xdc] }

To answer your question, i've got a stream of audio that works just fine (i can dump that into audacity and it plays well at 48khz), i encoded it like this

            opus::Encoder::new(48000, opus::Channels::Stereo, opus::Application::Voip).unwrap(),
----
----
            len = self.encoder().encode(x, &mut encoded_data).unwrap();

then i grab encoded_data, trim it down to len and send it away.

Edit: Turns out it was the sample size to be a little too small. Pushing it up to const sample_size: usize = 48 * 20 * 2; made it work-ish (i was assuming mono when the source is stereo).

Now the audio is somewhat hearable but sluggish, i believe this has to do with the difference in frequency between the source (44.1k) and the closest one libopus likes (48k).