Streamer packet discarded

[fm-at-ct]

Hi, I'm playing with your library that could be used in a project I'm working on (I'd like to avoid using the official WebRTC library from Google since is really big), specifically I've played with the streamer example, adding functionality (like sending audio captured from mic) to better understand how to use the library... BUT I've found a problem with the example that I'm not able to solve: there are audio packet discarded.

I noticed this problem while streaming an audio file with music inside... upon investigation I found that every chromium based browser has a useful page "chrome://webrtc-internals/" that shows a lot of information about the active WebRTC sessions, and there (in the RTCInboundRTPAudioStream_1 table) I found the packetsDiscarded counter (searching online I found the W3C spec that states that this counter represent the packets discarded due to being received too late or too early to be played).

The problem arises even with the audio present in the samples directory (albeit is less evident because the sound has a low dynamic). The problem is present in my customized streamer example as well as in the original one.

Any idea on how to solve this problem? The problem is in the program structure or in the library, since I've tested playing the same audio file using another WebRTC framework (go based) and there are none packet discarded.

PS: I've tested the program on a Linux machine (Ubuntu based), streaming locally.

[paullouisageneau]

I think this is caused by the naive way the streamer example handles elapsed time, everything is based on wall clock and timings relies on sleep which is very imprecise. This is a problem particularly because people use this demo as reference, so it should reflect good practice.

For instance, in the sending loop, it waits for a period, then sends a sample timestamped according to the wall clock. This is incorrect, as it should use the actual timestamp of the sample, and induces significant drift, possibly causing such audio issues.

[fm-at-ct]

Hi, thanks for the response. I figured that the sleep() was a cause for time drift (confirmed by some debug messages with timestamp in micros), so I changed the final part of the method Stream::unsafePrepareForSample() from

    ...
    auto elapsed = currentTime - startTime;
    if (nextTime > elapsed) {
        auto waitTime = nextTime - elapsed;
        mutex.unlock();
        usleep(waitTime);
        mutex.lock();
    }
    return {ss, sst};

to

    const auto elapsed = currentTime - startTime;
    if (nextTime > elapsed) {
        const auto waitTime = nextTime - elapsed;
        mutex.unlock();
        const auto wakeTime = currentTime + waitTime;
        struct timespec ts;
        ts.tv_sec = wakeTime / (1000*1000);
        ts.tv_nsec = (wakeTime - ts.tv_sec * (1000*1000)) * 1000;
        int res = 0;
        do {
            res = clock_nanosleep(CLOCK_MONOTONIC, TIMER_ABSTIME, &ts, NULL);
        } while (res != 0);
        mutex.lock();
    }
    return {ss, sst};

and

uint64_t currentTimeInMicroSeconds() {
    struct timeval time;
    gettimeofday(&time, NULL);
    return uint64_t(time.tv_sec) * 1000 * 1000 + time.tv_usec;
}

to

uint64_t currentTimeInMicroSeconds() {
    uint64_t micros;
    struct timespec tspec;
    clock_gettime(CLOCK_MONOTONIC, &tspec);
    micros = (uint64_t(tspec.tv_sec) * 1000*1000) + (tspec.tv_nsec / 1000);
    return micros;
}

but the problem is present anyway... I need to further investigate, maybe there is something wrong elsewhere!

[paullouisageneau]

Your changes look good, did you confirm timestamps were more precise?

[fm-at-ct]

Hi, yes the timestamps was more precise, but the problem "was" present anyway... lately I've not worked on that project so I haven't investigate more on the problem... I home to come back on the project soon!