To trigger actions on DTMF rtp events' start and end. (AUD-5522)

[Zoyolin]

Describe the situation I'm using the config.data_cb.receive_audio() callback to act on DTMF events coming from the VOIP (RTP/SIP). It is registered to the esp sip stack through esp_rtc_init(&config). The esp sip stack receives dtmf rtp events which come in a burst of multiple packets. The esp stack parses the event and calls the callback with a DTMF-<digit> data buffer (I call them DTMF-<digit> events below). Parsing the data lets me retrieve the digit which is good information of course. However I'm in need to trigger actions as soon as the first dtmf event packet arrives as well as the end of the event.

Here is the data structure through which the callback is registered :

typedef int (*__esp_rtc_receive_audio)(unsigned char *data, int len, void *ctx);
typedef struct {
    [ . . . ]
    __esp_rtc_receive_audio     receive_audio;
    [ . . . ]
} esp_rtc_data_cb_t;
typedef struct {
    [ . . . ]
    esp_rtc_data_cb_t           *data_cb;            /*!< RTC data callback */
    [ . . . ]
} esp_rtc_config_t;
esp_rtc_handle_t esp_rtc_init(esp_rtc_config_t *config);

Describe the solution you'd like

• I'd like to be assured the DTMF-<digit> event occurs on the first dtmf rtp event packet.
• I'd like to know how much latency can be expected until config.data_cb.receive_audio() is called.
• I'd like to be given a new DTMF- event signalling the end of the dtmf. This one could optionally contain the duration of the dtmf event. DTMF rtp events can send multiple end-packets where the event duration is final. It would be good to receive on the callback something like DTMF-<digit>-<duration> upon receiving the first end-packed of the ongoing RTP EVENT. For my purposes, a simple `DTMF--END event sent with minimum latency upon receiving the first end-packet would be sufficient.

Describe the solution you'd NOT like It could be tempting to delay the DTMF-<digit> event in order for it to contain the final duration, but the timing of the event-start detection is even more critical than that of the event-end.

In the picture below, the the first end-packet is N°2478 that one would send the end event.

OS darwin arm64 23.5.0 Visual Studio Code version 1.90.2 Visual Studio Code language en Visual Studio Code shell /bin/zsh ESP-IDF Extension version 1.8.0 Git version 2.45.0 ESP-IDF version v4.4.5 ADF version v2.6-122-g28736657 Python version 3.12.4 Python's pip version 24.0 Chip used: ESP32-S3 custom board Build system: idf.py

[TempoTian]

In my understanding, what's your needs is have a method to get when does DTMF event end, am I right? For now when receive first DTMF RTP packet it will report "DTMF-ID”. If so I think the most easy way is sent "DTMF-ID-END" when last DTMF event received.

[Zoyolin]

Hello @TempoTian , Indeed, (1) you are right. Yes, receiving a DTMF-ID-END or DTMF-END-ID event in the audio stream would satisfy our needs.

However for the discussion's sake, a better approach could be not to mix the raw audio stream and the DTMF event info. Similarly to pjsip's handling it could be a new callback containing the raw rtp frame or the raw rtp payload only. Potentially as a new member of esp_rtc_data_cb_t. typedef int (*__esp_rtc_receive_dtmf)(unsigned char *data, int len, void *ctx);

 typedef struct {
    __esp_rtc_send_audio        send_audio;
    __esp_rtc_receive_audio     receive_audio;
    __esp_rtc_send_video        send_video;
    __esp_rtc_receive_video     receive_video;
    __esp_rtc_receive_dtmf      receive_dtmf;
} esp_rtc_data_cb_t;

thanks for being so responsive, much appreciated

[TempoTian]

I have implement it according your advice. if __esp_rtc_receive_dtmf provided, DTMF data will be callbacked by the new registered data callback, or else it will use old flow. The new library for esp32s3 is as attached, you can download and verify it, parse data is easy take following code as reference: esp_media_protocols.zip

#include <arpa/inet.h>
typedef struct {
    uint8_t eventid;
    uint8_t volume:6;
    uint8_t reserve:1;
    uint8_t end:1;
    uint16_t duration;
} esp_rtp_dtmf_data_t;

static int _receive_dtmf(unsigned char *data, int len, void *ctx)
{
    esp_rtp_dtmf_data_t* dtmf = (esp_rtp_dtmf_data_t*)data;
    printf("receive id:%d vol:%d end:%d duration:%d\n", 
        dtmf->eventid, dtmf->end, dtmf->volume, ntohs(dtmf->duration));
    return 0;
}

[Zoyolin]

Hi @TempoTian, Thanks a lot for this api! -- I'm on holiday at the moment but will try it soon ! --

[Zoyolin]

Hello @TempoTian Thanks a lot for providing these great APIs, (answering for !1227 too). We've integrated and tested it for our needs since a week and no issue arose 🥳 . Is it possible to know if and when this code could be part of a adf release? Thanks again

[TempoTian]

Code merged to esp-adf-libs now: https://github.com/espressif/esp-adf-libs/commit/831afe75657e0e86564a57e405eb71d8baea4d44 A break change is merged also, change esp_rtc_init/deinit to esp_rtc_service_init/deinit to avoid conflict with IDF rtc clock API.