not send null frame when micphone mute ?

[jasonz1987]

Describe the bug

I implemented a logic in my application. After I established a call, I would disable the local microphone to implement a function similar to one-way intercom. However, after my testing, I found that although the mute effect can be achieved, the After capturing the packet, I found that the device did not send a silent packet to the other party. I don’t know whether it is a BUG or a problem with my application?

this is my mute code

 for (int i = 0; i < info.getMedia().size(); i++) {
            Media media = getMedia(i);
            CallMediaInfo mediaInfo = info.getMedia().get(i);

            if (mediaInfo.getType() == pjmedia_type.PJMEDIA_TYPE_AUDIO
                    && media != null
                    && mediaInfo.getStatus() == pjsua_call_media_status.PJSUA_CALL_MEDIA_ACTIVE) {

                AudioMedia audioMedia = AudioMedia.typecastFromMedia(media);

                try {

                    if (mute)  audioMedia.adjustRxLevel((float) 0);
                    else audioMedia.adjustRxLevel((float) 2);

                } catch (Exception exc) {
                    BSipLogger.error(TAG, "setMute: error while connecting audio media to sound device", exc);
                    throw new RuntimeException(exc);
                }
            }
        }

i also try this code to mute ,it's also not work

 for (int i = 0; i < info.getMedia().size(); i++) {
            Media media = getMedia(i);
            CallMediaInfo mediaInfo = info.getMedia().get(i);

            if (mediaInfo.getType() == pjmedia_type.PJMEDIA_TYPE_AUDIO
                    && media != null
                    && mediaInfo.getStatus() == pjsua_call_media_status.PJSUA_CALL_MEDIA_ACTIVE) {

                AudioMedia audioMedia = AudioMedia.typecastFromMedia(media);

                // connect or disconnect the captured audio
                try {
                    AudDevManager mgr = account.instance.mEndpoint.audDevManager();
                    if (mute) mgr.getCaptureDevMedia().stopTransmit(audioMedia);
                    else mgr.getCaptureDevMedia().startTransmit(audioMedia);

                } catch (Exception exc) {
                    BSipLogger.error(TAG, "setMute: error while connecting audio media to sound device", exc);
                    throw new RuntimeException(exc);
                }
            }
        }

this is no any frame after "mute"

Steps to reproduce

1. A call B, tcpdump on B,
2. B set mute when call connected,
3. B is mute ,but not null frame send to B

PJSIP version

2.13

Context

android 9.0

Log, call stack, etc

ipInstance           com.byteee.fw.app                    E  setMute:true
2024-03-21 16:29:00.226 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:00                  speex  warning: Had to discard a playback frame (your application is buggy and/or got xruns)
2024-03-21 16:29:00.304 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:00                  speex  warning: Auto-filling the buffer (your application is buggy and/or got xruns)
2024-03-21 16:29:01.717 24383-24433 libEGL                  com.byteee.fw.app                    W  EGLNativeWindowType 0x6f953d8010 disconnect failed
2024-03-21 16:29:02.190 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:02                  speex  warning: Had to discard a playback frame (your application is buggy and/or got xruns)
2024-03-21 16:29:02.226 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:02                  speex  warning: Auto-filling the buffer (your application is buggy and/or got xruns)
2024-03-21 16:29:03.181 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:03       strm0x6f95225428  Jitter buffer empty (prefetch=0), plc invoked
2024-03-21 16:29:03.221 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:03       strm0x6f95225428  Jitter buffer starts returning normal frames (after 2 empty/lost)
2024-03-21 16:29:03.787 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:03           silencedet.c  Re-adjust threshold (in silence)to 22
2024-03-21 16:29:04.185 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:04       strm0x6f95225428  Start talksprut..
2024-03-21 16:29:04.192 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:04       strm0x6f95225428  Starting silence
2024-03-21 16:29:05.553 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:05                  speex  warning: Had to discard a playback frame (your application is buggy and/or got xruns)
2024-03-21 16:29:05.785 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:05                  speex  warning: Auto-filling the buffer (your application is buggy and/or got xruns)
2024-03-21 16:29:06.833 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:06                  speex  warning: Had to discard a playback frame (your application is buggy and/or got xruns)
2024-03-21 16:29:06.904 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:06                  speex  warning: Auto-filling the buffer (your application is buggy and/or got xruns)
2024-03-21 16:29:07.232 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:07                  speex  warning: Had to discard a playback frame (your application is buggy and/or got xruns)
2024-03-21 16:29:07.308 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:07       strm0x6f95225428  Jitter buffer empty (prefetch=0), plc invoked
2024-03-21 16:29:07.423 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:07                  speex  warning: Auto-filling the buffer (your application is buggy and/or got xruns)
2024-03-21 16:29:07.621 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:07       strm0x6f95225428  Jitter buffer starts returning normal frames (after 15 empty/lost)
2024-03-21 16:29:07.713 24383-24571 PJSIP android_track     com.byteee.fw.app                    D  TRACE: 16:29:07                  speex  warning: Had to discard a playback frame (your application is buggy and/or got

[nanangizz]

For one way communication, perhaps you can use media direction setting in call setting. There may still some RTP packets sent at the beginning for NAT hole puching and then the rate is reduced significantly to maintain NAT.

FYI, your first method relies on codec's VAD & DTX capability (try to search "VAD" in docs). For the second method, normally some (silent) packets will still be sent at some interval, e.g: every 5 seconds, for keep the NAT binding open.

[jasonz1987]

If I use the first method, the account configuration has enabled VAD configuration. this is my code:

   acfg.getMediaConfig().setStreamKaEnabled(true);
        acfg.getNatConfig().setUdpKaIntervalSec(3);

but silent packets will still be sent at every 5 seconds, so, how i control the interval ?

[nanangizz]

For codec's VAD/DTX, please check macro PJMEDIA_CODEC_MAX_SILENCE_PERIOD to be set in config_site.h.

[jasonz1987]

@nanangizz Tks very much, but there still hava a strange issue, Although I set up silent operation, and most of the time it will send silent frames, but sometimes it does not work properly. I am not sure what caused VAD to not send normally. The attachment is that I made several calls. After that, abnormal packet capture occurred. I don’t know if this is related to the device or the software? rtp11.pcap.zip

from the image ,the first time send client frame at No 9198,but threre is not any mark frame after 5 seconds

[sauwming]

You can try to debug into or add log information in pjmedia/src/pjmedia/g711.c (search for the text PJMEDIA_CODEC_MAX_SILENCE_PERIOD).

[jasonz1987]

this is my logs, the logs seemed that vad is work normally, but it still have not send any null frame ?

  if (priv->vad_enabled) {
        pj_bool_t is_silence;
        pj_int32_t silence_period;

        silence_period = pj_timestamp_diff32(&priv->last_tx,
                                             &input->timestamp);

        is_silence = pjmedia_silence_det_detect(priv->vad, 
                                                (const pj_int16_t*) input->buf, 
                                                (input->size >> 1), NULL);

        TRACE_((THIS_FILE, "Silence detected: %d", is_silence)); // 添加这行日志以查看是否检测到静音

        TRACE_((THIS_FILE, "PJMEDIA_CODEC_MAX_SILENCE_PERIOD: %d", PJMEDIA_CODEC_MAX_SILENCE_PERIOD)); // 打印 PJMEDIA_CODEC_MAX_SILENCE_PERIOD 的值
        TRACE_((THIS_FILE, "priv->last_tx: %d", priv->last_tx)); // 打印 priv->last_tx 的值
        TRACE_((THIS_FILE, "input->timestamp: %d", input->timestamp)); // 打印 input->timestamp 的值

        if (is_silence && 
            (PJMEDIA_CODEC_MAX_SILENCE_PERIOD == -1 ||
             silence_period < PJMEDIA_CODEC_MAX_SILENCE_PERIOD*8000/1000))
        {
            TRACE_((THIS_FILE, "Silence period within threshold. Sending silence."));
            output->type = PJMEDIA_FRAME_TYPE_NONE;
            output->buf = NULL;
            output->size = 0;
            output->timestamp = input->timestamp;
            return PJ_SUCCESS;
        } else {
            TRACE_((THIS_FILE, "Not silence or silence period exceeds threshold. Proceeding with encoding."));
            priv->last_tx = input->timestamp;
        }
    }

1(1).txt

rtp1(1).pcap.zip

[jasonz1987]

Due to the needs of project development, I gave up using VAD for RTP keep-alive, and instead used the PJMEDIA_STREAM_ENABLE_KA mechanism for keep-alive. Currently, this system works normally and can send empty packets regularly according to the set cycle.

[sauwming]

Understood. I will close the issue then.