Send video to the server via RTMP

[Rishat7c]

Implemented sending video to the server using the library ( https://github.com/pedroSG94/rtmp-rtsp-stream-client-java ) Wrote the constructor to send - everything works ( https://pastebin.com/haWHc3T2 ) In MainActivity, I start the stream as follows ( https://pastebin.com/5aYV16Gg )

After a couple of seconds, the video sent to the server begins to deteriorate. Quality deteriorates and huge pixels appear. This video is received by RTMP stream and is broadcast on video player ( https://www.youtube.com/watch?v=-TuDgkB7OsY )

P.S.

• videoEncoder class ( https://pastebin.com/XJGbAg6W )
• srsMuxer class ( https://pastebin.com/LJ5jPfKd )
• lightOpenGlView class ( https://pastebin.com/sZLe4eq0 )

What could be the problem? And where to start digging?

[Rishat7c]

or should I contact the DJI Developer Forum?

[Michael-DJI]

@Rishat7c thanks for your feedback, currently we are still tuning the interfaces that associated with live stream, so we may not have enough bandwidth to look into your problem, pls wait for our new version, thanks!

[Rishat7c]

@Michael-DJI Thank you for your reply. Can the functionality of sending video to the server via RTMP stream appear in the future?

p.s. Can you tell me in which direction to develop in order to transmit the video? :-)

[johnsonrobot]

@Michael-DJI @Rishat7c I want to understand how to finish. do you buy other dji product except the drone? And I want to understand how to get the drone of camera's data.

[Rishat7c]

@johnsonrobot
I answered your question in my youtube channel

@Michael-DJI Maybe the video received on the server is corrupted for the following reasons

1) Maybe there is not a keyframe generated in your decoder? 2) Theoretical, is it possible to realize the transfer of video from the phone to the UVC protocol?

[antman-s]

@Rishat7c @johnsonrobot I finished the function of Living broadcast with RTMP. Generally, if you connect rtmp successfully, you should focus on the direction of 'encoder' ,'decoder' and 'colorformat' that you choosed. it will caused difficult problems if you choose incorrectly.

[Rishat7c]

@johnchenn At the moment the server is broadcast in bad quality. I think the problem in the encoder' or in the decoder'

https://www.youtube.com/watch?v=8sPIZa4_T9I

On the Internet, I found the original encoder file for a normal phone camera and screwed it to the DJI. Source file ( https://pastebin.com/9bW3F0zV )

p.s. Can you share your encoder?

[Rishat7c]

@Michael-DJI @johnchenn I need know YUV format get from this method: https://github.com/DJI-Mobile-SDK-Tutorials/Android-VideoStreamDecodingSample/blob/master/android-videostreamdecodingsample/src/main/java/com/dji/videostreamdecodingsample/MainActivity.java#L335 YUV420SP, YUV420P, NV21, YV12, etc

[Rishat7c]

Are there really no specialists who can answer this question?

[antman-s]

@Rishat7c You should change the decode name in DJI SDK Demo "DJIVideoStreamDecoder.java" line 454 with "decoder = MediaCodec.createByCodecName(MediaSupportUtil.getInstance().getDecoderName());" and choose the pair of encoder which used in your smtp publisher encoder like "encoder = MediaCodec.createByCodecName(MediaSupportUtil.getInstance().getEncoderName());".

ps: thr encoder and decoder must be a pair，such as "OMX.qcom.video.encoder.avc, OMX.qcom.video.decoder.avc", and choose the common colorformat they supported.

[antman-s]

import android.media.MediaCodecInfo; import android.media.MediaCodecList;

import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map;

import static android.media.MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar; import static android.media.MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar;

/**

• Created by dell on 2018/1/5.
• 编解码器工具类，推流出问题优先从此类排查

• */

public class MediaSupportUtil { public static final String VIDEO_ENCODING_FORMAT = "video/avc";

private SupportedCodec mCodec;

private static MediaSupportUtil instance;

private MediaSupportUtil(){}

public static MediaSupportUtil getInstance(){
    if (instance == null){
        instance = new MediaSupportUtil();
    }

    return instance;
}

public String getEncoderName() {
    return mCodec == null ? "" : mCodec.encoderName;
}

public String getDecoderName() {
    return mCodec == null ? "" : mCodec.decoderName;
}

public int getKeyColorFormat() {
    return mCodec == null ? 21 : mCodec.colorFormat;
}

public boolean streamSupported() {
    return mCodec != null;
}

private class SupportedCodec{
    String type;
    String encoderName;
    String decoderName;
    int colorFormat;
}

/**
 * 检查是否有一对同样的编解码器
 * 规则：
 * 1、从设备选择所有可用的video/avc编解码器
 * 2、寻找配对的编解码器
 * 3、设置编解码器支持的颜色
 * 4、从所有可选配对的编解码器中根据优先条件选择最终使用的
 *
 * qcom优先，IMG, google均可，否则取第一个。最后MTK
 * 颜色：
 * 1. 默认颜色21
 *
 */
public void initCodec() {

    List<String> encodeList = new ArrayList<>();
    List<String> decodeList = new ArrayList<>();
    getAvailableVideoCodec(encodeList, decodeList);

    List<SupportedCodec> codecList = new ArrayList<>();

    Map<String, String[]> pairMap = getSupportedVideoCodec(encodeList, decodeList);
    for (Map.Entry<String, String[]> entry : pairMap.entrySet()) {
        if (isSupportedColorFormat(entry.getValue()[0], entry.getValue()[1], COLOR_FormatYUV420SemiPlanar)){
            SupportedCodec codec = new SupportedCodec();
            codec.type = entry.getKey();
            codec.encoderName = entry.getValue()[0];
            codec.decoderName = entry.getValue()[1];
            codec.colorFormat = COLOR_FormatYUV420SemiPlanar;
            codecList.add(codec);
        }else if (isSupportedColorFormat(entry.getValue()[0], entry.getValue()[1], COLOR_FormatYUV420Planar)){
            SupportedCodec codec = new SupportedCodec();
            codec.type = entry.getKey();
            codec.encoderName = entry.getValue()[0];
            codec.decoderName = entry.getValue()[1];
            codec.colorFormat = COLOR_FormatYUV420Planar;
            codecList.add(codec);
        }
    }

    if (codecList.isEmpty())
        return;

    for (SupportedCodec codec : codecList){
        if (codec.type.equals("qcom")){
            mCodec = codec;
            return;
        }
    }

    for (SupportedCodec codec : codecList){
        if (codec.type.equals("google")){
            mCodec = codec;
            return;
        }
    }

    mCodec = codecList.get(0);
}

private boolean isSupportedColorFormat(String encoderName, String decoderName, int keyColorFormat){
    boolean encoderSupported = false;
    boolean decoderSupported = false;

    int codecCount = MediaCodecList.getCodecCount();
    for (int i = 0; i < codecCount; i++) {
        MediaCodecInfo mci = MediaCodecList.getCodecInfoAt(i);
        if (!mci.getName().equals(encoderName) && !mci.getName().equals(decoderName))
            continue;

        String[] types = mci.getSupportedTypes();
        for (String type : types) {
            if (type.equalsIgnoreCase(VIDEO_ENCODING_FORMAT)) {
                MediaCodecInfo.CodecCapabilities capabilities = mci.getCapabilitiesForType(VIDEO_ENCODING_FORMAT);
                for (int colorFormat : capabilities.colorFormats) {
                    if (colorFormat == keyColorFormat) {
                        if (mci.getName().equals(encoderName))
                            encoderSupported = true;
                        else if (mci.getName().equals(decoderName))
                            decoderSupported = true;
                    }
                }
            }
        }
    }

    return encoderSupported && decoderSupported;
}

/**
 * 读取成对的编/解码器
 */
private Map<String, String[]> getSupportedVideoCodec(List<String> encodeList, List<String> decodeList){
    Map<String, String[]> codecMap = new HashMap<>();

    for (int i = 0; i < encodeList.size(); i++){
        String encoderName = encodeList.get(i);
        for (int j = 0; j < decodeList.size(); j++){
            String decoderName = decodeList.get(j);
            if (isPair(encoderName, decoderName)){
                String type = decoderName.substring(4, decoderName.indexOf(".", 4));
                String[] codecArray = codecMap.get(type);
                if (codecArray == null) {
                    codecArray = new String[2];
                    codecMap.put(type, codecArray);
                }

                codecArray[0] = encoderName;
                codecArray[1] = decoderName;
            }
        }
    }

    return codecMap;
}

/**
 * 成对出现的只有2个字符不同
 * @param encoderName OMX.google.h264.encoder
 * @param decoderName OMX.google.h264.decoder
 */
private boolean isPair(String encoderName, String decoderName){
    if (encoderName.length() != decoderName.length())
        return false;
    int differentCount = 0;
    for (int index = 0; index < encoderName.length(); index++){
        if (encoderName.charAt(index) != decoderName.charAt(index))
            differentCount++;
    }

    return differentCount == 2;
}

/**
 * 从设备读取所有可用的 Video 编/解码器
 */
private void getAvailableVideoCodec(List<String> encodeList, List<String> decodeList){
    int codecCount = MediaCodecList.getCodecCount();
    for (int i = 0; i < codecCount; i++) {
        MediaCodecInfo mci = MediaCodecList.getCodecInfoAt(i);
        String[] types = mci.getSupportedTypes();
        for (String type : types) {
            if (type.equalsIgnoreCase(VIDEO_ENCODING_FORMAT)) {
                if (mci.isEncoder())
                    encodeList.add(mci.getName());
                else
                    decodeList.add(mci.getName());
            }
        }
    }
}

}

[chrisail]

Hi.

@Rishat7c and @johnchenn, could you share your code? Pastebin code is no more available.

I also tried to use hxl-dy code and raullalves code. The problem is that both use YUV data, the consequence is that Surface View stop displaying drone video.

Could someone share code on how to:

• stream video with RTSP(like spydroid) or RTMP(like DJI GO 4) protocols,
• and at the same time display drone video.

Thank you.

[Rishat7c]

@johnchenn I initialized your encoder as you wrote above. I connected your class.

Now I'm confused, in what direction do I go further to implement sending this data to the server via RTMP?

[Rishat7c]

@johnchenn

could you share your mail? That I showed you my problem. Thank you for attention

[hartmann8]

@Rishat7c Thank you for your contribution. I have a similar problem as I want to send the video of the drone camera to a server via RTMP. Unfortunately I'm completely new to RTMP, VideoDecoding etc. . Your pastebin links do not work anymore. Can you share your code again?

Greetings from a frustrated DJI SDK user

[oscarmore2]

I had similar problem with while it display the motion picture. My sdk version 4.8.1. And I solved by using follow encoder setting:

        format = MediaFormat.createVideoFormat(MIME, 1280, 720);
        format.setInteger(MediaFormat.KEY_BIT_RATE, 1200);
        format.setInteger(MediaFormat.KEY_BITRATE_MODE, MediaCodecInfo.EncoderCapabilities.BITRATE_MODE_CQ);
        format.setInteger(MediaFormat.KEY_FRAME_RATE, KEY_FRAME_RATE);
        //format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 0);
        format.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible);
        format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 0);

[Michael-DJI]

in 4.9 we have some interfaces to stream the video to a RTMP server, here is the sample code: https://github.com/dji-sdk/Mobile-SDK-Android/blob/master/Sample%20Code/app/src/main/java/com/dji/sdk/sample/demo/camera/LiveStreamView.java

[Rishat7c]

@Michael-DJI @dji-dev Thank you and dji for the great work you have done!

Will this functionality be implemented on swift under ios?

[Michael-DJI]

@Rishat7c it has been inside the pool of requirements, but still need be reordered and then developed in future version.