Issue about receiving track in C API

[JaoCN]

Hello, I have an issue when using the track function in the C API. I tried to initiate an RTC connection via the web and send the video stream to my program through the track function. However, in the track message callback set by rtcSetMessageCallback(tr, trmessageCallback), I found that the size of the packets I received does not exceed 1100 bytes, which does not meet my expectations. Could you please advise me on how to handle this problem?

This is my program:

static void RTC_API tropenCallback(int id, void *ptr) {
    Peer *peer = (Peer *)ptr;
    peer->connected = true;
    printf("Track: Open\n");
}

static void RTC_API trclosedCallback(int id, void *ptr) {
    Peer *peer = (Peer *)ptr;
    peer->connected = false;
    printf("Track : Closed\n");
}

static void RTC_API trmessageCallback(int id, const char *message, int size, void *ptr) {
    Peer *peer = (Peer *)ptr;
    if (size < 0) { // negative size indicates a null-terminated string
        printf("%d tr Message : %s\n", id, message);
    } else {
        printf("%d tr Message : [binary of size %d]\n", id, size);
    }
}

static void RTC_API trackCallback(int pc, int tr, void *ptr) {
    Peer *peer = (Peer *)ptr;

    char buffer[102400];

    int ret = rtcGetTrackDescription(tr, buffer, 102400);
    if (ret < 0) {
        fprintf(stderr, "rtcGetTrackDescription failed, ret = %d\n", ret);
        return;
    }

    // printf("Track : Received with media description: \n%s\n", buffer);

    char mid[256];
    ret = rtcGetTrackMid(tr, mid, 256);
    if (ret < 0) {
        printf("rtcGetTrackMid failed\n");
        return ;
    }

    printf("mid = %s \n", mid);
    // if (rtcGetTrackMid(tr, mid, 256) < 0 || strcmp(mid, "video") != 0) {
    //  fprintf(stderr, "rtcGetTrackMid failed\n");
    //  return;
    // }

    rtcDirection direction;
    ret = rtcGetTrackDirection(tr, &direction);
    if (ret < 0) {
        printf("rtcGetTrackDirection failed\n");
        return;
    }

    printf("direction : %d\n", direction);
    // if (rtcGetTrackDirection(tr, &direction) < 0 || direction != RTC_DIRECTION_RECVONLY) {
    //  fprintf(stderr, "rtcGetTrackDirection failed\n");
    //  return;
    // }

    peer->tr = tr;

    rtcSetOpenCallback(tr, tropenCallback);
    rtcSetClosedCallback(tr, trclosedCallback);
    rtcSetMessageCallback(tr, trmessageCallback);
}

And in the log, I can see that：

Track: Open
Track: Open
4 tr Message : [binary of size 8]
4 tr Message : [binary of size 96]
4 tr Message : [binary of size 252]
4 tr Message : [binary of size 48]
4 tr Message : [binary of size 36]
4 tr Message : [binary of size 1034]
4 tr Message : [binary of size 996]
4 tr Message : [binary of size 1049]
4 tr Message : [binary of size 1029]
4 tr Message : [binary of size 252]
4 tr Message : [binary of size 1050]
4 tr Message : [binary of size 1048]
4 tr Message : [binary of size 1043]
4 tr Message : [binary of size 1035]
4 tr Message : [binary of size 1054]
4 tr Message : [binary of size 1045]
4 tr Message : [binary of size 1049]
4 tr Message : [binary of size 1017]
4 tr Message : [binary of size 1023]
4 tr Message : [binary of size 992]
4 tr Message : [binary of size 998]
4 tr Message : [binary of size 1047]
4 tr Message : [binary of size 1041]
4 tr Message : [binary of size 1021]
4 tr Message : [binary of size 1018]
4 tr Message : [binary of size 1057]
4 tr Message : [binary of size 1049]
4 tr Message : [binary of size 1046]
4 tr Message : [binary of size 1049]
4 tr Message : [binary of size 1039]
4 tr Message : [binary of size 1057]
4 tr Message : [binary of size 1013]
4 tr Message : [binary of size 1004]
4 tr Message : [binary of size 1012]
4 tr Message : [binary of size 1055]
4 tr Message : [binary of size 1055]
4 tr Message : [binary of size 1039]
4 tr Message : [binary of size 1027]
4 tr Message : [binary of size 1057]
4 tr Message : [binary of size 1050]
4 tr Message : [binary of size 997]

Track successfully opened, and I can receive around 20 data per second. However, the size of the packet is problematic, because I send H264 streams, in general the size of a H264 stream packet is definitely much larger than 1,000 bytes, and I collect these packets, the use of PotPlayer can not be correctly parsed movie and television information!

[paullouisageneau]

This is expected, the track receives the stream as RTP packets, as you can see in the media-receiver example for instance.

The library currently does not have a media handler to depacketize and reassemble H264 frames internally, the corresponding issue is https://github.com/paullouisageneau/libdatachannel/issues/676

[JaoCN]

Thank you for your answer. My idea is to receive the video stream via track and then decode and rendering. You mention that The library currently does not have a media handler to depacketize and reassemble H264 frames internally does it mean I need to make one more process to unpack RTP packets to h264 frames?

I've tried using datachannel to receive H264 streams and it works, but some errors occurred when I deckded it in Embedded devices. So I'm trying to use Track to receive data.

What do you think I should do next? Should I try to depacketize the RTP packets received by the Track to H.264 strams, or should I focus on resolving the decoding errors of the H.264 streams received through the data channel?

Thank you again for your support, and I look forward to your response^ ^