Video rotation from Android and iOs

[hoa88]

Hi skymaze,

I have created sample connect from mobile to your library, but when receive video from mobile, video is rotated How to fix this issue. On other platforms, video frame from WebRTC include rotation params, but in your library, i can't see how to get it for rotation.

Please help me check this

[skymaze]

Hi, mediasoup server and client won't rotate media stream, so does pymediasoup you can rotate video when recording with MediaRecorder plus extra FFmpeg options: Media sinks

[hoa88]

Hi, I know that it is possible to rotate based on Media sinks. But how to get rotation value from Video Frame. I see in WebRTC core, video frame include rotation value, but in your library, i can't see how to get it correctlly.

Please help check it

[skymaze]

I see. rotation info is include in consumer's RtpParameters, check headerExtensions for urn:3gpp:video-orientation:

{
    "codecs": [
        {
            "mimeType": "video/VP8",
            "payloadType": 101,
            "clockRate": 90000,
            "parameters": {},
            "rtcpFeedback": [
                {
                    "type": "transport-cc",
                    "parameter": ""
                },
                {
                    "type": "ccm",
                    "parameter": "fir"
                },
                {
                    "type": "nack",
                    "parameter": ""
                },
                {
                    "type": "nack",
                    "parameter": "pli"
                }
            ]
        },
        {
            "mimeType": "video/rtx",
            "payloadType": 102,
            "clockRate": 90000,
            "parameters": {
                "apt": 101
            },
            "rtcpFeedback": []
        }
    ],
    "headerExtensions": [
        {
            "uri": "urn:ietf:params:rtp-hdrext:sdes:mid",
            "id": 1,
            "encrypt": false,
            "parameters": {}
        },
        {
            "uri": "http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time",
            "id": 4,
            "encrypt": false,
            "parameters": {}
        },
        {
            "uri": "http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01",
            "id": 5,
            "encrypt": false,
            "parameters": {}
        },
        {
            "uri": "urn:3gpp:video-orientation",
            "id": 11,
            "encrypt": false,
            "parameters": {}
        },
        {
            "uri": "urn:ietf:params:rtp-hdrext:toffset",
            "id": 12,
            "encrypt": false,
            "parameters": {}
        }
    ],
    "encodings": [
        {
            "ssrc": 926753301,
            "rtx": {
                "ssrc": 926753302
            },
            "scalabilityMode": "S3T3",
            "maxBitrate": 5000000
        }
    ],
    "rtcp": {
        "cname": "fi0Z8g1vHHWl7ZkF",
        "reducedSize": true,
        "mux": true
    },
    "mid": "1"
}

[skymaze]

https://www.etsi.org/deliver/etsi_ts/126100_126199/126114/13.02.00_60/ts_126114v130200p.pdf rtcrtpreceiver It seems that aiortc didn’t deal with CVO payload in the RTP packet. Maybe you should open an issue there.

[hoa88]

Thanks, I have check on mobile and rotated from mobile instead from server