RTC: Fix video and audio track pt_ is not change in player before publisher. v5.0.207 v6.0.111

[w41203208]

For WebRTC: when player before publisher, it will happen track pt didn't change.

• At source change step, change track pt

---

Co-authored-by: john hondaxiao@tencent.com

[xiaozhihong]

Thank you for submitting the pull request. Could you describe the scenario in which you encountered this issue that results in playback failure?

TRANS_BY_GPT4

[w41203208]

I play a WebRTC URL on the Unity platform first, and then I publish a WebRTC URL on the Web platform. It can occur that the WebRTC track's payload type (PT) does not match the PT of the player, but rather corresponds to the PT of the publisher. During WebRTC streaming, at the player, the WebRTC system ensures whether the track has the correct player PT for streaming.

In particular, since I play the WebRTC stream before publishing, SRS defaults to using the default PT (Chrome 102) at the player. However, the default PT does not match the PT of the player's track

[winlinvip]

The PT and SSRC do not actually need to be altered; they can be used as provided by the stream. Perhaps consider modifying the SSRC as well while at it?

TRANS_BY_GPT4

[w41203208]

At this code ( negotiate_play_capability ), if I play a WebRTC stream on Unity platform, pt_ofpublisher of player track will be from Unity track pt and it's not same with web platform ( publisher ).

for (int j = 0; j < (int)track_descs.size(); ++j) {
            SrsRtcTrackDescription* track = track_descs.at(j)->copy();

            // We should clear the extmaps of source(publisher).
            // @see https://github.com/ossrs/srs/issues/2370
            track->extmaps_.clear();

            // We should clear the rtcp_fbs of source(publisher).
            // @see https://github.com/ossrs/srs/issues/2371
            track->media_->rtcp_fbs_.clear();

            // Use remote/source/offer PayloadType.
            track->media_->pt_of_publisher_ = track->media_->pt_;
            track->media_->pt_ = remote_payload.payload_type_;

            vector<SrsMediaPayloadType> red_pts = remote_media_desc.find_media_with_encoding_name("red");
            if (!red_pts.empty() && track->red_) {
                SrsMediaPayloadType red_pt = red_pts.at(0);

                track->red_->pt_of_publisher_ = track->red_->pt_;
                track->red_->pt_ = red_pt.payload_type_;
            }

            track->mid_ = remote_media_desc.mid_;
            uint32_t publish_ssrc = track->ssrc_;

            vector<string> rtcp_fb;
            remote_payload.rtcp_fb_.swap(rtcp_fb);
            for (int j = 0; j < (int)rtcp_fb.size(); j++) {
                if (nack_enabled) {
                    if (rtcp_fb.at(j) == "nack" || rtcp_fb.at(j) == "nack pli") {
                        track->media_->rtcp_fbs_.push_back(rtcp_fb.at(j));
                    }
                }
                if (twcc_enabled && remote_twcc_id) {
                    if (rtcp_fb.at(j) == "transport-cc") {
                        track->media_->rtcp_fbs_.push_back(rtcp_fb.at(j));
                    }
                    track->add_rtp_extension_desc(remote_twcc_id, kTWCCExt);
                }
            }

            track->ssrc_ = SrsRtcSSRCGenerator::instance()->generate_ssrc();

            // TODO: FIXME: set audio_payload rtcp_fbs_,
            // according by whether downlink is support transport algorithms.
            // TODO: FIXME: if we support downlink RTX, MUST assign rtx_ssrc_, rtx_pt, rtx_apt
            // not support rtx
            if (true) {
                srs_freep(track->rtx_);
                track->rtx_ssrc_ = 0;
            }

            track->set_direction("sendonly");
            sub_relations.insert(make_pair(publish_ssrc, track));
        }

And then, becuase pt is not same with publisher pt, this code will not execute. Cause Unity platform to receive incorrect pt. WebRTC will regard pkt as an error and unhandled it at frontend.

 // Should update PT, because subscriber may use different PT to publisher.
    if (track_desc_->media_ && pkt->header.get_payload_type() == track_desc_->media_->pt_of_publisher_) {
        // If PT is media from publisher, change to PT of media for subscriber.
        pkt->header.set_payload_type(track_desc_->media_->pt_);
    } else if (track_desc_->red_ && pkt->header.get_payload_type() == track_desc_->red_->pt_of_publisher_) {
        // If PT is RED from publisher, change to PT of RED for subscriber.
        pkt->header.set_payload_type(track_desc_->red_->pt_);
    } else {
        // TODO: FIXME: Should update PT for RTX.
    }

[xiaozhihong]

The pull request has been merged. Thank you very much for your contribution. Due to some modifications in the code style and the integration of other changes, a force push was necessary. As a result, the commits within this pull request will appear as if they were submitted by me. I hope you don't mind.

TRANS_BY_GPT4