Artifacts on video

[lekting]

Your environment.

• Version: GoLang 1.17.3, Pion WebRTC 3.1.11
• Browser: Chrome, 96.0.4664.110

Some text:

Greetings, I made the implementation of streaming using WebRTC for a quick start of the stream. Everything was done, but a problem arose, it was that artifacts appeared randomly on the video (see image). For streaming, I use GStreamer to convert from rtp to rtmp. But, I found out that the problem is created even at the moment of reading the track packet from the buffer. The question is, how can this be fixed? i

Accordingly, the further the user is from the server, the more often artifacts are, but, on a pure WebRTC browser implementation (peer-server-peer), this is not the case.

I read in different sources that perhaps the problem is that webrtc has the ability to constantly change the bitrate and just at the junction of the change it creates this artifact and also read that the built-in utilities in webrtc can fix it.

Code:

func createWebRTCConn(saver *webmSaver, channel string) *Connection {

    // create stream pipeline
    streamPipeLine := <-createOwnStreamPipeline()

    log.Println("created streamPipeline", streamPipeLine)

    // if our streampipeline got an error - throw
    if streamPipeLine == nil {
        log.Println("Error")
        return nil
    }

    // Prepare the configuration
    config := webrtc.Configuration{
        ICEServers: []webrtc.ICEServer{
            {
                URLs: []string{"stun:stun.l.google.com:19302"},
            },
            {
                URLs:       []string{"turn:*hidden*:3478?transport=tcp"},
                Credential: "*hidden*",
                Username:   "*hidden*",
            },
        },
    }

    m := &webrtc.MediaEngine{}

    if err := m.RegisterCodec(webrtc.RTPCodecParameters{
        RTPCodecCapability: webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeVP8, ClockRate: 90000, Channels: 0, SDPFmtpLine: "", RTCPFeedback: nil},
        PayloadType:        96,
    }, webrtc.RTPCodecTypeVideo); err != nil {
        panic(err)
    }
    if err := m.RegisterCodec(webrtc.RTPCodecParameters{
        RTPCodecCapability: webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeOpus, ClockRate: 48000, Channels: 0, SDPFmtpLine: "", RTCPFeedback: nil},
        PayloadType:        111,
    }, webrtc.RTPCodecTypeAudio); err != nil {
        panic(err)
    }

    i := &interceptor.Registry{}
    if err := webrtc.RegisterDefaultInterceptors(m, i); err != nil {
        panic(err)
    }

    // Create the API object with the MediaEngine
    api := webrtc.NewAPI(webrtc.WithMediaEngine(m), webrtc.WithInterceptorRegistry(i))

    // Create a new RTCPeerConnection
    peerConnection, err := api.NewPeerConnection(config)
    if err != nil {
        panic(err)
    }

    if _, err = peerConnection.AddTransceiverFromKind(webrtc.RTPCodecTypeAudio); err != nil {
        panic(err)
    } else if _, err = peerConnection.AddTransceiverFromKind(webrtc.RTPCodecTypeVideo); err != nil {
        panic(err)
    }

    // Set a handler for when a new remote track starts, this handler copies inbound RTP packets,
    // replaces the SSRC and sends them back
    peerConnection.OnTrack(func(track *webrtc.TrackRemote, receiver *webrtc.RTPReceiver) {
        // Send a PLI on an interval so that the publisher is pushing a keyframe every rtcpPLIInterval
        go func() {
            ticker := time.NewTicker(time.Second * 2)
            defer ticker.Stop()
            for range ticker.C {
                if errSend := peerConnection.WriteRTCP([]rtcp.Packet{&rtcp.PictureLossIndication{MediaSSRC: uint32(track.SSRC())}}); errSend != nil {
                    break
                }
            }
        }()

        go func() {
            rtcpBuf := make([]byte, 1500)
            for {
                if _, _, rtcpErr := receiver.Read(rtcpBuf); rtcpErr != nil {
                    return
                }
            }
        }()

        codecName := strings.Split(track.Codec().RTPCodecCapability.MimeType, "/")[1]
        log.Printf("Track has started, of type %d: %s \n", track.PayloadType(), codecName)

        // reading our packets from webrtc
        buf := make([]byte, 1400)
        for {
            i, _, readErr := track.Read(buf)
            // TODO: close connection
            if readErr != nil {
                return
            }

            // if we got audio track
            if codecName == "opus" {
                streamPipeLine.AudioLine.Push(buf[:i])

                continue
            }

            // or if our track is video
            streamPipeLine.VideoLine.Push(buf[:i])
        }
    })

    // Set the handler for ICE connection state
    // stop record if host is down
    peerConnection.OnICEConnectionStateChange(func(connectionState webrtc.ICEConnectionState) {
        if connectionState == webrtc.ICEConnectionStateFailed {
            recorder := GetRecorder(channel)

            if recorder == nil {
                return
            }

            recorder.StopStream()
        }
    })

    return &Connection{
        peerConnection: peerConnection,
        streamPipeLine: streamPipeLine,
    }
}

Image here: click (ibb)

[isqad]

How did you resolve the issue?

[Sean-Der]

Hey @lekting

Are you going from WebRTC -> RTMP ?

You can ensure that get a perfect feed with WebRTC by inspecting the Sequence Numbers. Do you know if you are having issues Browser -> Pion or Pion -> GStreamer.

[lekting]

Hey @lekting

Are you going from WebRTC -> RTMP ?

You can ensure that get a perfect feed with WebRTC by inspecting the Sequence Numbers. Do you know if you are having issues Browser -> Pion or Pion -> GStreamer.

Hey! Problem still available. For now i know only some facts: WebRTC in Browser always changing bitrate from 200kb to 4mb, https://ibb.co/F0NJV2m <- image But, in case WebRTC Browser -> WebRTC Browser quality is poor, but no artifacts. I don't know how its working

[lekting]

Fixed by switching to sdp connection and changing GStreamer to FFMPEG