Missing parameters cause consume request to fail

neilyoung commented 10 months ago

While building the rtpCapabilities the stack assembles various elements. IMHO it misses hereby to take care of the matchingLocalCodec.parameters, which are packed into the localParameters element of ExtendedCodec.

Those are not taken over before this line https://github.com/skymaze/pymediasoup/blob/6fd3564ed690fe59ab886334fe136bca84648a7e/pymediasoup/ortc.py#L162

and will later on make a consume request fail due to non matching parameters here: https://github.com/versatica/mediasoup/blob/0b4d4ef9f2bb0b9d117e48f707c970d406ee5022/node/src/ortc.ts#L1458

This doesn't harm with audio and VP8 streams, but is essential to make H.264 subscriptions work.

I "hotfixed" it by inserting a line to take over the localParameters in ortc.py:

        codec: RtpCodecCapability = RtpCodecCapability(**extendedCodec.dict())
        codec.parameters = extendedCodec.localParameters
        rtpCapabilities.codecs.append(codec)

skymaze commented 10 months ago

Thanks, I released version 0.2.3, which should fix this issue.

neilyoung commented 10 months ago

Thanks for the quick response and good that this nice lib is still alive

skymaze commented 10 months ago

I am no longer using mediasoup, but I wrote this lib according to the official mediasoup-client structure, so it can be easily maintained. Feel free to open a issue or make a PR.

neilyoung commented 10 months ago

@skymaze May I ask you if simulcast is supposed to work?

In analogy to my web solution I was adding "encodings" parameters to the sendTransport.produce() call. Track is the default VideoStreamTrack(), encoding H.264. It works w/o "encodings" parameter, but doesn't work with encodings, even though the consumer is notified about simulcast tracks and L3T3. The server spits warnings, continuously:

 mediasoup:WARN:Channel [pid:42091] RTC::Producer::ReceiveRtpPacket() | no stream found for received packet [ssrc:2458002806] +31ms

The video does not appear on the remote side.

                highestBitrate = 6000000
                mediumBitrate = highestBitrate / 2
                lowestBitrate = highestBitrate / 10

                encoding = RtpEncodingParameters(
                    maxBitrate=lowestBitrate,
                    priority='very-low',
                    rid="r0",
                    scalabilityMode='L3T3',
                    scaleResolutionDownBy=4
                )
                encodings.append(encoding)    

                encoding = RtpEncodingParameters(
                    maxBitrate=mediumBitrate,
                    priority='low',
                    rid="r1",
                    scalabilityMode='L3T3',
                    scaleResolutionDownBy=2
                )
                encodings.append(encoding)    

                encoding = RtpEncodingParameters(
                    maxBitrate=highestBitrate,
                    priority='high',
                    rid="r2",
                    scalabilityMode='L3T3',
                    scaleResolutionDownBy=1
                )
                encodings.append(encoding)    

            videoProducer: Producer = await self._sendTransport.produce(
                track = track,
                encodings = encodings,
                stopTracks = True,
                appData = appData
            )

skymaze commented 10 months ago

Maybe it is related to this code,, aiortc does not implement the corresponding API. Or related to default scalabilityMode here

skymaze / pymediasoup

Missing parameters cause consume request to fail #9