search

fishjam-dev / membrane_rtc_engine

Customizable Real-time Communication Engine/SFU library focused on WebRTC.
Apache License 2.0
140 stars 11 forks source link

[RTC-477] Allow for muting tracks without renegotiation #392

Closed roznawsk closed 3 months ago

roznawsk commented 3 months ago

This allows for explicitly muting and then unmuting tracks.

The flow is as follows:

  1. Send mute event from frontend AND replace the track with null so no data is sent.

This is how it looks like on the FE:

webrtc.replaceTrack(videoStreamIdRef.current, null, null, "mute");

const mediaEvent = generateMediaEvent("muteTrack", { trackId: trackId });
this.sendMediaEvent(mediaEvent);

The above is POC, and will probably end up with a dedicated function in membrane-webrtc-js.

  1. The mute event is handled by TrackSender which sends %TrackVariantPaused{reason: :muted} events.
  2. Upon receiving this events the TrackReceiver marks the currently active variant as muted_variant
  3. The unmute event is sent from FE as well as the track being replaced with an actual video
  4. The TrackSender sends TrackVariantResumed events to TrackReceiver
  5. The VariantSelector within TrackReceiver requests the last paused variant, ignoring the limitation of the bandwidth, which at this point is probably 0.

I don't think that setting muted variant as inactive is possible. That's because both when switching to muted and inactive state we check whether the variant is in the active state first.

codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 38.46154% with 32 lines in your changes are missing coverage. Please review.

Project coverage is 54.89%. Comparing base (acc80de) to head (d7c54c8).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #392 +/- ## ========================================== - Coverage 55.19% 54.89% -0.31% ========================================== Files 75 75 Lines 3299 3339 +40 ========================================== + Hits 1821 1833 +12 - Misses 1478 1506 +28 ``` | [Files](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev) | Coverage Δ | | |---|---|---| | [webrtc/lib/webrtc/variant\_selector.ex](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?src=pr&el=tree&filepath=webrtc%2Flib%2Fwebrtc%2Fvariant_selector.ex&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev#diff-d2VicnRjL2xpYi93ZWJydGMvdmFyaWFudF9zZWxlY3Rvci5leA==) | `84.72% <100.00%> (+1.38%)` | :arrow_up: | | [webrtc/lib/webrtc/media\_event.ex](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?src=pr&el=tree&filepath=webrtc%2Flib%2Fwebrtc%2Fmedia_event.ex&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev#diff-d2VicnRjL2xpYi93ZWJydGMvbWVkaWFfZXZlbnQuZXg=) | `19.27% <0.00%> (-0.48%)` | :arrow_down: | | [webrtc/lib/webrtc/track\_receiver.ex](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?src=pr&el=tree&filepath=webrtc%2Flib%2Fwebrtc%2Ftrack_receiver.ex&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev#diff-d2VicnRjL2xpYi93ZWJydGMvdHJhY2tfcmVjZWl2ZXIuZXg=) | `0.00% <0.00%> (ø)` | | | [webrtc/lib/webrtc\_endpoint.ex](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?src=pr&el=tree&filepath=webrtc%2Flib%2Fwebrtc_endpoint.ex&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev#diff-d2VicnRjL2xpYi93ZWJydGNfZW5kcG9pbnQuZXg=) | `0.00% <0.00%> (ø)` | | | [webrtc/lib/webrtc/variant\_tracker.ex](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?src=pr&el=tree&filepath=webrtc%2Flib%2Fwebrtc%2Fvariant_tracker.ex&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev#diff-d2VicnRjL2xpYi93ZWJydGMvdmFyaWFudF90cmFja2VyLmV4) | `70.00% <0.00%> (-25.46%)` | :arrow_down: | | [webrtc/lib/webrtc/track\_sender.ex](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?src=pr&el=tree&filepath=webrtc%2Flib%2Fwebrtc%2Ftrack_sender.ex&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev#diff-d2VicnRjL2xpYi93ZWJydGMvdHJhY2tfc2VuZGVyLmV4) | `63.49% <5.26%> (-9.66%)` | :arrow_down: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev) ------ [Continue to review full report in Codecov by Sentry](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?dropdown=coverage&src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev). Last update [acc80de...d7c54c8](https://app.codecov.io/gh/fishjam-dev/membrane_rtc_engine/pull/392?dropdown=coverage&src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fishjam-dev).