awslabs / amazon-kinesis-video-streams-webrtc-sdk-c

Amazon Kinesis Video Streams Webrtc SDK is for developers to install and customize realtime communication between devices and enable secure streaming of video, audio to Kinesis Video Streams.
https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-c/group__PublicMemberFunctions.html
Apache License 2.0
1.04k stars 315 forks source link

[Bug]: The receiveGstreamerAudioVideo don't work in the v1.11.0 #2076

Open linzhenggang1 opened 1 week ago

linzhenggang1 commented 1 week ago

Please confirm you have already done the following

Please answer the following prompt

Describe the bug

When I ran the kvsWebrtcClientMasterGstSample, I found that it didn't play any audio.

I compared the source code of 1.10.1 and 1.11.0 and found that you deleted the code below in /samples/kvsWebrtcClientMasterGstSample.c from v1.11.0.

pSampleConfiguration->receiveAudioVideoSource = receiveGstreamerAudioVideo;

Is this a mistake? If No, could you tell me how to make the kvsWebrtcClientMasterGstSample play audio from the remote peer?

Thank you.

Expected Behavior

The kvsWebrtcClientMasterGstSample plays audio from the remote peer. (The receiveGstreamerAudioVideo task can run normally)

Current Behavior

The kvsWebrtcClientMasterGstSample doesn't play audio from the remote peer. (The receiveGstreamerAudioVideo task doesn't run)

Reproduction Steps

  1. start the kvsWebrtcClientMasterGstSample as the Master
  2. start the https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-js/examples/index.html as Viewer
  3. make some sound on the Viewer side and check whether the Master side play that

WebRTC C SDK version being used

v1.11.0

If it was working in a previous version, which one?

1.10.1

Compiler and Version used

gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0

Operating System and version

Ubuntu 24

Platform being used

Linux

sirknightj commented 1 week ago

Hi @linzhenggang1, thanks for your concern. I was looking through the diff and wasn't able to find the line you mentioned above. Could you help me find it and/or the commit? https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/compare/v1.10.1..v1.11.0

To get the audio to playback for the MasterGstSample, you can add the following to https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/blob/v1.11.0/samples/kvsWebrtcClientMasterGstSample.c#L392:

+    pSampleConfiguration->receiveAudioVideoSource = receiveGstreamerAudioVideo;
-    pSampleConfiguration->mediaType = SAMPLE_STREAMING_VIDEO_ONLY;
+    pSampleConfiguration->mediaType = SAMPLE_STREAMING_AUDIO_VIDEO;
linzhenggang1 commented 1 week ago

@sirknightj Thank you for your reply.

I found it from your URL link https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/compare/v1.10.1..v1.11.0. image

Maybe you have to click "Load diff" on the kvsWebrtcClientMasterGstSample.c section.

sirknightj commented 1 week ago

HI @linzhenggang1, thanks for the clarification.

I traced back the change to this pull request: #1972, which indicates it was removed intentionally due to issues encountered with the video track.

Are the changes I suggested above working for you to enable the audio playback you're looking for?

linzhenggang1 commented 1 week ago

Hi @sirknightj Thanks for your work.

I tied the source code you suggested above in v1.11.0 and v1.10.2, but they didn't work and even didn't accept a CTRL + C command.

In addition, the v1.10.1 is working well in my environment.

Is the v1.11.0 and v1.10.2 well working in your environment?

When the receiveGstreamerAudioVideo thread does not work, the onGstAudioFrameReady function will never be called, so how will it play sounds?