search

nextcloud / talk-android

📱😀 Video & audio calls through Nextcloud on Android
Other
529 stars 234 forks source link

Video calls using HPB shows no video from android app to android app #3358

Open muchachagrande opened 11 months ago

muchachagrande commented 11 months ago

Steps to reproduce

  1. Initiates a video call from an Android Talk app to another Android Talk app
  2. Answer the video call from the receiving end

Expected behaviour

The call is established and both users starts to see and hear to each other.

Actual behaviour

The call establishes and both users starts to hear each other but can't see the others video.

Device brand and model

Tested on various smart phone models with Android using Talk GPlay version and Lineage using Talk F-Droid version.

Nextcloud Talk app version

Tested with various versions. The problem has persisted since version 15 or so.

Nextcloud server version

Tested with various versions. It has happened since version 25 or so.

Talk version

Tested with various versions. It has happened since version 15 or so.

Custom Signaling server configured

Yes (specify version in Additional Information)

Custom TURN server configured

Yes

Custom STUN server configured

Yes

Android logs

No response

Server log

Nothing on the server log that day

Additional information

This problem has been happening through various Signaling versions. I think I first saw this with Signaling v1.1.0. Also have been experimenting this using many different versions of Janus and is actually happening with the new 1.x branch that recently was made compatible with Talk.

Other observed details:

ASerbinski commented 10 months ago

I can confirm this.

pmarini-nc commented 10 months ago

I can also confirm this

himpierre commented 9 months ago

Is this fixed with 18.0.0-rc1? If not, am I in the wrong movie? :)

muchachagrande commented 9 months ago

May be most users are not impacted by this problem or may be most users do not use HPB. I don't know.

MarkPartlett commented 9 months ago

I can confirm this

muchachagrande commented 7 months ago

Will this issue be addressed in the next version ? or in some future version ?

skl256 commented 6 months ago

I can confrm, 18.0.1

mircokam commented 5 months ago
Connection type Cameras work Cameras only work if...
Android to Android ! they are toggled off and on on both devices
Android to iOS ✓
Android to Web ✓
iOS to Web ✓

Tested versions: Android app: 18.1.0 iOS app: 18.0.3 Nextcloud: 28.0.3 Talk: 18.0.5 nextcloud-spreed-signaling: 1.2.3 Janus: 1.1.2-1 (Debian Bookworm)

muchachagrande commented 5 months ago

I've been trying to find some clues in the signaling log and found nothing of interest. Compared a web to android and a android to android video call log and in both cases the video room is opened correctly. May be there is some handling issue on the code that prevents the video from appearing at first. After that, if you cycle the camera off and on, the video starts to show on the other end. The log doesn't show anything when you cycle the camera, so I think the channels are working correctly.

muchachagrande commented 5 months ago

Another detail about this issue. I have experimented from time to time that in some video calls, one peer can see the other at the beginning, so only one side has to cycle the camera to make it appear on the other end. This is a random event. Most of the times the video calls starts with no video on both ends.

jakobroehrl commented 4 months ago

Tested it yesterday, can confirm the bug. HPB with android <-> android the video does not show up

Byter3 commented 1 month ago

This issue is here with us for idk more than two years at this point?! Can't the devs just implement a script, too auto toggle the cameraas on the android app when a call started, just as a work around? Please

3999

SystemKeeper commented 1 month ago

I tried to look into this issue, it seems to be a twofold issue.

Overwriting video state (problem on receiving side)

It seem the following is happening

  1. We handle the stream change https://github.com/nextcloud/talk-android/blob/45b18a7f9da717c51325bd5319946145a8f4b729/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java#L181-L193 Here we determine that we have a media stream and if there's at least one video stream, we call callParticipantModel.setVideoAvailable(true);. This would be enough to show the video
  2. We handle the ice connection state change After we already handled the stream change, we now handle the ice connection state change to CHECKING: https://github.com/nextcloud/talk-android/blob/45b18a7f9da717c51325bd5319946145a8f4b729/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java#L171-L179 This resets the available video again: callParticipantModel.setVideoAvailable(null);. Which results in no video.

I am not totally sure where this race condition comes from or wether it's expected. A simple fix would be to just checking state NEW here, from my POV:

diff --git a/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java b/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java
index 50c1ae669..3f9f83ed2 100644
--- a/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java
+++ b/app/src/main/java/com/nextcloud/talk/call/CallParticipant.java
@@ -171,8 +171,7 @@ public class CallParticipant {
     private void handleIceConnectionStateChange(PeerConnection.IceConnectionState iceConnectionState) {
         callParticipantModel.setIceConnectionState(iceConnectionState);

-        if (iceConnectionState == PeerConnection.IceConnectionState.NEW ||
-                iceConnectionState == PeerConnection.IceConnectionState.CHECKING) {
+        if (iceConnectionState == PeerConnection.IceConnectionState.NEW) {
             callParticipantModel.setAudioAvailable(null);
             callParticipantModel.setVideoAvailable(null);
         }

Not sending initial media state (problem on sending side)

The problem above would not be an issue, if talk-android would correctly send the initial media state.

https://github.com/nextcloud/talk-android/blob/45b18a7f9da717c51325bd5319946145a8f4b729/app/src/main/java/com/nextcloud/talk/webrtc/PeerConnectionWrapper.java#L301-L315

This will basically do nothing when using an HPB, because the wrapper of the participant does not have a localStream attached (only the publisher does). Even then it would not work, because the data should be send via the publisher and not the individual wrappers. So the initial state should be correctly send as well, to fix both problems.

MelBourbon commented 2 weeks ago

Another support for my users would be if I could set on server side that at the beginning of a video call the camera is initially disabled and needs to be enabled. Is this possible?