search

nextcloud / talk-android

📱😀 Video & audio calls through Nextcloud on Android
Other
527 stars 226 forks source link

Videocalls stop working when turning off PIP Mode #1677

Closed AndyXheli closed 2 years ago

AndyXheli commented 2 years ago

Steps to reproduce

  1. Turn off PIP
  2. Join Call

Expected behaviour

Once you join the call rings once and hangs up.

Actual behaviour

Should join the call

Device information

Device: Fairphone2/… Oneplus N200 5G Android version: 10/… Oxygen OS 11.0.3.0.DE17AA Talk version: 11.0.0 QA

Server information

Nextcloud version: (see admin overview page: /index.php/settings/admin/overview) 22.2.1 Talk version: (see apps admin page: /index.php/settings/apps) 12.1.2 Custom Signaling server configured: yes/no and version (see talk admin settings: /index.php/index.php/settings/admin/talk#signaling_server) No Custom TURN server configured: yes/no (see talk admin settings: /index.php/settings/admin/talk#turn_server) Yes Custom STUN server configured: yes/no (see talk admin settings: /index.php/settings/admin/talk#stun_server) Yes

https://user-images.githubusercontent.com/59488153/141344361-f4f693d7-5147-43d8-a82e-b540fb91f461.mp4

mahibi commented 2 years ago

thanks for the report. seems to happen for videocalls only. i have a look

mahibi commented 2 years ago

well there is logic that enterPipMode is called when onUserLeaveHint is executed: https://github.com/nextcloud/talk-android/blob/d602bf3dfe601abed3f98bd1d574e0776a2aecc4/app/src/main/java/com/nextcloud/talk/activities/CallBaseActivity.java#L91

When in enterPipMode it's recognized that pip is not possible, the activity is finished (yes, that not cool but will be improved in the future. most probably with a background service&notification).

The main question is why onUserLeaveHint is executed when a videocall starts?!

grafik

So enterPipMode is already called on start and it kills the activity...

Anyone has an idea why onUserLeaveHint could be called when a videocall starts??

mahibi commented 2 years ago

log until activity is finished:

D/ChatController: getRoomInfo
I/okhttp.OkHttpClient: --> GET http://192.168.8.106/nextcloud8nov/ocs/v2.php/apps/spreed/api/v4/room/wup2odnv
I/okhttp.OkHttpClient: Authorization: ██
I/okhttp.OkHttpClient: User-Agent: Mozilla/5.0 (Android) Nextcloud-Talk v13.0.0 Alpha 1
    Accept: application/json
I/okhttp.OkHttpClient: OCS-APIRequest: true
I/okhttp.OkHttpClient: --> END GET
I/okhttp.OkHttpClient: <-- 200 OK http://192.168.8.106/nextcloud8nov/ocs/v2.php/apps/spreed/api/v4/room/wup2odnv (726ms)
I/okhttp.OkHttpClient: Date: Fri, 12 Nov 2021 12:18:58 GMT
I/okhttp.OkHttpClient: Server: Apache/2.4.29 (Ubuntu)
I/okhttp.OkHttpClient: Referrer-Policy: no-referrer
I/okhttp.OkHttpClient: X-Content-Type-Options: nosniff
I/okhttp.OkHttpClient: X-Download-Options: noopen
I/okhttp.OkHttpClient: X-Frame-Options: SAMEORIGIN
I/okhttp.OkHttpClient: X-Permitted-Cross-Domain-Policies: none
I/okhttp.OkHttpClient: X-Robots-Tag: none
I/okhttp.OkHttpClient: X-XSS-Protection: 1; mode=block
I/okhttp.OkHttpClient: Expires: Thu, 19 Nov 1981 08:52:00 GMT
I/okhttp.OkHttpClient: Cache-Control: no-cache, no-store, must-revalidate
I/okhttp.OkHttpClient: Pragma: no-cache
I/okhttp.OkHttpClient: Content-Security-Policy: default-src 'none';base-uri 'none';manifest-src 'self';frame-ancestors 'none'
I/okhttp.OkHttpClient: Feature-Policy: autoplay 'none';camera 'none';fullscreen 'none';geolocation 'none';microphone 'none';payment 'none'
I/okhttp.OkHttpClient: X-Nextcloud-Talk-Hash: f4b1ac5fd96fb1b28efb0193425cdb7f914a40f7
I/okhttp.OkHttpClient: Keep-Alive: timeout=5, max=100
I/okhttp.OkHttpClient: Connection: Keep-Alive
I/okhttp.OkHttpClient: Content-Type: application/json; charset=utf-8
I/Timeline: Timeline: Activity_launch_request time:1812905085 intent:Intent { cmp=com.nextcloud.talk2/com.nextcloud.talk.activities.CallActivity (has extras) }
I/okhttp.OkHttpClient: {"ocs":{"meta":{"status":"ok","statuscode":200,"message":"OK"},"data":{"id":2,"token":"wup2odnv","type":1,"name":"user1","displayName":"user1","objectType":"","objectId":"","participantType":1,"participantFlags":7,"readOnly":0,"hasPassword":false,"hasCall":true,"canStartCall":true,"lastActivity":1636719400,"lastReadMessage":400,"unreadMessages":0,"unreadMention":false,"unreadMentionDirect":false,"isFavorite":false,"canLeaveConversation":true,"canDeleteConversation":false,"notificationLevel":1,"notificationCalls":1,"lobbyState":0,"lobbyTimer":0,"lastPing":1636719399,"sessionId":"gYFOgj0WiNynbeRiGpyCqefr5pJl+znDe2udBIcoh\/wYjOcNgVUVBOIXdpaH\/Xn1pOjgAZBNRWuhK9DWQkcLsW4\/v9FQn3s0Bps3S9lBpBhIbRl1yXhu+0qMD2G+qWtx\/hbsAFhsBH02LgYM9pDF0N+Oghutly\/lrHzmSyD7Dr226LFxQfEb2foxd6uToW\/wJar92fpmbMwTGuGKNqJsPK2ztMwKGmMq7oQS7dDgF7A1w8ZaSPGgJnmK7vE9RKY","lastMessage":{"id":400,"token":"wup2odnv","actorType":"users","actorId":"user2","actorDisplayName":"User 2","timestamp":1636719400,"message":"Sie haben einen Anruf begonnen","messageParameters":{"actor":{"type":"user","id":"user2","name":"User 2"}},"systemMessage":"call_started","messageType":"system","isReplyable":false,"referenceId":""},"sipEnabled":0,"actorType":"users","actorId":"user2","attendeeId":3,"permissions":126,"attendeePermissions":0,"callPermissions":0,"defaultPermissions":0,"canEnableSIP":false,"attendeePin":"","description":"","lastCommonReadMessage":390,"listable":0,"callFlag":7}}}
I/okhttp.OkHttpClient: <-- END HTTP (1454-byte body)
D/ChatController: currentConversation.toString : Conversation{roomId='2', token='wup2odnv', name='user1', displayName='user1', description='', type=ROOM_TYPE_ONE_TO_ONE_CALL, lastPing=1636719399, participants=null, participantType=OWNER, actorId=user2, actorType=users, hasPassword=false, sessionId='gYFOgj0WiNynbeRiGpyCqefr5pJl+znDe2udBIcoh/wYjOcNgVUVBOIXdpaH/Xn1pOjgAZBNRWuhK9DWQkcLsW4/v9FQn3s0Bps3S9lBpBhIbRl1yXhu+0qMD2G+qWtx/hbsAFhsBH02LgYM9pDF0N+Oghutly/lrHzmSyD7Dr226LFxQfEb2foxd6uToW/wJar92fpmbMwTGuGKNqJsPK2ztMwKGmMq7oQS7dDgF7A1w8ZaSPGgJnmK7vE9RKY', password='null', isFavorite=false, lastActivity=1636719400, unreadMessages=0, unreadMention=false, lastMessage=ChatMessage(isGrouped=false, isOneToOneConversation=false, activeUser=null, selectedIndividualHashMap=null, isDeleted=false, jsonMessageId=400, token=wup2odnv, actorType=users, actorId=user2, actorDisplayName=User 2, timestamp=1636719400, message=Sie haben einen Anruf begonnen, messageParameters={actor={name=User 2, id=user2, type=user}}, systemMessageType=CALL_STARTED, replyable=false, parentMessage=null, readStatus=NONE, messageTypesToIgnore=[REGULAR_TEXT_MESSAGE, SYSTEM_MESSAGE, SINGLE_LINK_VIDEO_MESSAGE, SINGLE_LINK_AUDIO_MESSAGE, SINGLE_LINK_MESSAGE, SINGLE_NC_GEOLOCATION_MESSAGE, VOICE_MESSAGE]), objectType='', notificationLevel=ALWAYS, conversationReadOnlyState=CONVERSATION_READ_WRITE, lobbyState=LOBBY_STATE_ALL_PARTICIPANTS, lobbyTimer=0, lastReadMessage=400, callFlag=7, canLeaveConversation=true, canDeleteConversation=false, notificationCalls=1}
    currentConversation.sessionId : gYFOgj0WiNynbeRiGpyCqefr5pJl+znDe2udBIcoh/wYjOcNgVUVBOIXdpaH/Xn1pOjgAZBNRWuhK9DWQkcLsW4/v9FQn3s0Bps3S9lBpBhIbRl1yXhu+0qMD2G+qWtx/hbsAFhsBH02LgYM9pDF0N+Oghutly/lrHzmSyD7Dr226LFxQfEb2foxd6uToW/wJar92fpmbMwTGuGKNqJsPK2ztMwKGmMq7oQS7dDgF7A1w8ZaSPGgJnmK7vE9RKY
D/CallActivity: onCreate
W/unknown:WrappingUtils: Don't know how to round that drawable: android.graphics.drawable.VectorDrawable@d6afdf8
W/unknown:WrappingUtils: Don't know how to round that drawable: android.graphics.drawable.VectorDrawable@228774b
W/unknown:WrappingUtils: Don't know how to round that drawable: android.graphics.drawable.VectorDrawable@1ae2241
W/unknown:WrappingUtils: Don't know how to round that drawable: android.graphics.drawable.VectorDrawable@4da93e6
W/unknown:WrappingUtils: Don't know how to round that drawable: android.graphics.drawable.VectorDrawable@2071bd4
W/unknown:WrappingUtils: Don't know how to round that drawable: android.graphics.drawable.VectorDrawable@3ac0872
W/unknown:WrappingUtils: Don't know how to round that drawable: android.graphics.drawable.VectorDrawable@4fe84c3
I/org.webrtc.Logging: EglBase14Impl: SDK version: 28. isEGL14Supported: true
I/org.webrtc.Logging: EglBase14Impl: Using OpenGL ES version 2
W/nextcloud.talk2: type=1400 audit(0.0:200129): avc: denied { read } for name="u:object_r:vendor_camera_prop:s0" dev="tmpfs" ino=13803 scontext=u:r:untrusted_app:s0:c193,c257,c512,c768 tcontext=u:object_r:vendor_camera_prop:s0 tclass=file permissive=0
E/libc: Access denied finding property "vendor.camera.aux.packagelist"
I/org.webrtc.Logging: WebRtcAudioManagerExternal: Sample rate is set to 48000 Hz
I/org.webrtc.Logging: WebRtcAudioManagerExternal: Sample rate is set to 48000 Hz
I/org.webrtc.Logging: JavaAudioDeviceModule: createAudioDeviceModule
    JavaAudioDeviceModule: HW NS will be used.
I/org.webrtc.Logging: JavaAudioDeviceModule: HW AEC will be used.
    WebRtcAudioEffectsExternal: ctor@[name=main, id=2]
I/org.webrtc.Logging: WebRtcAudioRecordExternal: ctor@[name=main, id=2]
I/org.webrtc.Logging: WebRtcAudioTrackExternal: ctor@[name=main, id=2]
V/AudioTrack: getMinFrameCount=3844: afFrameCount=1920, afSampleRate=48000, afLatency=80
I/org.webrtc.Logging: WebRtcAudioRecordExternal: enableBuiltInAEC(true)
    WebRtcAudioEffectsExternal: setAEC(true)
I/org.webrtc.Logging: WebRtcAudioRecordExternal: enableBuiltInNS(true)
I/org.webrtc.Logging: WebRtcAudioEffectsExternal: setNS(true)
I/org.webrtc.Logging: PeerConnectionFactory: onNetworkThreadReady
I/org.webrtc.Logging: PeerConnectionFactory: onSignalingThreadReady
D/MagicAudioManager: ctor
D/MagicBluetoothManager: ctor
I/org.webrtc.Logging: PeerConnectionFactory: onWorkerThreadReady
D/MagicAudioManager: defaultAudioDevice: SPEAKER_PHONE
D/CallActivity: Starting the audio manager...
D/MagicAudioManager: start
    AudioManager starts...
I/AudioManager: In isSpeakerphoneOn(), calling application: com.nextcloud.talk2
W/AudioManager: Use of stream types is deprecated for operations other than volume control
    See the documentation of requestAudioFocus() for what to use instead with android.media.AudioAttributes to qualify your playback use case
D/MagicAudioManager: Audio focus request granted for VOICE_CALL streams
D/MagicBluetoothManager: start
I/AudioManager: In isBluetoothScoAvailableOffCall(), calling appilication: com.nextcloud.talk2, return value: true
D/MagicBluetoothManager: BluetoothAdapter: enabled=false, state=OFF, name=Redmi, address=02:00:00:00:00:00
D/MagicBluetoothManager: HEADSET profile state: DISCONNECTED
    Bluetooth proxy for headset profile has started
    start done: BT state=HEADSET_UNAVAILABLE
D/MagicAudioManager: --- updateAudioDeviceState: wired headset=false, BT state=HEADSET_UNAVAILABLE
    Device status: available=[], selected=NONE, user selected=NONE
D/MagicAudioManager: setAudioDeviceInternal(device=SPEAKER_PHONE)
I/AudioManager: In isSpeakerphoneOn(), calling application: com.nextcloud.talk2
I/AudioManager: In setSpeakerphoneOn(), on: true, calling application: com.nextcloud.talk2
D/MagicAudioManager: New device status: available=[SPEAKER_PHONE, EARPIECE], selected=SPEAKER_PHONE
D/CallActivity: onAudioManagerDevicesChanged: [SPEAKER_PHONE, EARPIECE], selected: SPEAKER_PHONE
D/MagicAudioManager: --- updateAudioDeviceState done
D/MagicProximitySensor: Proximity sensor: name=stk3x3x alsprx, vendor: Sensortek, power: 0.1, resolution: 0.05000305, max range: 5.000305, min delay: 0, type: android.sensor.proximity, max delay: 65535000, reporting mode: 1, isWakeUpSensor: true
D/MagicAudioManager: AudioManager started
E/libc: Access denied finding property "vendor.camera.aux.packagelist"
W/nextcloud.talk2: type=1400 audit(0.0:200130): avc: denied { read } for name="u:object_r:vendor_camera_prop:s0" dev="tmpfs" ino=13803 scontext=u:r:untrusted_app:s0:c193,c257,c512,c768 tcontext=u:object_r:vendor_camera_prop:s0 tclass=file permissive=0
I/org.webrtc.Logging: CallActivity: Looking for front facing cameras.
I/org.webrtc.Logging: CallActivity: Creating front facing camera capturer.
W/nextcloud.talk2: type=1400 audit(0.0:200131): avc: denied { read } for name="u:object_r:vendor_camera_prop:s0" dev="tmpfs" ino=13803 scontext=u:r:untrusted_app:s0:c193,c257,c512,c768 tcontext=u:object_r:vendor_camera_prop:s0 tclass=file permissive=0
E/libc: Access denied finding property "vendor.camera.aux.packagelist"
I/org.webrtc.Logging: EglRenderer: selfVideoRenderersetMirrorHorizontally: true
I/org.webrtc.Logging: EglBase14Impl: Using OpenGL ES version 2
D/CallActivity: initViews
W/nextcloud.talk2: type=1400 audit(0.0:200132): avc: denied { read } for name="u:object_r:vendor_camera_prop:s0" dev="tmpfs" ino=13803 scontext=u:r:untrusted_app:s0:c193,c257,c512,c768 tcontext=u:object_r:vendor_camera_prop:s0 tclass=file permissive=0
E/libc: Access denied finding property "vendor.camera.aux.packagelist"
I/org.webrtc.Logging: EglRenderer: selfVideoRendererInitializing EglRenderer
I/org.webrtc.Logging: EglRenderer: selfVideoRendererEglBase.create shared context
I/org.webrtc.Logging: EglBase14Impl: Using OpenGL ES version 2
D/CallActivity: initGridAdapter
I/Timeline: Timeline: Activity_launch_request time:1812905794 intent:Intent { act=android.content.pm.action.REQUEST_PERMISSIONS pkg=com.google.android.packageinstaller (has extras) }
D/CallActivity: updateSelfVideoViewPosition
D/MagicProximitySensor: Proximity sensor => FAR state
D/MagicAudioManager: setAudioDeviceInternal(device=SPEAKER_PHONE)
I/AudioManager: In isSpeakerphoneOn(), calling application: com.nextcloud.talk2
I/Choreographer: Skipped 38 frames!  The application may be doing too much work on its main thread.
I/org.webrtc.Logging: SurfaceViewRenderer: selfVideoRenderer: onMeasure(). New size: 220x2252
I/org.webrtc.Logging: SurfaceViewRenderer: selfVideoRenderer: onMeasure(). New size: 220x2252
I/org.webrtc.Logging: SurfaceViewRenderer: selfVideoRenderer: onMeasure(). New size: 220x2252
I/chatty: uid=10449(com.nextcloud.talk2) identical 4 lines
I/org.webrtc.Logging: SurfaceViewRenderer: selfVideoRenderer: onMeasure(). New size: 220x2252
I/org.webrtc.Logging: SurfaceViewRenderer: selfVideoRenderer: onMeasure(). New size: 220x2172
I/chatty: uid=10449(com.nextcloud.talk2) identical 2 lines
I/org.webrtc.Logging: SurfaceViewRenderer: selfVideoRenderer: onMeasure(). New size: 220x2172
I/org.webrtc.Logging: EglRenderer: selfVideoRenderersetLayoutAspectRatio: 0.10128913
W/Looper: Slow Looper main: doFrame is 644ms late because of 3 msg, msg 2 took 59ms (late=90ms h=android.os.Handler c=io.reactivex.android.schedulers.HandlerScheduler$ScheduledRunnable), msg 3 took 556ms (late=128ms h=android.app.ActivityThread$H w=159)
I/org.webrtc.Logging: EglRenderer: selfVideoRendererDuration: 4026 ms. Frames received: 0. Dropped: 0. Rendered: 0. Render fps: ,0. Average render time: NA. Average swapBuffer time: NA.
D/CallBaseActivity: Activity was finished because PIP is not available.
mahibi commented 2 years ago

so far @tobiasKaminsky and me don't understand why onUserLeaveHint() is called. the problem is also known for others (https://stackoverflow.com/questions/44729874/onuserleavehint-gets-called-when-entering-activity) but so far without any solution..

mahibi commented 2 years ago

i still don't get it. my last guess was that there was some config change when the activity started so it would be restarted which might explain the execution of onUserLeaveHint. To catch any configChange and handle it myself i put all possible config changes that are documented in https://developer.android.com/guide/topics/manifest/activity-element#config into the manifest file. But onConfigurationChanged is not called. So this means it can't be caused by a configChange..?!

i have to give up for today..

mahibi commented 2 years ago

for now this is solved by the workaround https://github.com/nextcloud/talk-android/pull/1685 in the long term this must be solved in a better way --> https://github.com/nextcloud/talk-android/issues/1689