rom1v / sndcpy

Android audio forwarding (scrcpy, but for audio)
MIT License
3.26k stars 299 forks source link

Quest 2 #75

Open rzzza opened 3 years ago

rzzza commented 3 years ago

I'm trying to get this working on the Quest 2 but unable to click the "start now" button within the headset. Is there anyway around this?

emersonbf commented 3 years ago

I have the same problem. Any solution ?

KptnKMan commented 3 years ago

Also trying to make this work on Quest/Quest2, does anyone have any advice?

I have scrcpy working without issues with this command:

PS C:\Programs\scrcpy> .\scrcpy.exe --crop 1600:900:2017:510 -b 8000000 --max-fps 60 --max-size 0 -n --window-title "Scrcpy Stream" -s 192.168.1.129:5555
INFO: scrcpy 1.11 <https://github.com/Genymobile/scrcpy>
C:\Programs\scrcpy\scrcpy-server: 1 file pushed. 1.9 MB/s (25454 bytes in 0.012s)
INFO: Initial texture: 1600x896

Then I can see my Quest 2 stream.

With sndcpyI don't get a prompt, or anything. I get this:

PS C:\Programs\sndcpy> .\sndcpy 192.168.1.129:5555
Waiting for device 192.168.1.129:5555...
Performing Streamed Install
Success
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.rom1v.sndcpy/.MainActivity }
Warning: Activity not started because the  current activity is being kept for the user.
Press Enter once audio capture is authorized on the device to start playing...

I press Enter, and get this:

Playing audio...
The system cannot find the path specified.
KptnKMan commented 3 years ago

Further to that, the Oculus device specs page doesn't list what version of Android the Quest 1 / Quest 2 is running.

A thread on Reddit reports (A year ago, as of this writing) it as:

Android 7.1.1
kernel: 4.4.21 -perf+
Security patch level: July 28 2018

As sndcpy states that it requires Android 10, is this a hard limitation?

rzzza commented 3 years ago

Quest 2 is running Android 10

KptnKMan commented 3 years ago

Good to know, I wanted to verify what was happening so I poked around using ABD commands. I got all the ADB build info remotely:

Quest1 (Not installed the v23 update yet) :

PS C:\Users\kptnkman> adb -s 192.168.1.28:5555 shell getprop
[ro.build.ab_update]: [true]
[ro.build.branch]: [releases-msm8998-monterey-v20]
[ro.build.characteristics]: [default]
[ro.build.date]: [Mon Sep 28 16:36:15 PDT 2020]
[ro.build.date.utc]: [1601336175]
[ro.build.description]: [vr_monterey-user 7.1.1 NGI77B 904120.19760.0 release-keys]
[ro.build.display.id]: [user-904120.19760.0]
[ro.build.fingerprint]: [oculus/vr_monterey/monterey:7.1.1/NGI77B/904120.19760.0:user/release-keys]
[ro.build.flavor]: [vr_monterey-user]
[ro.build.host]: [sandcastle927.atn5.facebook.com]
[ro.build.id]: [NGI77B]
[ro.build.product]: [monterey]
[ro.build.system_root_image]: [true]
[ro.build.tags]: [release-keys]
[ro.build.type]: [user]
[ro.build.user]: [twsvcscm]
[ro.build.version.all_codenames]: [REL]
[ro.build.version.base_os]: []
[ro.build.version.codename]: [REL]
[ro.build.version.incremental]: [9041200197600000]
[ro.build.version.oculus]: [904120.19760.0]
[ro.build.version.preview_sdk]: [0]
[ro.build.version.release]: [7.1.1]
[ro.build.version.sdk]: [25]
[ro.build.version.security_patch]: [2018-11-05]

Quest2 (With v23 update):

PS C:\Users\kptnkman> adb -s 192.168.1.29:5555 shell getprop
[ro.build.ab_update]: [true]
[ro.build.branch]: [releases-oculus-10.0-hollywood-v23]
[ro.build.characteristics]: [default]
[ro.build.date]: [Wed Nov 11 06:40:57 PST 2020]
[ro.build.date.utc]: [1605105657]
[ro.build.description]: [hollywood-user 10 QP1A.190711.020 12018400076000000 release-keys]
[ro.build.display.id]: [user-1201840.7600.0]
[ro.build.fingerprint]: [oculus/hollywood/hollywood:10/QP1A.190711.020/12018400076000000:user/release-keys]
[ro.build.flavor]: [hollywood-user]
[ro.build.host]: [sandcastle229.atn6.facebook.com]
[ro.build.id]: [QP1A.190711.020]
[ro.build.product]: [hollywood]
[ro.build.shutdown_timeout]: [0]
[ro.build.tags]: [release-keys]
[ro.build.type]: [user]
[ro.build.user]: [twsvcscm]
[ro.build.version.all_codenames]: [REL]
[ro.build.version.base_os]: []
[ro.build.version.codename]: [REL]
[ro.build.version.incremental]: [12018400076000000]
[ro.build.version.min_supported_target_sdk]: [23]
[ro.build.version.preview_sdk]: [0]
[ro.build.version.preview_sdk_fingerprint]: [REL]
[ro.build.version.release]: [10]
[ro.build.version.sdk]: [29]
[ro.build.version.security_patch]: [2019-09-05]
KptnKMan commented 3 years ago

So after some more poking around, I got the prompt to work! Unfortunately, when the prompt appears, the screen is blank for all casting and scrcpy. So I took a picture through the lens with my phone: image

This seems like its almost there. I can't seem to scroll down to hit the "Start Now" button. Does anyone know a way to confirm it (While using touch controllers)?

@rom1v Do you have any advice?

Edit: I can scroll down, but the button does not show still. Any other way to confirm? image

Sp00nb0b-cyber commented 3 years ago

i have that same issue, i just cant get the button, be great for a work around

KptnKMan commented 3 years ago

Does anyone know if this prompt is part of the Android 10 OS or is this part of sndcpy?

Is it possible to remove the prompt or have a --confirm flag on sndcpy?

rom1v commented 3 years ago

Does anyone know if this prompt is part of the Android 10 OS or is this part of sndcpy?

Android.

KptnKMan commented 3 years ago

Thanks @rom1v I appreciate the answer. I think I found the code here to manage this prompt: https://github.com/rom1v/sndcpy/blob/f0995ca4d0002ef288c74c70d4c0eb48354449d5/app/src/main/java/com/rom1v/sndcpy/MainActivity.java#L16-L27

In your opinion, is it possible to auto-approve that request?

rom1v commented 3 years ago

The whole point of this code is to request the permission to the system (which requests to the user to accept by opening a dialog).

So it's not possible to accept programmatically from the Android app which requests the permissions (or the user permission mechanism would be severely flawed).

However, it's probably possible to click on "accept" by injecting keyboard events (with scrcpy for example) or using UIAutomator (started with shell rights).

KptnKMan commented 3 years ago

The whole point of this code is to request the permission to the system (which requests to the user to accept by opening a dialog). So it's not possible to accept programmatically from the Android app which requests the permissions (or the user permission mechanism would be severely flawed).

Yeah I totally understand that functionality, and its intent of course, just trying to find a workaround that will work with Oculus Touch controllers. ¯\_(ツ)_/¯

However, it's probably possible to click on "accept" by injecting keyboard events (with scrcpy for example) or using UIAutomator (started with shell rights).

Do you know the keyboard command required? Is it just an Enter command, or some particular key its expecting?

I can try test the scrcpy method and see if that works. I also see that UIAutomator uses ADB transparently, so I can poke around with that also.

rom1v commented 3 years ago

(By UIAutomator, I was referring to https://developer.android.com/training/testing/ui-automator)

KptnKMan commented 3 years ago

Oh sorry, thanks for the link, I'll take a look.

omeryagmurlu commented 3 years ago

I've managed to confirm the prompt using a bluetooth keyboard. Connect your keyboard through experimental features and when you get the prompt press the windows button. It cuts out all the audio from the quest but I was still unable to get any sound from my computer. It just says playing audio and quits

edit: I get the same behaviour with my s9 plus too, so it's probably something I'm doing wrong. I'll edit this after I figure out what's wrong. It works on my other android devices now, so that was not the culprit After confirming the prompt and pressing enter in the terminal, sndcpy displays Playing audio... and after 2 seconds quits. Strange thing here is that after confirming the prompt, quest's sound cuts off and doesn't come back till I change the volume level.

Taeronwastaken commented 3 years ago

Ive got the .bat file and then i open it while scrcpy is running. I am able to close the the prompt and when i do, the bat closes and no audio. Is this a clue to help the situation?

Paxy commented 3 years ago

Hi, but trying to troubleshoot Quest2 issue of sndcpy got following error messages in logcat:

05-12 14:57:21.799 735 1701 E ServiceUtilities: Request denied by app op: 27 05-12 14:57:21.799 735 1701 E AudioPolicyIntefaceImpl: startInput permission denied: recording not allowed for uid 10098 pid 26824 05-12 14:57:21.799 735 1701 W AudioFlinger: start(214): startInput failed, status -1 05-12 14:57:21.803 1481 1481 D OculusNotificationListenerService: Notification posted: 0|com.rom1v.sndcpy|1|null|10098 05-12 14:57:21.803 26824 27327 E AudioRecord: start(254): status -1 05-12 14:57:21.803 2515 2515 I [OAO] VrNotificationListenerService: onNotificationPosted(): key = 0|com.rom1v.sndcpy|1|null|10098, type = third_party 05-12 14:57:21.803 2515 2515 I [OAO] VrNotificationListenerService: onNotificationPosted(): user|package|tag|id = UserHandle{0}|com.rom1v.sndcpy|null|1

sndcpy app asked for permission and I was able to confirm it (unlike the similar post above). But when I try to connect to it using VLC, no data was received over the TCP socket, and the above error is logged in the Quest log.

Any idea how to allow additional permission for this app to be able to start recording?

hlavavit commented 3 years ago

About the confirming of said permission, had same issue. I found a solution and posted it here https://forums.oculusvr.com/t5/Oculus-Quest-Development/Problem-with-MediaProjection-and-permission-dialog/m-p/874818#M3307

basicly oculus screws up the dialog if it is shown from regulard activity, it must be from VR activity (fullscreen)

lividhen commented 3 years ago

I'm getting an issue where after running sndcpy.bat, inside the quest it says "sndcpy keeps stopping". Running "adb logcat | grep sndcpy" gives me this:

logcat ``` 10-03 14:37:07.433 2728 2728 I [SUI] AnytimeUIAndroidPanelApp: handleGenericMessage appStatus {"appState":[{"name":"com.oculus.explore/com.oculus.explore.PanelService","panels":[{"panelId":6,"state":"Background"}]},{"name":"com.android.settings/com.oculus.vrshell.desktop.PanelService","panels":[{"panelId":8,"state":"Background"}]},{"name":"com.oculus.store/com.oculus.store.PanelService","panels":[{"panelId":54,"state":"Background"}]},{"name":"dxidev.openhiddensettings/com.oculus.vrshell.desktop.PanelService","panels":[{"panelId":29,"state":"Background"}]},{"name":"com.oculus.tv/com.oculus.tv.PanelService","panels":[{"panelId":53,"state":"Background"}]},{"name":"com.weloveoculus.BMBF/com.oculus.vrshell.desktop.PanelService","panels":[{"panelId":55,"state":"Background"}]},{"name":"com.rom1v.sndcpy/com.oculus.vrshell.desktop.PanelService","panels":[{"panelId":70,"state":"Focused"}]}],"mostRecent":["com.rom1v.sndcpy/com.oculus.vrshell.desktop.PanelService","com.weloveoculus.BMBF/com.oculus.vrshell.desktop.PanelService","com.oculus.store/com.oculus.store.PanelService","com.oculus.tv/com.oculus.tv.PanelService","dxidev.openhiddensettings/com.oculus.vrshell.desktop.PanelService","com.oculus.explore/com.oculus.explore.PanelService","com.android.settings/com.oculus.vrshell.desktop.PanelService"]} 10-03 14:37:07.463 2125 2153 I LibraryProvider: running queryApp(com.rom1v.sndcpy) for UID 10026 10-03 14:37:07.475 2125 2153 W LibraryProvider: Cannot find entitlement for com.rom1v.sndcpy 10-03 14:37:07.476 2728 2728 W OVRLibrary: null cursor received for query content://com.oculus.ocms.library/apps/com.rom1v.sndcpy ```

os version 33.0.0.69.367.320877141

cwrau commented 2 years ago

I'm hitting the same problems on my Oculus Quest too, although it is on Android 10

I'm getting the following stacktrace from adb logcat;

E AndroidRuntime: FATAL EXCEPTION: main                                                                                                                                                                                                                                                                                                                                  
E AndroidRuntime: Process: com.rom1v.sndcpy, PID: 12431                                                                                                                                                                                                                                                                                                                  
E AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.rom1v.sndcpy/com.rom1v.sndcpy.MainActivity}: android.content.ActivityNotFoundException: Unable to find explicit activity class {com.android.systemui/com.android.systemui.media.MediaProjectionPermissionActivity}; have you declared this activity in your AndroidManifest.xml?
E AndroidRuntime: at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3279)                                                                                                                                                                                                                                                                          
E AndroidRuntime: at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3418)                                                                                                                                                                                                                                                                           
E AndroidRuntime: at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:83)                                                                                                                                                                                                                                                                
E AndroidRuntime: at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)                                                                                                                                                                                                                                                    
E AndroidRuntime: at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)                                                                                                                                                                                                                                                              
E AndroidRuntime: at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2025)                                                                                                                                                                                                                                                                                
E AndroidRuntime: at android.os.Handler.dispatchMessage(Handler.java:107)                                                                                                                                                                                                                                                                                                
E AndroidRuntime: at android.os.Looper.loop(Looper.java:214)                                                                                                                                                                                                                                                                                                             
E AndroidRuntime: at android.app.ActivityThread.main(ActivityThread.java:7389)                                                                                                                                                                                                                                                                                           
E AndroidRuntime: at java.lang.reflect.Method.invoke(Native Method)                                                                                                                                                                                                                                                                                                      
E AndroidRuntime: at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)                                                                                                                                                                                                                                                                   
E AndroidRuntime: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)                                                                                                                                                                                                                                                                                        
E AndroidRuntime: Caused by: android.content.ActivityNotFoundException: Unable to find explicit activity class {com.android.systemui/com.android.systemui.media.MediaProjectionPermissionActivity}; have you declared this activity in your AndroidManifest.xml?                                                                                                         
E AndroidRuntime: at android.app.Instrumentation.checkStartActivityResult(Instrumentation.java:2056)                                                                                                                                                                                                                                                                     
E AndroidRuntime: at android.app.Instrumentation.execStartActivity(Instrumentation.java:1716)                                                                                                                                                                                                                                                                            
E AndroidRuntime: at android.app.Activity.startActivityForResult(Activity.java:5211)                                                                                                                                                                                                                                                                                     
E AndroidRuntime: at android.app.Activity.startActivityForResult(Activity.java:5169)                                                                                                                                                                                                                                                                                     
E AndroidRuntime: at com.rom1v.sndcpy.MainActivity.onCreate(MainActivity.java:26)                                                                                                                                                                                                                                                                                        
E AndroidRuntime: at android.app.Activity.performCreate(Activity.java:7825)                                                                                                                                                                                                                                                                                              
E AndroidRuntime: at android.app.Activity.performCreate(Activity.java:7814)                                                                                                                                                                                                                                                                                              
E AndroidRuntime: at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1306)                                                                                                                                                                                                                                                                         
E AndroidRuntime: at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3254)                                                                                                                                                                                                                                                                          
E AndroidRuntime: ... 11 more                                                                                                                                                                                                                                                                                                                                            
ASchneiderBR commented 2 years ago

Hello everyone. Did you guys managed to make sndcpy to work with a Meta Quest 2? Thanks!

cwrau commented 2 years ago

I was trying this again today, and the error message changed;

06-11 13:49:45.146  1000  1030 I ActivityManager: Start proc 7064:com.rom1v.sndcpy/u0a100 for activity {com.rom1v.sndcpy/com.rom1v.sndcpy.MainActivity}
06-11 13:49:45.210  7064  7064 E om.rom1v.sndcp: Not starting debugger since process cannot load the jdwp agent.
06-11 13:49:45.456  2067  2067 D VrRuntimeService: onForegroundActivitiesChanged: pid=7064, uid=10100, hasForegroundActivities=true
06-11 13:49:46.027  7064  7085 I AdrenoGLES-0: QUALCOMM build                   : feb9270, If5f0b160d1
06-11 13:49:46.027  7064  7085 I AdrenoGLES-0: Build Date                       : 06/03/21
06-11 13:49:46.027  7064  7085 I AdrenoGLES-0: OpenGL ES Shader Compiler Version: EV031.35.05.00
06-11 13:49:46.027  7064  7085 I AdrenoGLES-0: Local Branch                     : 
06-11 13:49:46.027  7064  7085 I AdrenoGLES-0: Remote Branch                    : refs/tags/AU_LINUX_ANDROID_LA.UM.8.4.R1.10.00.00.559.062
06-11 13:49:46.027  7064  7085 I AdrenoGLES-0: Remote Branch                    : NONE
06-11 13:49:46.027  7064  7085 I AdrenoGLES-0: Reconstruct Branch               : NOTHING
06-11 13:49:46.027  7064  7085 I AdrenoGLES-0: Build Config                     : S L 8.0.12 AArch64
06-11 13:49:46.027  7064  7085 I AdrenoGLES-0: Driver Path                      : /system/vendor/lib64/egl/libGLESv2_adreno.so
06-11 13:49:46.032  7064  7085 I AdrenoGLES-0: PFP: 0x005ff114, ME: 0x005ff066
06-11 13:49:46.279  7064  7064 D AndroidRuntime: Shutting down VM
06-11 13:49:46.280  7064  7064 E AndroidRuntime: FATAL EXCEPTION: main
06-11 13:49:46.280  7064  7064 E AndroidRuntime: Process: com.rom1v.sndcpy, PID: 7064
06-11 13:49:46.280  7064  7064 E AndroidRuntime: java.lang.RuntimeException: Unable to start service com.rom1v.sndcpy.RecordService@e52c0c3 with Intent { act=com.rom1v.sndcpy.RECORD cmp=com.rom1v.sndcpy/.RecordService (has extras) }: java.lang.UnsupportedOperationException: Error: could not register audio policy
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:4114)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at android.app.ActivityThread.access$1800(ActivityThread.java:226)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1900)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at android.os.Handler.dispatchMessage(Handler.java:107)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at android.os.Looper.loop(Looper.java:214)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at android.app.ActivityThread.main(ActivityThread.java:7391)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at java.lang.reflect.Method.invoke(Native Method)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936)
06-11 13:49:46.280  7064  7064 E AndroidRuntime: Caused by: java.lang.UnsupportedOperationException: Error: could not register audio policy
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at android.media.AudioRecord$Builder.buildAudioPlaybackCaptureRecord(AudioRecord.java:661)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at android.media.AudioRecord$Builder.build(AudioRecord.java:681)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at com.rom1v.sndcpy.RecordService.createAudioRecord(RecordService.java:155)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at com.rom1v.sndcpy.RecordService.startRecording(RecordService.java:159)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at com.rom1v.sndcpy.RecordService.onStartCommand(RecordService.java:86)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:4096)
06-11 13:49:46.280  7064  7064 E AndroidRuntime:    ... 8 more
06-11 13:49:46.306  7064  7064 I Process : Sending signal. PID: 7064 SIG: 9
janek commented 1 year ago

Has anyone managed to find a solution, or workaround without a bluetooth keyboard? Also, why doesn't scrcpy (seemingly) have to ask for these permissions?