gujjwal00 / avnc

VNC Client for Android
GNU General Public License v3.0
598 stars 57 forks source link

Black screen when running on Asus Transformer Prime (4.1.1 :D) #85

Closed exikyut closed 2 years ago

exikyut commented 2 years ago

Hi,

I just came across an old TF201, and given how surprisingly (and sadly) unusable the ancient Android version it's running is, getting a decent VNC client installed is one of the more relevant things that would make this device remotely useful. I found an old version of RealVNC (it's headscratchingly similar to the latest version in terms of features and UI - huh, ok), but that client was never fast, so I wondered how hard it would be to get AVNC working.

Surprisingly, dragging the build down to API 16 was actually quite trivial; mostly editing some XML and stubbing out just a couple of unavailable API calls.

Unfortunately once I got past that stage I've now found that while I can connect and drag the mouse around, I can't see anything; just a black square where the remote desktop should be.

I/NativeVnc(23974): VNC server supports protocol version 3.8 (viewer 3.8)
I/NativeVnc(23974): We have 1 security types to read
I/NativeVnc(23974): 0) Received security type 1
I/NativeVnc(23974): Selecting security type 1 (0/1 in the list)
I/NativeVnc(23974): Selected Security Scheme 1
I/NativeVnc(23974): No authentication needed
I/NativeVnc(23974): VNC authentication succeeded
I/NativeVnc(23974): Desktop name "x11"
I/NativeVnc(23974): Connected to VNC server, using protocol version 3.8
I/NativeVnc(23974): VNC server default format:
I/NativeVnc(23974):   32 bits per pixel.
I/NativeVnc(23974):   Least significant byte first in each pixel.
I/NativeVnc(23974):   TRUE colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0
I/ClipboardService(19948): Got clipboard for user=0
I/ClipboardService(19948): Got clipboard for user=0
I/ActivityManager(19948): Displayed com.gaurav.avnc.debug/com.gaurav.avnc.ui.vnc.VncActivity: +288ms
I/ClipboardService(19948): Got clipboard for user=0
I/ClipboardService(19948): Got clipboard for user=0
D/ShaderCompiler(23974): Program [3] validation result: 1
D/ShaderCompiler(23974): Program [3] validation log: 
D/ShaderCompiler(23974): P1202: Texture's gl states do not match with shader's

On the one hand, "noooo! a GPU bug!"

On the other hand, at least there's something in the log (?).

Googling the error message finds the tiniest handful of links. https://jausoft.com/blog/2011/10/04/bug-hunting-on-android/ is the most interesting; it mentions that

mixing sampler2D w/ a non texture code path let’s GLSL compiler result in error P1202 Texture's gl states do not match with shader's

I'm completely out of my depth here, but googling "uniform sampler2d" tegra found me https://forums.developer.nvidia.com/t/sampler2d-in-vertexshader/29120, where someone official-looking says that "Tegra does not support vertex texture fetch" - not sure if that's what's relevant here (wow graphics is a rabbithole).

Unsure if https://stackoverflow.com/questions/6801832/android-textures-gl-states-do-not-match-with-shaders has anything interesting in it

https://blog.techlab-xe.net/tegra2%E3%81%A7%E3%81%AE%E4%B8%8D%E5%85%B7%E5%90%88/ is in Japanese but despite Google Translate doing a surprisingly good job at it, I don't think there's anything too interesting in here

https://4pda.to/forum/index.php?showtopic=418429&st=1800 mentions the same error but is sadly in Russian and the quality of the translation (plus my unfamiliarity with the subject matter) leaves me somewhat unmotivated to try and figure out if there is anything useful here

TIL AVNC is as fast as it is because it uses native TurboVNC and OpenGL :) - with the flip side of that coin being that I don't think there's a non-GL alternative "slow path" (which makes a lot of sense; the reasonable alternative is obviously "just use a different client").

I'm not entirely sure if adding "Supports Android 4.1!" to the feature matrix is actually a respectable achievement :smile: but I do remain a tiny bit curious about how straightforward it might be demonstrate a tentative proof of concept. (Maybe leave some breadcrumbs - a git revision and some modification instructions - for the next motivated person to work off of, perhaps.)

Perhaps there's a "uses GL but disables acceleration" startpoint I could work off of?

gujjwal00 commented 2 years ago

Hi @exikyut thanks for reporting this. I am happy to help in getting AVNC work on your device. I too like to keep my old devices alive for as long as possible (I still have a Pentium Dual Core system alive on Ubuntu 16.04).

AVNC uses really simple shaders (just 3 lines in total), so they might not be the issue here. Unfortunately, debugging graphics issues is really difficult without access the the actual device.

According to the 1st blog post you linked, the validation message you highlighted and the screen being blank might be two different issues. Can you please post the complete logs from the device? Maybe something else is broken?

TIL AVNC is as fast as it is because it uses native TurboVNC and OpenGL :) - with the flip side of that coin being that I don't think there's a non-GL alternative "slow path" (which makes a lot of sense; the reasonable alternative is obviously "just use a different client").

AVNC uses native LibVNCClient as VNC library and OpenGL for rendering, but another major factor is libjpeg-turbo which enables Tight encoding and provides SIMD-accellerated JPEG decoding. I can't praise it enough :1st_place_medal: !

I have been thinking about adding OpenGL ES 1 rendering as an alternative, but I don't know how much effort/time it will take.

I'm not entirely sure if adding "Supports Android 4.1!" to the feature matrix is actually a respectable achievement smile but I do remain a tiny bit curious about how straightforward it might be demonstrate a tentative proof of concept. (Maybe leave some breadcrumbs - a git revision and some modification instructions - for the next motivated person to work off of, perhaps.)

Yes, I don't want to lower the minimum API, because its not worth the effort in my opinion. But I can create a separate branch to hold these patches and release "experimental" APKs on GitHub if you want (assuming we can fix this issue).

exikyut commented 2 years ago

Hi @exikyut thanks for reporting this. I am happy to help in getting AVNC work on your device. I too like to keep my old devices alive for as long as possible (I still have a Pentium Dual Core system alive on Ubuntu 16.04).

Wow, I wasn't expecting this response :D

AVNC uses really simple shaders (just 3 lines in total), so they might not be the issue here. Unfortunately, debugging graphics issues is really difficult without access the the actual device.

I see the 3 lines in question in Shaders.kt in https://github.com/gujjwal00/avnc/tree/master/app/src/main/java/com/gaurav/avnc/ui/vnc/gl/... but then I wonder if any of the other files might somehow be involved too...

According to the 1st blog post you linked, the validation message you highlighted and the screen being blank might be two different issues. Can you please post the complete logs from the device? Maybe something else is broken?

FWIW, here's the result of adb reboot && adb logcat and then starting AVNC once the device reboots: tf201-logcat.txt

TIL AVNC is as fast as it is because it uses native TurboVNC and OpenGL :) - with the flip side of that coin being that I don't think there's a non-GL alternative "slow path" (which makes a lot of sense; the reasonable alternative is obviously "just use a different client").

AVNC uses native LibVNCClient as VNC library and OpenGL for rendering, but another major factor is libjpeg-turbo which enables Tight encoding and provides SIMD-accellerated JPEG decoding. I can't praise it enough 1st_place_medal !

Ah, I keep forgetting about that because of the association to TurboVNC, just feels so unintuitive to use given that it's Java... "look at me firing up the JVM for extra performance" lol

One of these days TigerVNC needs to look at it...

I have been thinking about adding OpenGL ES 1 rendering as an alternative, but I don't know how much effort/time it will take.

Huh. I wonder if it would provide better performance on modern devices? Awkward mumbling about Vulkan and compatibility

I'm not entirely sure if adding "Supports Android 4.1!" to the feature matrix is actually a respectable achievement smile but I do remain a tiny bit curious about how straightforward it might be demonstrate a tentative proof of concept. (Maybe leave some breadcrumbs - a git revision and some modification instructions - for the next motivated person to work off of, perhaps.)

Yes, I don't want to lower the minimum API, because its not worth the effort in my opinion. But I can create a separate branch to hold these patches and release "experimental" APKs on GitHub if you want (assuming we can fix this issue).

Wow, again, that's a higher level of support than I was expecting :D

Obviously users would be partially on their own, but still, this is very nice. Thanks!


So, over the past couple days I was thinking "I know, let's provide remote access!"

Ughhh. After fiddling with app_process (which just yells Segmentation fault at you all day when it doesn't want to come out and play) I finally got scrcpy to the point where

$ CLASSPATH=/data/local/tmp/scrcpy-server.jar ANDROID_DATA=/data/local/tmp app_process / com.genymobile.scrcpy.Server 1.24 log_level=info bit_rate=8000000 tunnel_forward=true     
[server] INFO: Device: asus Transformer Prime TF201 (Android 4.1.1)
[server] ERROR: Exception on thread Thread[main,5,main]
java.lang.AssertionError: java.lang.ClassNotFoundException: android.hardware.display.IDisplayManager$Stub
        at com.genymobile.scrcpy.wrappers.ServiceManager.getService(ServiceManager.java:40)
        at com.genymobile.scrcpy.wrappers.ServiceManager.getDisplayManager(ServiceManager.java:53)
        at com.genymobile.scrcpy.Device.<init>(Device.java:69)
        at com.genymobile.scrcpy.Server.scrcpy(Server.java:65)
        at com.genymobile.scrcpy.Server.main(Server.java:335)
        at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
        at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:235)
        at dalvik.system.NativeStart.main(Native Method)
Caused by: java.lang.ClassNotFoundException: android.hardware.display.IDisplayManager$Stub
        at java.lang.Class.classForName(Native Method)
        at java.lang.Class.forName(Class.java:217)
        at java.lang.Class.forName(Class.java:172)
        at com.genymobile.scrcpy.wrappers.ServiceManager.getService(ServiceManager.java:37)
        ... 7 more
Caused by: java.lang.NoClassDefFoundError: android/hardware/display/IDisplayManager$Stub
        ... 11 more
Caused by: java.lang.ClassNotFoundException: android.hardware.display.IDisplayManager$Stub
        at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:61)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:501)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:461)
        ... 11 more

Which it turns out there's very little I can do about, as Android 4.1 doesn't have android.hardware.display: https://android.googlesource.com/platform/frameworks/base.git/+/android-4.1.1_r6/core/java/android/hardware/ (while 4.2 does: https://android.googlesource.com/platform/frameworks/base.git/+/android-4.2_r1/core/java/android/hardware/). Ugh.

This is legitimately annoying, as...

 $ time screencap ss.png                      
    0m5.95s real     0m5.80s user     0m0.04s system

...taking screenshots using the built-in tools is... mind-numbingly (confusingly) slow. I'm guessing it's because it's firing up the framework each time. Argh.

(And wow, there's not a single /dev/fbX in sight, even if the device *was* rooted.)

Unfortunately screenrecord is unavailable too. Yay!

So I'm now currently at the point of going full scorched-earth with the micro-HDMI port. I have an HDMI capture dongle here.... I'm considering ordering a micro-HDMI cable (it'll probably come in handy for other things) and then seeing what can be done with that.

I'm idly curious if there are simple ways to do image or video capture in software - but only to a limited extent, because of the effective dead-ness of Android 4.x.

gujjwal00 commented 2 years ago

Lets first try to get clean shader validation logs.

From the SO Question, the message might be due to validation being done before texture is generated. I have pushed a new branch https://github.com/gujjwal00/avnc/tree/tegra-gpu , where texture is generated before shader compilation.

Please rebase your changes on this branch and try again. Hopefully, the message "P1202: Texture's gl states do not match with shader's" will be fixed by this change.

If we are lucky this will also fix the black screen, but I have a feeling it won't.

exikyut commented 2 years ago

Nooo D:

Sadly there is unfortunately no change:

D/ShaderCompiler( 3829): Program [3] validation result: 1
D/ShaderCompiler( 3829): Program [3] validation log: 
D/ShaderCompiler( 3829): P1202: Texture's gl states do not match with shader's

I uninstalled the app off the device to be sure it was being updated.

So to be clear, both the black screen and the validation message are both still present.

And for the purposes of disambiguation, my "rebase" consisted of moving val textureId = createTexture() to the top of this block:

    val textureId = createTexture()
    val program = ShaderCompiler.buildProgram(Shaders.VERTEX_SHADER, Shaders.FRAGMENT_SHADER)
    val aPositionLocation = glGetAttribLocation(program, A_POSITION)
    val aTextureCoordinatesLocation = glGetAttribLocation(program, A_TEXTURE_COORDINATES)
    val uProjectionLocation = glGetUniformLocation(program, U_PROJECTION)
    val uTexUnitLocation = glGetUniformLocation(program, U_TEXTURE_UNIT)

and adding a new glActiveTexture(GL_TEXTURE0) and commenting out the glBindTexture():

        glActiveTexture(GL_TEXTURE0)
        glBindTexture(GL_TEXTURE_2D, texturesObjects[0])
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR)
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR)
        //glBindTexture(GL_TEXTURE_2D, 0)
        return texturesObjects[0]
gujjwal00 commented 2 years ago

Well, that's unfortunate! Whats even more confusing is the line Program [3] validation result: 1. 1 is the value of GL_TRUE which implies that shaders were validated successfully. This is why I originally thought that the last line was not the reason for black screen.

I will try to find other leads.

gujjwal00 commented 2 years ago

Hi @exikyut, can you please grab libGLESv2_tegra.so from your device (probably at /vendor/lib/egl/libGLESv2_tegra.so) and upload it here?

exikyut commented 2 years ago

Wow, this thing doesn't even have find. Yikes. ls -lR to the rescue; it was actually in /system/lib.

Cue me going "ooh these other things are all interesting" and then they aren't actually relevant, but I came across a few GLES-related files while looking for the one you mentioned. I note that libGLESv2_tegra.so is rather tiny compared to libGLESv2_tegra_impl.so - so either you actually wanted _impl, or you were just looking for symbols or something and please ignore me :)

    17636  2022-07-10 23:04   tegra/system_lib_egl/libGLESv1_CM_tegra.so
       23  2022-07-10 23:04   tegra/system_lib_egl/egl.cfg
   890560  2022-07-10 23:04   tegra/system_lib_egl/libEGL_tegra_impl.so
    83188  2022-07-10 23:04   tegra/system_lib_egl/libGLES_android.so
     9492  2022-07-10 23:04   tegra/system_lib_egl/libEGL_tegra.so
   622216  2022-07-10 23:04   tegra/system_lib_egl/libGLESv2_tegra_impl.so
   767372  2022-07-10 23:04   tegra/system_lib_egl/libGLESv2_perfhud.so
   105128  2022-07-10 23:04   tegra/system_lib_egl/libGLESv1_CM_tegra_impl.so
    19860  2022-07-10 23:04   tegra/system_lib_egl/libEGL_perfhud.so
    17632  2022-07-10 23:04   tegra/system_lib_egl/libGLESv2_tegra.so
    17636  2022-07-10 23:04   tegra/system_lib_egl/libGLESv1_CM_perfhud.so
  3844721  2022-07-10 22:59   tegra/ls-lR.txt
        0  2022-07-10 23:05   tegra/system_lib/
   266188  2022-07-10 23:04   tegra/system_lib/libGLES_trace.so
    21488  2022-07-10 23:05   tegra/system_lib/libGLESv2.so
    21488  2022-07-10 23:05   tegra/system_lib/libGLESv1_CM.so

tegra.zip

ls-lR is included for completeness. Of course it got very distracted with the contents of /proc etc.

gujjwal00 commented 2 years ago

Thanks for including all files. As you suspected, validation routine was in libGLESv2_tegra_impl.so. Unfortunately, I could not identify the exact reason for the error because there are a lot of checks and internal constants.

I think the only option now is to manually debug/isolate the issue.

I have pushed another commit https://github.com/gujjwal00/avnc/commit/61a4ea50326d6b37eb990d6066fb526f6dc50e63 (patch), which disables the use of sampler2D. Please apply this patch and run the app. Hopefully you will see a color pattern. You should be able to zoom/scroll the pattern.

exikyut commented 2 years ago

!!!!! :D

yay

I can indeed zoom the pattern.

I can't pan it around, but two-finger pinching it zooms it relative to the center of the screen.

Wow, this is fascinating. Where'd the pattern come from?!

By the way, I just realized you tried to disassemble the GLES library. Thanks for the effort you're making :3

Oh, and for completeness,

I/NativeVnc(24042): VNC server supports protocol version 3.8 (viewer 3.8)
I/NativeVnc(24042): We have 1 security types to read
I/NativeVnc(24042): 0) Received security type 1
I/NativeVnc(24042): Selecting security type 1 (0/1 in the list)
I/NativeVnc(24042): Selected Security Scheme 1
I/NativeVnc(24042): No authentication needed
I/NativeVnc(24042): VNC authentication succeeded
I/NativeVnc(24042): Desktop name "x11"
I/NativeVnc(24042): Connected to VNC server, using protocol version 3.8
I/NativeVnc(24042): VNC server default format:
I/NativeVnc(24042):   32 bits per pixel.
I/NativeVnc(24042):   Least significant byte first in each pixel.
I/NativeVnc(24042):   TRUE colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0
I/ClipboardService( 9228): Got clipboard for user=0
I/ClipboardService( 9228): Got clipboard for user=0
I/ClipboardService( 9228): Got clipboard for user=0
I/ClipboardService( 9228): Got clipboard for user=0
D/ShaderCompiler(24042): Program [3] validation result: 1
D/ShaderCompiler(24042): Program [3] validation log: 
I/ActivityManager( 9228): Displayed com.gaurav.avnc.debug/com.gaurav.avnc.ui.vnc.VncActivity: +330ms
I/dalvikvm(24042): Could not find method android.view.View$BaseSavedState.<init>, referenced from method androidx.viewpager2.widget.ViewPager2$SavedState.<init>
W/dalvikvm(24042): VFY: unable to resolve direct method 2715: Landroid/view/View$BaseSavedState;.<init> (Landroid/os/Parcel;Ljava/lang/ClassLoader;)V
D/dalvikvm(24042): VFY: replacing opcode 0x70 at 0x0000
gujjwal00 commented 2 years ago

That is encouraging! Although the pattern should be smooth like this:

texture

But this can be figured out later.

I can't pan it around, but two-finger pinching it zooms it relative to the center of the screen.

Zoom-In until pattern is larger than the screen, then you will be able to pan it. Also, zooming happens relative to the center of touch points (i.e. the two fingers).

Wow, this is fascinating. Where'd the pattern come from?!

So instead of using the frame received from server, I switched to using fixed colors for fragments (i.e. screen pixels). Because You can still zoom/pan around, it implies that other parts of rendering pipeline are working correctly. Only texture rendering is somehow broken.

More importantly, that link to Russian forum might have given me a clue about validation message. I will push a commit in few minutes.

gujjwal00 commented 2 years ago

Here is the commit: https://github.com/gujjwal00/avnc/commit/17960f8ab3fa0da93ddebebe069f81943f2d5e9c (patch). Previous two commits mentioned in this issue should be reverted before applying this.

Now the shader program will be validated during render pass, so the P1202 validation message should not appear. And if we are lucky, real problem will be logged.

Let the app run for ~10 seconds on blank screen to allow multiple validation attempts.

exikyut commented 2 years ago

Sorry for the delay, initially thought I'd made a mistake reverting the previous two patches so waited until I'd had a second look. I'm not honestly sure - I applied the patch you linked after reverting everything, unfortunately that's got me back to a black screen again.

It's quite possible I made a mistake somewhere. Here's the current state of what I'm working from:

diff.txt

It's mostly XML changes and a couple of stubbed out function calls that I can figure out later.

As for the new changes, things have gone in a distinctly "nooo" direction.

I/NativeVnc(15220): VNC server supports protocol version 3.8 (viewer 3.8)
I/NativeVnc(15220): We have 1 security types to read
I/NativeVnc(15220): 0) Received security type 1
I/NativeVnc(15220): Selecting security type 1 (0/1 in the list)
I/NativeVnc(15220): Selected Security Scheme 1
I/NativeVnc(15220): No authentication needed
I/NativeVnc(15220): VNC authentication succeeded
I/NativeVnc(15220): Desktop name "x11"
I/NativeVnc(15220): Connected to VNC server, using protocol version 3.8
I/NativeVnc(15220): VNC server default format:
I/NativeVnc(15220):   32 bits per pixel.
I/NativeVnc(15220):   Least significant byte first in each pixel.
I/NativeVnc(15220):   TRUE colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0
I/ClipboardService(  420): Got clipboard for user=0
I/ClipboardService(  420): Got clipboard for user=0
I/ActivityManager(  420): Displayed com.gaurav.avnc.debug/com.gaurav.avnc.ui.vnc.VncActivity: +391ms
I/ClipboardService(  420): Got clipboard for user=0
I/ClipboardService(  420): Got clipboard for user=0
(1)
D/ShaderCompiler(15220): Program [3] validation result: 1
D/ShaderCompiler(15220): Program [3] validation log: 
D/ShaderCompiler(15220): P1202: Texture's gl states do not match with shader's
I/dalvikvm(15220): Could not find method android.view.View$BaseSavedState.<init>, referenced from method androidx.viewpager2.widget.ViewPager2$SavedState.<init>
W/dalvikvm(15220): VFY: unable to resolve direct method 2716: Landroid/view/View$BaseSavedState;.<init> (Landroid/os/Parcel;Ljava/lang/ClassLoader;)V
D/dalvikvm(15220): VFY: replacing opcode 0x70 at 0x0000
D/NsdService(  420): Stop service discovery
D/MDnsDS  (  115): Stopping discover with ref 0x40283a3c
D/MDnsDS  (  115): freeServiceRef 2
D/MDnsDS  (  115): triggering rescan
D/MDnsDS  (  115): Monitor poll got data pollCount = 2, 2
D/MDnsDS  (  115): Monitor found [1].revents = 32 - calling ProcessResults
D/MDnsDS  (  115): controlSocket shows revent= 1
D/MDnsDS  (  115): MDnsSdListener::Monitor got 1
D/MDnsDS  (  115): MDnsSdListener::Monitor poll rescanning - size=10, live=0
D/MDnsDS  (  115): Going to poll with pollCount 1
D/NsdService(  420): stopServiceDiscovery: 2
(2)
D/ShaderCompiler(15220): Program [3] validation result: 1
D/ShaderCompiler(15220): Program [3] validation log: 
D/ShaderCompiler(15220): P1202: Texture's gl states do not match with shader's
(3)
D/ShaderCompiler(15220): Program [3] validation result: 1
D/ShaderCompiler(15220): Program [3] validation log: 
D/ShaderCompiler(15220): P1202: Texture's gl states do not match with shader's
D/ShaderCompiler(15220): Program [3] validation result: 1
D/ShaderCompiler(15220): Program [3] validation log: 
D/ShaderCompiler(15220): P1202: Texture's gl states do not match with shader's
D/ShaderCompiler(15220): Program [3] validation result: 1
D/ShaderCompiler(15220): Program [3] validation log: 
D/ShaderCompiler(15220): P1202: Texture's gl states do not match with shader's
D/ShaderCompiler(15220): Program [3] validation result: 1
D/ShaderCompiler(15220): Program [3] validation log: 
D/ShaderCompiler(15220): P1202: Texture's gl states do not match with shader's

The first P1202 comes when the app starts up (1), then there's a second one (2) a couple seconds later. Then nothing. Then when I track my finger across the screen it starts printing more (3).

:<

gujjwal00 commented 2 years ago

Another dead end 😞!

Your diff looks normal to me.

Is it possible to setup remote debugging on this device? If adb over network can be enabled on this device, and it has an IPv6 address, I can connect via adb remotely and try different things.

exikyut commented 2 years ago

I don't think my ISP understands what an IPv6 is yet :smile_cat: but I think I might have something working here.

I must confess I've been fiddling with this for the past day or two and got hung up on the speed of taking screenshots - which on this device, is "oh okay (wow)" slooow. Capturing is fast, it's the encoding (JPG or PNG) that's torture. I actually tried getting libjpeg-turbo going, but *even that* was taking like 2 seconds (!) to encode. (Apparently /dev/graphics/fbX "is broken on ICS" and you have to do a proprietary Binder dance to get working screenshot data, so I was actually capturing what appeared to be remnants of the boot splashscreen - but that (mostly black screen) was what was taking 2 seconds to encode. I bailed at that point lol)

I eventually realized that PNG encoding time is directly proportional to the complexity of what's on the screen: the homescreen wallpaper will very reliably take 6 seconds to encode (D:), but if you're just staring at a black screen (oh, hey, like AVNC :/) the image will only take 1.9 seconds to encode, which is not entirely the end of the world.

Sooo,

ssh boop@z7.vxe6.net -N -L 127.0.0.1:5037:127.0.0.1:5037 -v

should work, possibly after an adb kill-server so SSH can bind and forward the port. (If I do adb tcpip locally then the connection would be going over Wi-Fi - the router's less than a meter away from the device but its Wi-Fi chip is... USB is ideal. lol. So I'm forwarding the copy of adbd running on the machine the device is connected to. Hopefully this isn't too inconvenient.)

Once connected, the current-best method to get screenshots that I've yet identified is

adb shell screencap -p /data/local/tmp/ss.png; adb pull /data/local/tmp/ss.png

Unfortunately Android 4.1's adb doesn't understand adb exec, and adb shell CRLF-scrambles the PNGs if you try and just capture them through stdout :(. So, gotta go through the filesystem. Hmph.

For what it's worth I'm actually just doing ./gradlew installDebug to get the APK installed, which handles launching it as well.

I'm not exactly sure how the timezones line up, but it's possible I might be around to poke at the device and help things along :)

Oh, right, nearly forgot - I added your GitHub key to SSH. By all means let me know if you'd prefer to use a different key.

gujjwal00 commented 2 years ago

Thanks.

So the idea is to run one or more tests from VncActivityTest. These tests use a dummy server so they don't depend on any external server. I will add some Thread.sleep() calls at the end of tests, run them, and then capture a screenshot to check if frame is rendered correctly.

Fingerprint of my adb client: 8b:85:92:f4:cd:aa:8e:e5:e1:26:9b:80:aa:94:f4:36 (Edit: Nevermind, this is not needed as adb server is running on your machine)

exikyut commented 2 years ago

Huh, something happened to the adb server. I just restarted it, and left while true; do adb start-server; sleep 15; done in a tmux session to keep it up. adb start-server seems to be a noop if adbd is already running, and sleep is nothing, so that should be fine.

And - facepalm, I'd completely forgotten about the server thing. Of course. Woops. For what it's worth there's actually a VNC server reachable at 172.29.202.85, just showing a blank desktop.

...and I now realize that I am of course tapping on the screen to interact and connect to the VNC server. Yay.

Ooooohhh. The app's been reinstalled using a different certificate so Android wiped the user data with the server entry in it.

Argh, scrcpy officially not supporting Android 4.x is a real inconvenience :/

OK, I added a test VNC server to the server list, which is just showing a grey desktop (xsetroot solid '#dddddd') so screenshots of it encode quickly (although of course the view is very much black for now). This server can be selected with

adb shell input tap 627 170

There's unfortunately another hitch: connected to the boring USB port on my PC, the battery isn't getting enough current to charge. It's slowly draining. It's noticeably emptier than it was last night. I'm pretty sure it was saying 43%, now it's at 28%.

I might have a solution to this problem. I recently learned about and ordered one of those USB Y-cables with two plugs that connect the data pins to one plug and the power pins to another (or both). If this cable works the way I hope it does (and the device doesn't rely on a USB PD-esque handshake that uses the data pins to ask for enough power from the charger), I should be able to get it enough current while it's attached over ADB. Now to just stare at the mailbox... it'll be here in AliExpress Time™ :P (so either a week or a fortnight - yeah, lol).

In the interim, if you can give me an idea what window of time you'd be looking to poke at this thing, I'll try and charge it outside that window. It only needs a couple of hours, so feel free to make that time window as wide as you like. If you feel like doing the time zone conversion, I'm in AEST and I find https://dateful.com/time-zone-converter very helpful.

The tablet's currently connected to ADB at the moment. I guess I'm unplugging it to charge it when it goes flat or when you let me know, whichever happens first :P

gujjwal00 commented 2 years ago

I tried for couple of hours in the morning, but could not even get the test running.

First, io.mockk library refused to compile for API<21, so I had to remove it and remove bunch of tests.

Then, due to the inclusion of testing framework, method count in APK went above 65k. So I had to enable multi-dexing. Which broke the testing framework because classloader could not find the test runner classes in main DEX file. So I had to keep everything inside main dex file.

Even after all that, test framework simply could not start the activities for some reason. And screencap is able to capture the the homescreen correctly but when AVNC is launched, only navigation bar is visible and rest of the screen is black.

So I thought its best to wait untill you are available, and we can try together.

exikyut commented 2 years ago

*Activity chime*

I was just reading through the thread wondering what would happen next. Apparently this :D!

Intimidated headdesk Regarding the testing headache... ouch. Complexity explosions FTL :<

Regarding the screen being black: yup, that's unfortunately what I've been staring at all this time. Try switching the shader back to the version that draws the test pattern. Everything else has been permanently black. :(

Edit: I wonder if there's somewhere we can chat in realtime? Hrm, I kind of like Telegram... I'm i336_ on there

gujjwal00 commented 2 years ago

Yeah we can chat on Telegram, let me ping you.

exikyut commented 2 years ago

I must sadly concede defeat to the tidal wave of unread channels I have buried in there :crying_cat_face: I can't see your ping. I can try and ping you?

gujjwal00 commented 2 years ago

I couldn't find you!

Me: https://t.me/GauravUjjwal

gujjwal00 commented 2 years ago

Fixed in https://github.com/gujjwal00/avnc/commit/c6716150256bc9e0d6f14a626282208c04ff032b

Summary: Tegra GPU seems to be very strict about when non-power-of-two textures can be sampled in fragment shader. If texture (i.e. server's screen resolution) is NPOT, and wrap mode is anything other than GL_CLAMP_TO_EDGE, sampler will always return black color.

exikyut commented 2 years ago

Meant to get back to this tab sooner, got sidetracked a bit.

I discovered the (no-root!) droidVNC-NG VNC server for Android a while back, and discovered that it's so fast I can create a hacky remote viewfinder by opening the camera app and streaming the display. It's absolutely awesome.

The old version of RealVNC I found - and I would presume newer versions also, given the speed the app runs at on all my devices (old and new) - manages around 1-2 FPS. A few other apps manage maybe 5-10 FPS or so, if that. AVNC wipes out every app on the Play Store (that are still available for this old device) and manages a stable 19-22 FPS. It's absolutely awesome that I can play with this.

Wanted to say thanks for taking the time to dive in to this issue pretty much immediately and figure out what was going on. It was a lot of fun, and extremely educational. Much appreciated!

gujjwal00 commented 2 years ago

Thanks @exikyut!

And yes, AVNC is better than a lot of apps available on Play Store in many areas. Just need to get some publicity to it.