DeviceFarmer / minicap

Stream real-time screen capture data out of Android devices.
Other
294 stars 76 forks source link

where is minicap.so for android sdk 33? #41

Closed summerlife closed 6 months ago

summerlife commented 2 years ago

i need minicap.so for android sdk 33. how can i get that file?

I saw https://github.com/DeviceFarmer/minicap/issues/40 I try to use minicap-debug.apk, but the screenshot interval is unstable in different scenarios and the screenshot interval is very long minicap-debug.apk doesn't meet my requirements,So I still need the screenshot of minicap. so for Android sdk 33。Can you help me

varundtsfi commented 2 years ago

Hi @xia-ming I guess @pcrepieux is working on it. I have also tried but did not get the frames.

summerlife commented 2 years ago

thanks,I hope this function can be realized as soon as possible

pcrepieux commented 2 years ago

I am on it but had to reconfigure a full AOSP build environment. Will let you know if I have something running properly. Meanwhile, I remind you there is an apk version of minicap which is just fine.

summerlife commented 2 years ago

Hi,@pcrepieux。I tried the APK, but the screenshot interval is unstable, which does not meet my needs

varundtsfi commented 2 years ago

I have tried to build aosp 13 and implement the suggestions but did not get the success.

I am able to get the minicap.so but i am not getting the correct displayID and because of that i am not getting the frames.

Now @pcrepieux can get something.

joao-borges commented 2 years ago

@pcrepieux @varundtsfi hello, do you guys have any update on this? I cannot use the APK for many reasons.

pcrepieux commented 2 years ago

I am currently stuck because of a compilation issue with AOSP (which is precisely the kind of thing I wanted to avoid with the apk). I'm trying get it work but don't have much time to spend on this task. I am curious, why can't you use the apk ?

joao-borges commented 2 years ago

@pcrepieux we tried the minicap apk in the past and it somehow interfered with the appium usage on our application. Since this was less of a priority, we rolled back to using SO and have not gone back ever since.

I appreciate if you could have any estimation on how long or hard it would be so I can communicate to my stakeholders and maybe push forward again for the apk solution.

varundtsfi commented 2 years ago

@pcrepieux @varundtsfi hello, do you guys have any update on this? I cannot use the APK for many reasons.

I have successfully built the Android AOSP with minicap by applying the new changes required in Android 13.

When i am trying to start minicap.so then due to correct displayID i am not able to get frames .

I am still trying to figure out the issues. @pcrepieux is also busy with other stuffs.

joao-borges commented 2 years ago

@varundtsfi I guess you guys are still busy just wanted to circle back here to check if there's any updates. Thanks.

varundtsfi commented 2 years ago

@pcrepieux do you have any updates please @joao-borges is chasing

summerlife commented 2 years ago

I'm still waiting for minicap.so for Android 13,Please let me know if there is any progress。thanks

JesseCodeBones commented 2 years ago

@xia-ming it seems the build for Android.mk failed, I am trying with soong build, but still in progress

pcrepieux commented 2 years ago

Looks like I am more or less in the same situation as @JesseCodeBones. @varundtsfi could you detail the steps that you used to properly build minicap.so ?

varundtsfi commented 2 years ago

@pcrepieux I have used the following steps to build the Android_13 Revision 1 I have checked out the branch repo init -u https://android.googlesource.com/platform/manifest -b android-13.0.0_r1 Once download the code then started the following to build the aosp cd Android_13_R1 Directory where I have downloaded the AOSP code then . build/envsetup.sh lunch 2 (aosp_arm64-eng) make -j4

Now copied the minicap code inside the external directory then again . build/envsetup.sh lunch 2 (aosp_arm64-eng) make -j4 minicap

I have sent an email to you for all the details.

pcrepieux commented 2 years ago

Thanks @varundtsfi I'll double check because you've apparently made it successfully the same way as I used to do 🤔

summerlife commented 2 years ago

If there is any progress, please inform me here . Thank you,@JesseCodeBones @pcrepieux @varundtsfi

JesseCodeBones commented 2 years ago

Looks like I am more or less in the same situation as @JesseCodeBones. @varundtsfi could you detail the steps that you used to properly build minicap.so ?

I finally passed the build of shared lib with PR: https://github.com/DeviceFarmer/minicap/pull/43 but the executable still have compatible issue. as the jpeg lib are involved as static lib, previous executable should also work. do you have time to have a test on it? it is late today, I will try to build executable and test this weekend.

varundtsfi commented 2 years ago

Hi @joao-borges I have tried this but getting only one frame then screen got stuck.

Are you facing the same issue?

JesseCodeBones commented 2 years ago

Hi @joao-borges I have tried this but getting only one frame then screen fot stuck.

Are you facing the same issue?

any console output or logcat logs?

pcrepieux commented 2 years ago

I finally passed the build of shared lib with PR: #43 but the executable still have compatible issue. as the jpeg lib are involved as static lib, previous executable should also work. do you have time to have a test on it? it is late today, I will try to build executable and test this weekend.

I won't be able to test right now, but will as soon as possible. Thanks @JesseCodeBones for starting this effort !

joao-borges commented 2 years ago

Hi @joao-borges I have tried this but getting only one frame then screen got stuck.

Are you facing the same issue?

@varundtsfi we faced different issues with minicap apk in the past and we abandoned it, but I don't remember of frames being lost.

We're re-implementing it because as a long term solution is better maintainable then the so library, so we'll see in the upcoming weeks.

varundtsfi commented 2 years ago

Hi @JesseCodeBones , @pcrepieux and @joao-borges I have validated the @JesseCodeBones intel libs which is working fine and I have build the same for arm64-v8a in which frame stuck issue I have reported also fixed now. Screenshot_2022-11-15_19-53-52 Now android-33 support is available for two architecture 1:- x86_64 2:- arm64-v8a I am building for armeabi-v7a and x86 soon I will add. @JesseCodeBones I you want to check arm64-v8a shared library you can download from the following link. https://github.com/varundtsfi/Android12Support_withso/tree/main/aosp/android-33

@pcrepieux Can I add in @JesseCodeBones https://github.com/DeviceFarmer/minicap/pull/43 or Do need to send another PR. else @JesseCodeBones you can download the binaries from the mentioned link and add those to your https://github.com/DeviceFarmer/minicap/pull/43

pcrepieux commented 2 years ago

That sounds great. Could you guys document a bit more how to build it for different architectures. #43 might be a good start for doing so. (I humbly confess I still am unable to properly build this on my side...)

varundtsfi commented 2 years ago
JesseCodeBones commented 2 years ago

umbly confess I still am unable to properly build this on my side..

Great work~~!! you can create another PR or reuse my PR as you want because my PR is a draft. you can reorgnize the code as you want (I assume we should do that).

For the minicap executable, the version 33 donot export turbojpeg.h any more, so I am still handling the development for that. I will create another issue.

ltwtns commented 1 year ago

Thanks all for the work. Wondering if there will be a so file for arm64?

varundtsfi commented 1 year ago

Thanks all for the work. Wondering if there will be a so file for arm64? You can get it from here and soon you can take it from here in master branch too.

https://github.com/varundtsfi/Android12Support_withso/tree/main/aosp%2Fandroid-33

remysabliet commented 1 year ago

Hello here,

It sounds that this repo is not really maintained anymore. We can see for example that the last updates date back to Jan, 8 2022. We can also see that SDK 31, 32, 33 architecture minicap.so files are missings for all 4 architectures.

@varundtsfi Nicely provided fixed built version for sdk31, 32 and 33 https://github.com/varundtsfi/Android12Support_withso => Working for SDK31, not tested on SDK32, SDK33)

This repository when imported, @devicefarmer/minicap-prebuilt 2.7.1 just provide compiled library (up to android-30), What should be the next step to integrate @varundtsfi works within this published repository ?

How do you manage in your side guys ? Are you just creating your own folder and droping all the compiled minicap.so in your side projects ?

Would you mind sharing your insights on this topic ?

宜しくお願いします

JesseCodeBones commented 1 year ago

Hello here,

It sounds that this repo is not really maintained anymore. We can see for example that the last updates date back to Jan, 8 2022. We can also see that SDK 31, 32, 33 architecture minicap.so files are missings for all 4 architectures.

@varundtsfi Nicely provided fixed built version for sdk31, 32 and 33 https://github.com/varundtsfi/Android12Support_withso => Working for SDK31, not tested on SDK32, SDK33)

This repository when imported, @devicefarmer/minicap-prebuilt 2.7.1 just provide compiled library (up to android-30), What should be the next step to integrate @varundtsfi works within this published repository ?

How do you manage in your side guys ? Are you just creating your own folder and droping all the compiled minicap.so in your side projects ?

Would you mind sharing your insights on this topic ?

宜しくお願いします

Agree, should we setup some a maintenance group and make this project more active? @varundtsfi @pcrepieux

varundtsfi commented 1 year ago

I will add all the missing minicap.so files but I'm a little bit busy. Just wait a little while soon and you will get all the missing things .

remysabliet commented 1 year ago

Thank you so much @varundtsfi , looking forward to it.

pcrepieux commented 1 year ago

I confess I am unlikely to invest a lot on the binary library since the apk does the job for stf. Contributions would of course be welcome but please keep in mind this is an open source project, so it should provide source code, and reproductible build instructions. Don't get me wrong. That's great if people managed to get the .so file and are willing to share it but that, I am afraid, is only half of the path.

JesseCodeBones commented 1 year ago

@pcrepieux totally understand the situation. for the SDK 33 issue, can we merge the PR https://github.com/DeviceFarmer/minicap/pull/43 ? we can refactor the structure in future, but currently, user can build lib file to their environment by themself. WDYT?

remysabliet commented 1 year ago

@pcrepieux when you say apk, are you talking about STFService.apk ?

@JesseCodeBones can we really build those files on a MAC for all 4 architectures for all android version up to 33 ? I thought a specific setup requiring C++ was required.

JesseCodeBones commented 1 year ago

@pcrepieux when you say apk, are you talking about STFService.apk ?

@JesseCodeBones can we really build those files on a MAC for all 4 architectures for all android version up to 33 ? I thought a specific setup requiring C++ was required.

we already had the build: https://github.com/varundtsfi/Android12Support_withso/tree/main/aosp%2Fandroid-33 I suppose it covered most platforms

remysabliet commented 1 year ago

In case in the close future this repo is "abandoned", i would like to try to build them myself.

I am waiting for the right to install NDK package from my security team then i will try it what is mentioned in the README file to build those binaries.

However I would like to know more about this apk that pcrepieux mentioned. Some people mentioned it doesnt conver everything so a bit doubtful but if there is nothing else supported would like to give a try as well.

pcrepieux commented 1 year ago

The apk is available as part of the minicap-prebuilt npm module: https://www.npmjs.com/package/@devicefarmer/minicap-prebuilt source code is here: https://github.com/DeviceFarmer/minicap/tree/master/experimental

Note that you will need a ndk to build minicap, but you will also need full AOSP build environment to build the .so (this is why I gave a try to an apk based approach)

remysabliet commented 1 year ago

@pcrepieux Thank you!

remysabliet commented 1 year ago

@pcrepieux Just Fyi, STF is using 3 binaries to stream device

Code Source

For a fujitsu device (FJ-41A) because the so file is missing for SDK31, if I do not add the minicap.so file manually we are not able to stream the device at all.

So it Sounds like apk alone is not enough for Streaming to work even for STF, all 3 type of files sounds necessary.

pcrepieux commented 1 year ago

@remysabliet, STF still uses the binary + API matching .so files but also falls back to the apk if it didn't work. It's been done like this at the early stage of the apk introduction in order to limit the risk of breaking something. It is possible to "force" this option to either favor bin or apk. see https://github.com/DeviceFarmer/stf/commit/bb1c20d1f7bc662fc155f84b0af3374f7c7aeabd for more information. That being said, there might be a bug. So if the information above didn't help, let me know.

summerlife commented 1 year ago

@pcrepieux @varundtsfi @joao-borges Hi, I have a question。

  1. If the minicap capture interval on the iPhone is 16.67ms, and the refresh rate of the iPhone is 90hz (frame interval 11.11ms), then does the minicap acquire all the pictures frame by frame, or will some frames be discarded?
  2. What is the fastest screenshot interval of minicap on iPhone?
varundtsfi commented 6 months ago

We can close this issue. we have pushed the required so files in the following PR

https://github.com/DeviceFarmer/minicap/pull/58