Android: When we can expect new version of this ANE for Android ?

[jkpatel1463]

Hello @rainbowcreatures

Overall Great effort! still needs improvement , specially in Android.

We found that, software mode for capturing video in android is not usable at all. Produced video is too laggy. So When can we expect new version of android for the ANE ? Is there any workaround to improve the speed of video OR Creation process of Video ?

[rainbowcreatures]

Android software mode is not great for real-time video encoding, it is meant as a workaround for something like creative apps where users create animation which can be then encoded / recorded over a longer period of time. Real-time recording of webcam, will produce laggy results you mention.

The only solution is figuring out the GLES capture issue. I'll get back to this a bit more over the weekend again and see what more can be done without owning the device(s) you report issues on.

Thanks, Pavel

[jkpatel1463]

Hello @rainbowcreatures

Can you give me some of the device of android in which, Flashy-wrapper works fine? so that i can present a demo in those device. After completing the demo, I have more time to increase its stability.

[rainbowcreatures]

Hey @jkpatel1463 , well on my Lenovo K6 POWER and Lenovo K6 Note there didn't seem to be a problem when testing. It might also depend on the combination of device and app used though - I'm using exampleCamera app bundled with FW for the majority of my tests.

Which devices do you have available for testing?

As for making capture faster in software mode, what definitely helps is lower resolution and lower fps. That might counter what you are after though, which is better video quality. I'll recheck your GLES issue today.

[jkpatel1463]

Hello @rainbowcreatures

Okay, I will capture video in my device and share you device log with capture video.

Here are the list of devices, i have: Device: Micromax A116 OS: Android 4.2.1 Device Link: http://www.gsmarena.com/micromax_a116_canvas_hd-5256.php

Device: Samsung Galaxy S5 Os:Android 6.0.1 Device Link: http://www.gsmarena.com/samsung_galaxy_s5-6033.php

Device: Samsung Galaxy Note5 Duos Os: Android 6.0.1 Device Link: http://www.gsmarena.com/samsung_galaxy_note5_duos-7508.php

Device: Moto G (2nd Generation) Os:Android 6.0 Device Link: http://www.gsmarena.com/motorola_moto_g_dual_sim_(2nd_gen)-6648.php

Device: Moto Nexus 6 Os:Android 7.1.1 Device Link: http://www.gsmarena.com/motorola_nexus_6-6604.php

As for making capture faster in software mode, what definitely helps is lower resolution and lower fps. I am capturing whole screen in every device with 24FPS.

I'll recheck your GLES issue today. Thank you, waiting for your reply :)

[rainbowcreatures]

@jkpatel1463 I did email you regarding that*

[rainbowcreatures]

@jkpatel1463 Also note, that the minimum Android version supported for the video capturing is 4.3

So the Micromax device definitely won't work, the video API was severly lacking before Android 4.3.

[jkpatel1463]

Hello @rainbowcreatures

Okay, I will test it in all other device except Micromax & If i found anything then i will submit crashlog with device information.

I found a mail with ANE for Android. Thank you for that. I will integrate it now and message you back. I want to be sure about Android ANE: Should i use Software mode with this new ANE ?

[rainbowcreatures]

Try the GLES capture (hardware) again, the crash report that you sent was in hardware mode, wasn't it? I'm interested if I can see more error details from that, so I added some extra logging which might(hopefully) provide more clues for me.

[jkpatel1463]

Hello @rainbowcreatures

Device which are mentioned below, are used to test the app with new ANE. Device: Moto G (2nd Generation) Os:Android 6.0 Device: Moto Nexus 6 Os:Android 7.1.1

When i try to use the new ANE with hardward-mode/Software-mode, Packaging filed and throws a error which is written below:

Error occurred while packaging the application:

dx tool failed:
UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Landroid/support/graphics/drawable/AnimatedVectorDrawableCompat$1;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:122)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:161)
    at com.android.dx.command.dexer.Main.processClass(Main.java:737)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:678)
    at com.android.dx.command.dexer.Main.access$300(Main.java:83)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:607)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
    at com.android.dx.command.dexer.Main.processOne(Main.java:637)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:506)
    at com.android.dx.command.dexer.Main.runMultiDex(Main.java:335)
    at com.android.dx.command.dexer.Main.run(Main.java:245)
    at com.android.dx.command.dexer.Main.main(Main.java:215)
    at com.android.dx.command.Main.main(Main.java:106)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Landroid/support/v4/graphics/BitmapCompat;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:122)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:161)
    at com.android.dx.command.dexer.Main.processClass(Main.java:737)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:678)
    at com.android.dx.command.dexer.Main.access$300(Main.java:83)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:607)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
    at com.android.dx.command.dexer.Main.processOne(Main.java:637)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:506)
    at com.android.dx.command.dexer.Main.runMultiDex(Main.java:335)
    at com.android.dx.command.dexer.Main.run(Main.java:245)
    at com.android.dx.command.dexer.Main.main(Main.java:215)
    at com.android.dx.command.Main.main(Main.java:106)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Landroid/support/v4/widget/ViewDragHelper$Callback;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:122)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:161)
    at com.android.dx.command.dexer.Main.processClass(Main.java:737)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:678)
    at com.android.dx.command.dexer.Main.access$300(Main.java:83)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:607)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
    at com.android.dx.command.dexer.Main.processOne(Main.java:637)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:506)
    at com.android.dx.command.dexer.Main.runMultiDex(Main.java:335)
    at com.android.dx.command.dexer.Main.run(Main.java:245)
    at com.android.dx.command.dexer.Main.main(Main.java:215)
    at com.android.dx.command.Main.main(Main.java:106)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Landroid/support/v4/graphics/ColorUtils;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:122)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:161)
    at com.android.dx.command.dexer.Main.processClass(Main.java:737)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:678)
    at com.android.dx.command.dexer.Main.access$300(Main.java:83)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:607)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
    at com.android.dx.command.dexer.Main.processOne(Main.java:637)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:506)
    at com.android.dx.command.dexer.Main.runMultiDex(Main.java:335)
    at com.android.dx.command.dexer.Main.run(Main.java:245)
    at com.android.dx.command.dexer.Main.main(Main.java:215)
    at com.android.dx.command.Main.main(Main.java:106)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Landroid/support/v4/app/FragmentManagerImpl$OpGenerator;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:122)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:161)
    at com.android.dx.command.dexer.Main.processClass(Main.java:737)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:678)
    at com.android.dx.command.dexer.Main.access$300(Main.java:83)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:607)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
    at com.android.dx.command.dexer.Main.processOne(Main.java:637)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:506)
    at com.android.dx.command.dexer.Main.runMultiDex(Main.java:335)
    at com.android.dx.command.dexer.Main.run(Main.java:245)
    at com.android.dx.command.dexer.Main.main(Main.java:215)
    at com.android.dx.command.Main.main(Main.java:106)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Landroid/support/v4/media/MediaBrowserServiceCompat$ServiceBinderImpl$2;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:122)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:161)
    at com.android.dx.command.dexer.Main.processClass(Main.java:737)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:678)
    at com.android.dx.command.dexer.Main.access$300(Main.java:83)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:607)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
    at com.android.dx.command.dexer.Main.processOne(Main.java:637)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:506)
    at com.android.dx.command.dexer.Main.runMultiDex(Main.java:335)
    at com.android.dx.command.dexer.Main.run(Main.java:245)
    at com.android.dx.command.dexer.Main.main(Main.java:215)
    at com.android.dx.command.Main.main(Main.java:106)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Landroid/support/v4/BuildConfig;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:122)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:161)
    at com.android.dx.command.dexer.Main.processClass(Main.java:737)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:678)
    at com.android.dx.command.dexer.Main.access$300(Main.java:83)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:607)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
    at com.android.dx.command.dexer.Main.processOne(Main.java:637)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:506)
    at com.android.dx.command.dexer.Main.runMultiDex(Main.java:335)
    at com.android.dx.command.dexer.Main.run(Main.java:245)
    at com.android.dx.command.dexer.Main.main(Main.java:215)
    at com.android.dx.command.Main.main(Main.java:106)

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: already added: Landroid/support/graphics/drawable/TypedArrayUtils;
    at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:122)
    at com.android.dx.dex.file.DexFile.add(DexFile.java:161)
    at com.android.dx.command.dexer.Main.processClass(Main.java:737)
    at com.android.dx.command.dexer.Main.processFileBytes(Main.java:678)
    at com.android.dx.command.dexer.Main.access$300(Main.java:83)
    at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:607)
    at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)
    at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)
    at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)
    at com.android.dx.command.dexer.Main.processOne(Main.java:637)
    at com.android.dx.command.dexer.Main.processAllFiles(Main.java:506)
    at com.android.dx.command.dexer.Main.runMultiDex(Main.java:335)
    at com.android.dx.command.dexer.Main.run(Main.java:245)
    at com.android.dx.command.dexer.Main.main(Main.java:215)
    at com.android.dx.command.Main.main(Main.java:106)
8 errors; aborting

When i replaced the new ANE with existing ANE, then Packaging can be done successfully and application can be debugged. I will capture video of it and deliver you in mail.

[jkpatel1463]

Hello @rainbowcreatures

Send you screencast of the error by wetransfer. please correct me if i am wrong.

Thank you again.

[rainbowcreatures]

Hi @jkpatel1463 . I've got a possible path to solve the Android capture problems.

I've been looking at the Android API and there is a new API for video capture available. The only issue is, it is available only since Android 5.0+. Also, it will need capture video permission from the apps user.

However, all the heavy lifting of GLES / software or whatever caputring can be forgotten. This is provided by Android OS since Lollipop - it sends the screen content directly to the surface for me to record. Capturing the screen content via GLES has been the most problematic part of both iOS/Android, but especially on Android where the devices are so different.

From what I Googled around, I can't still say it will be 100% reliable, but it will probably be a significant improvement on reliability. I think for Android I'll need to create a new ANE around this, and simplify it vs what it is now. I think it is the only sane path forward... For android 4.3+ I'll still keep the previous method of GLES capture but it will be simplified as well (no multithreading, that one is also causing issues I think).

[jkpatel1463]

Hello @rainbowcreatures

Suggestion: Can't we use android lib who provide screen-capturing functionalities ?

For example: Here is one application named AZ Screen Recorder - No Root which provide functionalities of screen recording, so If we can integrate such functionalities of AZ Screen Recorder, then our task might be easy. does it makes sense ?

[rainbowcreatures]

Hey @jkpatel1463 , this is a separate application. I don't see how you could make this work other than suggesting people to download this app.

Like I said, I think using Android 5.0 SDK MediaProjection API is the best way forward and would make sense for me to invest time into(I could offer this as part of FW SDK).

The AZ Screen Recorder is for Lollipop. I can almost bet they are using the same API.

https://wwws.nightwatchcybersecurity.com/2016/04/13/research-securing-android-applications-from-screen-capture/

Quote: "Since Android 5.0, there exists a new MediaProjection API that allows apps to record videos and take screenshots of screens belonging to other apps."

[rainbowcreatures]

I'll close this down as its not an "issue" but more a conversation about where FW should be heading for Android.