search

nova-video-player / aos-AVP

NOVA opeN sOurce Video plAyer: main repository to build them all
Apache License 2.0
3.38k stars 201 forks source link

Android R migration #336

Closed courville closed 2 years ago

courville commented 4 years ago

In order to target SDK 30 you need:

diff --git a/build.gradle b/build.gradle
index ea5d6de..bd9c633 100644
--- a/build.gradle
+++ b/build.gradle
@@ -115,7 +115,7 @@ android {
         sourceCompatibility JavaVersion.VERSION_1_8
         targetCompatibility JavaVersion.VERSION_1_8
     }
-    compileSdkVersion 29
+    compileSdkVersion 'android-R'
     buildToolsVersion '29.0.3'
     sourceSets {
         main {
@@ -130,7 +130,7 @@ android {
     defaultConfig {
         generatedDensities = []
         minSdkVersion 21
-        targetSdkVersion 29
+        targetSdkVersion 'R'
         multiDexEnabled true
         versionCode getDate()
         version = '4.48.1'

When launching the app you get this obvious introspection not allowed anymore to solve:

2020-04-26 13:56:05.111 7491-7550/org.courville.nova W/.courville.nov: Accessing hidden method Landroid/os/storage/StorageVolume;->getPath()Ljava/lang/String; (greylist-max-q,test-api, reflection, denied)
2020-04-26 13:56:05.111 7491-7550/org.courville.nova W/System.err: java.lang.NoSuchMethodException: android.os.storage.StorageVolume.getPath []
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at java.lang.Class.getMethod(Class.java:2072)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at java.lang.Class.getMethod(Class.java:1693)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at com.archos.filecorelibrary.ExtStorageManager.updateAllVolumes(ExtStorageManager.java:127)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at com.archos.filecorelibrary.ExtStorageManager.getExtStorageManager(ExtStorageManager.java:51)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at com.archos.mediaprovider.video.Blacklist.isBlacklisted(Blacklist.java:84)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at com.archos.mediaprovider.video.VideoStoreImportImpl$Job.<init>(VideoStoreImportImpl.java:246)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at com.archos.mediaprovider.video.VideoStoreImportImpl.handleScanCursor(VideoStoreImportImpl.java:171)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at com.archos.mediaprovider.video.VideoStoreImportImpl.doScan(VideoStoreImportImpl.java:331)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at com.archos.mediaprovider.video.VideoStoreImportImpl.doFullImport(VideoStoreImportImpl.java:105)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at com.archos.mediaprovider.video.VideoStoreImportService.doImport(VideoStoreImportService.java:417)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at com.archos.mediaprovider.video.VideoStoreImportService.handleMessage(VideoStoreImportService.java:364)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:102)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at android.os.Looper.loop(Looper.java:216)
2020-04-26 13:56:05.112 7491-7550/org.courville.nova W/System.err:     at android.os.HandlerThread.run(HandlerThread.java:67)
courville commented 4 years ago

androidR branch created for Video/MediaLib/FileCoreLibrary

courville commented 3 years ago

getPath is greylisted reflection denied on API30: See https://github.com/nova-video-player/aos-FileCoreLibrary/commit/e81467beab3e6c47090448674870d8b7bbeb1c46 See https://github.com/nova-video-player/aos-FileCoreLibrary/commit/04839cae17ed8547006c74eec99e660e11cd180e

courville commented 3 years ago

Pending google play to be ready to manage properly android.permission.MANAGE_EXTERNAL_STORAGE and request exemption.

See https://github.com/nova-video-player/aos-Video/commit/fa42e916de4ffca7758d68e83d8b0f5915d3a1b3

courville commented 3 years ago

Check https://support.google.com/googleplay/android-developer/answer/9956427 to request exemption.

courville commented 3 years ago

Exemption requested and denied. For the record here are the elements sent. App functionality requiring the permission (limit 500 chars):

Nova is a video player/manager that catalogs (though MediaStore.Files import)  and presents all videos as a structured medialibrary (Movie/TVshow/Anime) after media information scraping (themoviedb/thetvdb cloud services).
It needs to:
- access all videos already present on the device wherever located (internal, SDcard, USB)
- act as a file manager (import/delete/download) for all media files even those not recognized as such by Android: e.g. NFO metadata, SUB/SSA subtitles, XML resume points

Why can't your app follow good confidentiality practices (i.e. Storage Access Framework or Media Store API or Scoped Storage) (limit 500 chars):

Without loss of functionality, scoped storage cannot be used since Nova needs to index all preexisting videos & related files at any location (internal/SDcard/USB)
MediaStoreAPI does not allow managing (read/download/delete) all media files processed today by Nova: e.g. NFO metadata, SUB/SSA subtitle, XML resume points
Storage Access Framework is not compatible with Nova full device media persistent library design capturing overall device content
Nova is open-source and can be audited on github

Less than 30s video explaining the claim: https://home.courville.org/nova_video_player-faq/nova-MANAGE_EXTERNAL_STORAGE-permission.mp4

Generic reject email received from Google without any specific information.

Overall this is a very frustrating experience.

courville commented 3 years ago

Note that Google failed to implement on Android TV (at least API30 emulator) the required dialog to manage properly android.permission.MANAGE_EXTERNAL_STORAGE:

2021-07-10 15:25:25.798 4151-4151/org.courville.nova E/AndroidRuntime: FATAL EXCEPTION: main
    Process: org.courville.nova, PID: 4151
    android.content.ActivityNotFoundException: No Activity found to handle Intent { act=android.settings.MANAGE_ALL_FILES_ACCESS_PERMISSION (has extras) }
        at android.app.Instrumentation.checkStartActivityResult(Instrumentation.java:2067)
        at android.app.Instrumentation.execStartActivity(Instrumentation.java:1727)
        at android.app.Activity.startActivityForResult(Activity.java:5320)
        at androidx.activity.ComponentActivity.startActivityForResult(ComponentActivity.java:574)
        at android.app.Activity.startActivityForResult(Activity.java:5278)
        at androidx.activity.ComponentActivity.startActivityForResult(ComponentActivity.java:560)
        at android.app.Activity.startActivity(Activity.java:5664)
        at android.app.Activity.startActivity(Activity.java:5617)
        at com.archos.mediacenter.video.browser.PermissionChecker$4.onClick(PermissionChecker.java:161)
        at androidx.appcompat.app.AlertController$ButtonHandler.handleMessage(AlertController.java:167)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:223)
        at android.app.ActivityThread.main(ActivityThread.java:7656)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)

This is a mess.

courville commented 3 years ago

For SSA/ASS cf. https://android-review.googlesource.com/c/platform/packages/providers/MediaProvider/+/1764553

courville commented 3 years ago

Found old issue on issuetracker somehow related https://issuetracker.google.com/issues/136774034 And precious command:

adb shell content query --uri "content://media/external/file" --projection name:_data.
courville commented 3 years ago

API30debacle plan of record:

courville commented 3 years ago

In terms of deadline 2021/11/1 for nova cf. https://support.google.com/googleplay/android-developer/answer/9859152#zippy=%2Ctarget-api-level-requirements-for-play-console

API level requirement Starting date
Android 8.0 (API level 26) August 1, 2018: Required for new appsNovember 1, 2018: Required for app updates
Android 9 (API level 28) August 1, 2019: Required for new appsNovember 1, 2019: Required for app updates
Android 10 (API level 29)* August 3, 2020: Required for new appsNovember 2, 2020: Required for app updates
Android 11 (API level 30)* August 2, 2021: Required for new appsNovember 1, 2021: Required for app updates
moneytoo commented 2 years ago

As mentioned here, it is still possible to use the classic File API, though it is limited to media files only (just like the MediaStore API). The greylisted methods could be possibly replaced by MediaStore.getExternalVolumeNames() (?).

courville commented 2 years ago

Reading https://stackoverflow.com/questions/56468539/getexternalstoragepublicdirectory-deprecated-in-android-q leads to the conclusion that nova needs to use MediaStore API.

Code sample to implement create/delete/etc... classes compatible with Android Q/R (FileUtilsQ.java) are available here https://gist.github.com/fiftyonemoon/433b563f652039e32c07d1d629f913fb and mentioned here https://stackoverflow.com/questions/64582269/how-can-i-delete-file-on-android-11-api-30-without-system-confirmation-dialog

This requires however to have proper content:// uri and https://stackoverflow.com/questions/7305504/convert-file-uri-to-content-uri provides some more or less working methods.

courville commented 2 years ago

Experimenting a bit:

Some useful query methods via adb:

To get the content uri, this asynchronous way works:

MediaScannerConnection.scanFile(mContext, new String[]{ uri.getPath() },
                null, // mimetypes
                new MediaScannerConnection.OnScanCompletedListener() {
                    @Override
                    public void onScanCompleted(String path, Uri uri) {
                        contentUri = uri;
                        log.debug("scanFile: contentUri " + contentUri + " for path " + path);
                    }
                });

Note that the hack in ExternalSDFileWriter make Android believe that the file is an image via insertion in the MediaStore.Images.Media does not work anymore and will need to be removed from the code.

courville commented 2 years ago

TODO:

courville commented 2 years ago

NFO files (text/plain mime type only https://cs.android.com/android/platform/superproject/+/master:packages/providers/MediaProvider/src/com/android/providers/media/util/MimeUtils.java?q=MimeUtils.java) can only be created in Documents/Download folders. It does not make sense to prefix them in app private directory getFilesDir() since nobody would access them. See https://developer.android.com/training/data-storage Alternative is to use getExternalFilesDir(null) /storage/emulated/0/Android/data/org.courville.nova/files visible from users and where nova can write. Links to refresh crash courses on directories: https://gist.github.com/granoeste/5574148 https://medium.com/microsoft-mobile-engineering/scoped-storage-in-android-10-android-11-28d58d989f3c

courville commented 2 years ago

Some TODO (for me):

courville commented 2 years ago

API31 targeted for v6.0.30 onward.

moneytoo commented 2 years ago

VLC 3.4.3 beta 6 available from the public testing channel on Google Play uses the MANAGE_EXTERNAL_STORAGE permission.

courville commented 2 years ago

@moneytoo thanks for sharing this news. It hurts since I have been denied this privilege by Google. I am glad for vlc to have managed to lobby for it. At least nova complies with the security requirements of Android... /me cries into a pillow

courville commented 2 years ago

/me having fun looking at https://www.reddit.com/r/androiddev/comments/rmgdis/how_come_vlc_is_allowed_to_use_manage_external/ Thanks @moneytoo

courville commented 2 years ago

@moneytoo, I just made the connection with just video player (I am slow sorry). FYI nova relies on the old Archos Video Player AVOS C based multimedia engine that used to be the software basis for all Archos multimedia PMPs.

courville commented 2 years ago

From the VLC folks and for track record https://issuetracker.google.com/issues/143549606

courville commented 2 years ago

OK just noticed on Google Play console that I have been awarded the android.permission.MANAGE_EXTERNAL_STORAGE sensitive permission (after it has been denied). It appeared out of the blue somehow.

@moneytoo you should check too (finger crossed).

EDIT: I also checked that I can deploy an internal release with the perm.

moneytoo commented 2 years ago

Knowing Google, something like that usually sounds too good to be true. But if it sticks, it's great. Could it be because you left a version of an app with such permission in some release channel untouched and they eventually approved it? At least for me, I only see my declaration for this permission in App content section and nothing more.

courville commented 2 years ago

OK this was indeed too good to be true. After granted deployment it has been cancelled. FTR this is what was indicated on the google play console:

Capture d’écran 2022-09-22 à 19 01 18
moneytoo commented 1 year ago

BTW, it's not only VLC possessing such permission, but a "few more" other players at least: 

MX Player (com.mxtech.videoplayer.ad)
MX Player Pro (com.mxtech.videoplayer.pro)
VLC (org.videolan.vlc)
OPlayer Lite (com.olimsoft.android.oplayer)
OPlayer (com.olimsoft.android.oplayer.pro)
KMPlayer (com.kmplayer)
XPlayer (video.player.videoplayer)
PLAYit (com.playit.videoplayer)
HD Video Player by iJoysoft (free.online.hd.video.player)
Video Player by Leopard V7 (com.mine.videoplayer)
courville commented 1 year ago

@moneytoo indeed list is expanding... Not quite sure what is the magic for requesting permission since it has been denied twice. I hate to resort to this, but I might ping some google folks to understand better the process. Please let me know if you find more.

moneytoo commented 1 year ago

My opinion is that the magic is either support of some not normally indexed file format (.iso, .wmv, .rm, possibly some other subtitle or audio formats) or some advanced file functionality (local p2p file sharing, folder encryption or lockings etc.).

moneytoo commented 1 year ago

https://www.reddit.com/r/androiddev/comments/rmgdis/comment/jnddkp8/?context=3

courville commented 1 year ago

@moneytoo thanks for sharing, have you requested on play console the right?

courville commented 1 year ago

FTR after downloading a couple (not being exhaustive there) of video players on apkpure (I know...), a simple for i in *apk; do aapt d permissions "$i" | grep '(package|MANAGE)'; done provides:

package: com.kmplayer
uses-permission: name='android.permission.MANAGE_EXTERNAL_STORAGE'
package: org.xbmc.kodi
uses-permission: name='android.permission.MANAGE_EXTERNAL_STORAGE'
package: com.mxtech.videoplayer.ad
uses-permission: name='android.permission.MANAGE_EXTERNAL_STORAGE'
package: video.player.videoplayer
uses-permission: name='android.permission.MANAGE_EXTERNAL_STORAGE'
package: com.kmp.video
uses-permission: name='android.permission.MANAGE_EXTERNAL_STORAGE'
package: com.mine.videoplayer
uses-permission: name='android.permission.MANAGE_EXTERNAL_STORAGE'
package: org.videolan.vlc
uses-permission: name='android.permission.MANAGE_EXTERNAL_STORAGE'
courville commented 1 year ago

OK after discussing with Google Developper Support, I was not able to get someone that could help on MANAGE_EXTERNAL_STORAGE permission. However, declaration on Play Console "application content section", the "access to all files" is marked treated and seems enabled. I gave another shot at publishing the nova on internal track and it has not been removed by Google for a week. I reworked the code to grant the permission to be "universally compatible" with Android ecosystem. What I learned is:

What is "fun" is this special case for Android TV.

No wonder that boxes supporting USB storage does not want to migrate to Android 12. Staying on Android 11 waiting for Android 14 is the only way to get apps such as file managers working without fiddling with Android framework introducing some "customizations" forking from a vanilla base.

courville commented 11 months ago

Couple of references for the record on Android not detecting correctly media files: