Android: SAF & FileDescriptor

[PaulWoitaschek]

I need to use this library on android using an uri from SAF. Therefore it would be great if you implemented an api for using a file descriptor.

An example for how to implement this so the uir can be passed to the NDK can be found here: https://github.com/wseemann/FFmpegMediaMetadataRetriever/blob/b42986fb4044c54131379407598d8ac0ff1a3326/gradle/fmmr-library/core/src/main/java/wseemann/media/FFmpegMediaMetadataRetriever.java#L214

[HBiSoft]

@tanersener

I don't know what will that idiocy_fopen_fd function return for files on the cloud?

I assume you mean if a file was selected from Dropbox, Google Drive etc, I will base my answer on that -> When selecting a file from Dropbox, for example, the file gets downloaded first(by the Dropbox application) before returning the Uri. So, the file is not actually on the cloud.

It will only work if the Uri has a ContentProvider and all applications that share files to other applications (like Dropbox, Google Drive etc) have to have a ContentProvider.

[tanersener]

@HBiSoft I looked at idiocy_fopen_fd function again and I noticed that idiocy_fopen_fd function does not return a path. I don't think I can use that.

[pawaom]

some explanation about scoped storage https://proandroiddev.com/working-with-scoped-storage-8a7e7cafea3

it seems that getContentResolver().openFileDescriptor(contentUri, fileOpenMode) is not able to handle files in SDcard I have tested the same code works for files on the internal(phone's) storage but for SDcard there is an issue of permission denied

came across this https://stackoverflow.com/a/28805474/3126760

but how can we use it with FFMpeg

this shows the changes for Q

https://youtu.be/3EtBw5s9iRY?t=1006

I have not tested it on Android Q, has any one done it, Google has asked native Library developers to Contact for Bugs and reports @tanersener , @HBiSoft have you tried to reach out to them regarding the issues

https://youtu.be/3EtBw5s9iRY?t=1718

[pawaom]

I have tested on a Physical device Moto C android 7.0 (Api 24)and an Emulator(Pixel XL Api 29 ) with the same file on SDcard

The Strange thing was the same code worked on Emulator(Pixel XL Api 29 )

but not on Moto C android 7.0 (Api 24)

I guess it has to do with the changes in android Api 29

so my guess is to use cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.DATA) for Less than Api 29 and use "/proc/" + pid + "/fd/" + fd for Api > 29

I changed the code and it worked properly with

if (Build.VERSION.SDK_INT > Build.VERSION_CODES.Q) {
String mediaFile = "/proc/" + pid + "/fd/" + fd;
                rc = FFmpeg.execute("-y -i " + mediaFile + " -filter:v scale=1280:720 -c:a copy " + outputPath);
else{
 String Videopath =  null;
                String[] Projection = {
                        MediaStore.Video.Media._ID,
                        MediaStore.Video.Media.DATA};
                String selection = MediaStore.Video.Media._ID + " like ?";
                String[] selectionArgs = new String[]{
                        "" + videoId + ""};
                Cursor cursor = getContentResolver().query(MediaStore.Video.Media.EXTERNAL_CONTENT_URI,
                        Projection, selection, selectionArgs, null);
                if (cursor.moveToFirst()) {
                    do {
                       Videopath  = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.DATA));
                    } while (cursor.moveToNext());
                }
                if (cursor != null) {
                    cursor.close();
                }
rc = FFmpeg.execute("-y -i " + Videopath  + " -filter:v scale=1280:720 -c:a copy " + outputPath);
}
            }

Can any one confirm its working from them as well

[tanersener]

@pawaom What do you recommend asking google? I don't think asking anything about a non-documented API (/proc/self/fd/) is a good idea?

[HBiSoft]

@tanersener @pawaom I think we should put this issue on hold until the next release of Android, where presumably file paths will be returned specifically for situations like this. Even though using file paths are more memory/battery intensive, we (using FFmpeg) only call the file once.

I'm still not sure how this will work, so we will have to wait and see. (I will be asking a question on StackOverflow about this and let you know.)

For now, we can request legacy storage android:requestLegacyExternalStorage="true" till the next release.

---

I asked a question about this on StackOverflow. It is still too early to know how this will work. Apparently, Google will release their first beta release for Android R next month, so will we have to wait and see.

[pawaom]

@pawaom What do you recommend asking google? I don't think asking anything about a non-documented API (/proc/self/fd/) is a good idea?

the have mentioned, we can reach out to them even for specific usecase https://youtu.be/3EtBw5s9iRY?t=1730

the Things that we need to know What are the best practices to use this Library

1) should we use (/proc/self/fd/) for all cases 2) or should we use it only for Above Api 29 3) you can present your own case of not being able to run on older devices(Android 7 etc)(Why it doesnot work there but on API 29, do we need to make changes to the library) 4) can we use MediaStore.Video.Media.RELATIVE_PATH for Above Api 29 I have tested for that and the execution failed 5) the problems faced for pipe protocol what can be done about that, (any other issues that you might face)

hope you reach out to them and clear a few doubts we all developers have

[pawaom]

@tanersener @pawaom I think we should put this issue on hold until the next release of Android, where presumably file paths will be returned specifically for situations like this. Even though using file paths are more memory/battery intensive, we (using FFmpeg) only call the file once.

I'm still not sure how this will work, so we will have to wait and see. (I will be asking a question on StackOverflow about this and let you know.)

For now, we can request legacy storage android:requestLegacyExternalStorage="true" till the next release.

I asked a question about this on StackOverflow. It is still too early to know how this will work. Apparently, Google will release their first beta release for Android R next month, so will we have to wait and see.

I have tested for MediaStore.Video.Media.RELATIVE_PATH for Above Api 29 and the execution failed I dont know if they are going to provide file paths

this issue needs to be resolved soon

[HBiSoft]

@pawaom As I said in my previous comment, we will have to wait for Android R. We are not the only developers that have this issue (Have a look here).

I have tested for MediaStore.Video.Media.RELATIVE_PATH for Above Api 29 and the execution failed

I do not think RELATIVE_PATH was intended to retrieve the file path. Instead, it is used to save media to a specific location in the MediaStore, for example Movies/MyFolder. If you query a file from MediaStore that has the RELATIVE_PATH column, it will return the MediaStore directory, for example DCIM/SomeFolder and not the path as you would expect it.

I dont know if they are going to provide file paths

Currently, it is unclear and there is no point in speculating. Google knows about this issue and they will provide us with an alternative.

[pawaom]

@tanersener @pawaom

Apparently, Google will release their first beta release for Android R next month, so will we have to wait and see.

Finally android 11 is released https://techcrunch.com/2020/02/19/google-launches-the-first-developer-preview-of-android-11/

https://android-developers.googleblog.com/2020/02/Android-11-developer-preview.html

https://www.androidpolice.com/2020/02/19/scoped-storage-in-android-11-will-have-exemptions-for-older-apis-and-core-apps-like-file-managers/

some thing called opt-in raw file path access for media is allowed

https://developer.android.com/preview/privacy/storage#media-files-raw-paths

[alexcohn]

This probably involves the new MANAGE_EXTERNAL_STORAGE permission.

[HBiSoft]

@alexcohn I'm not sure, I think MANAGE_EXTERNAL_STORAGE is intented for app that requires broad access like file managing applications.

Some apps have a core use case that requires broad file access, such as file management or backup & restore operations.

The permissions to access it also looks ugly Allow access to manage all files where I only want the user to select one file.

Also from the docs:

Starting in Android 11, apps that have the READ_EXTERNAL_STORAGE permission can read a device's media files using direct file paths and native libraries. This new capability allows your app to work more smoothly with third-party media libraries.

But this doesn't help because how are we supposed to retrieve the path from MediaStore.. We can use the file path but be can't retrieve it (unless there is a way I don't know of, without using the _data column).

[pawaom]

Android has put up some additional documentation regarding Storage access in android 11

here are the links https://medium.com/androiddevelopers/modern-user-storage-on-android-e9469e8624f9

https://developer.android.com/preview/privacy/storage

and This is from commonsware

https://commonsware.com/blog/2020/03/22/r-raw-paths-all-files-access.html

@tanersener , @HBiSoft we can even file feedback and get specific help for use cases impacted by this feature

here

https://google.qualtrics.com/jfe/form/SV_9HOzzyeCIEw0ij3?Source=scoped-storage

[HBiSoft]

@pawaom I've opened an issue on issuetracker

It seems that we can continue using the data column to access the file path. I have tested this and I was able to pass a file to mobile-ffmpeg on Android R. This includes the SD Card.

The only issue I have now is that on Android Q, this doesn't work. I'm waiting for Google to reply. I feel they should give us the option to enable legacy storage programmatically. We will then be able to access files on Android Q as well.

I will let you know when they reply.

@tanersener I think you can close this issue as there is nothing you have to do on the library side. Thank you for your support and great library.

[pawaom]

@HBiSoft , according to this

https://medium.com/androiddevelopers/modern-user-storage-on-android-e9469e8624f9

they have written

However, we understand that many apps depend heavily on APIs that use file paths, including third-party libraries, that cannot easily switch to using file descriptors. So in Android 11, APIs and libraries that use file paths will be enabled again. Your app can use the requestLegacyExternalStorage manifest attribute to ensure compatibility for users running Android 10

also if you read commonware's blog, it seems we need to use old way for upto android p , then for android Q requestLegacyExternalStorage and again for android R use the new access Raw file path thing, all this is really confusing

I have also filed an issue with them from the link they provided

https://google.qualtrics.com/jfe/form/SV_9HOzzyeCIEw0ij3?Source=scoped-storage

If few others do the same they might consider it to be urgent

[HBiSoft]

@pawaom

Your app can use the requestLegacyExternalStorage manifest attribute to ensure compatibility for users running Android 10

If that's the case, then I'm all sorted. I've tested and everything seems to be working fine.

[tanersener]

Thanks everyone who contributed to sort this out 👍

[pawaom]

finally they have understood it

https://issuetracker.google.com/issues/151407044#comment10

Do keep requestLegacyExternalStorage in the manifest if you need legacy storage on Q devices. It will be ignored on R devices once you target R. We are improving the documentation on this.

Re DATA column: will publish more guidance on this soon

I am adding this comment just for others who might refer to it in the future

[pawaom]

this can be useful, @alexcohn , @HBiSoft @tanersener https://youtu.be/RjyYCUW-9tY?t=300

and

https://github.com/android/uamp/issues/325#issuecomment-627368296

[HBiSoft]

@pawaom I'm not sure why you are sharing this?

We've had access to files/paths from the MediaStore since API 1, the only issue (why this question was started in the first place) was that it was going to be removed(as it is in Android 10 without requesting legacy storage), but Google decided to not remove access in Android R (Instead they introduced FUSE, which is basically the same as it always was).

I clarified everything in my answer here as well - https://stackoverflow.com/a/60917774/5550161

[pawaom]

https://developer.android.com/preview/privacy/storage#media-direct-file-native

@HBiSoft , according to the new explanation , this is just an updated info for any one who might check it in the future, I shared tagging you and a few others, because the changes in Android 10 had caused confusion, I was unaware of your answer in Stackoverflow, sorry for the trouble

[alexcohn]

There are two little problems with the fopen() approach. For one, it does not work on Android 10 and there doesn't seem to be an intention to fix that for the ~20% of devices currently active. Second, this does not work smoothly for Google Drive and other document providers. I have addressed both issues in https://github.com/tanersener/mobile-ffmpeg/pull/440.

[pawaom]

There are two little problems with the fopen() approach. For one, it does not work on Android 10 and there doesn't seem to be an intention to fix that for the ~20% of devices currently active. Second, this does not work smoothly for Google Drive and other document providers. I have addressed both issues in #440.

they have mentioned

If you access media files using the File API or native libraries, it's recommended that you opt out of scoped storage by setting requestLegacyExternalStorage to true in your app's manifest file. That way, your app behaves as expected on devices that run Android 10.

[HBiSoft]

@alexcohn I had a quick look at your pull request, it looks very promising. Could you perhaps push it to a different branch so we can test it?

[alexcohn]

Which different branch could it be? Feel free to test directly from https://github.com/alexcohn/mobile-ffmpeg/tree/experiment/scoped-storage

[SoluLabLive]

@alexcohn @tanersener any plan to give updated Gradle version for these changes?

[xiaos]

But soon Android 11 will come, and we cannot opt out of scoped storage, how to solve this issue?

[Naresh95375]

any progress in support on Android 11?

[MenilV]

After a couple of days of struggle with this one, here's a workaround that supports Android 11 and will probably be good enough until this one gets released:

• since the FFmpeg.execute(...) requires a file path you can always create a temp File in your cache dir with val copy = File.createTempFile("file-name", ".extension", context.cacheDir)
• use the contentResolver to open the inputstream from the Uri and copy the input stream to the newly created file:

context.contentResolver.openInputStream(uri)?.apply{
    copy.outputStream().use { fileOut ->
            this.copyTo(fileOut)
        }
}

• now you can send the file path to FFmpeg copy.path and be Android 11 compatible.
• cleanup the temp file afterward copy.delete()

Tested on Samsung s8+ (Android 10 rooted) and Pixel 4a (Android 11).

Drawbacks of this approach might include memory issues if the file itself is larger.

[HBiSoft]

I'm not sure why people keep asking for Android 11 support. You can pass an Uri thanks to this pull request - https://github.com/tanersener/mobile-ffmpeg/pull/440

There's no need to copy the file first, just pass the Uri to the library.

There's only one issue left, I could find on Android 11 - https://github.com/tanersener/mobile-ffmpeg/issues/634 (it looks like this will be fixed soon)

[aGreatDay]

@HBiSoft

There's no need to copy the file first, just pass the Uri to the library.

Is this available in release 4.4?

First, I create a new MediaStore entry with contentResolver.insert which returns a Uri, and pass the uri.toString() as the outputfile for FFmpeg.execute(...). This results in:

mobile-ffmpeg: [NULL @ 0xf465a000] Unable to find a suitable output format for 'content://media/external_primary/video/media/2908'
mobile-ffmpeg: content://media/external_primary/video/media/2908: Invalid argument

[RowlandOti]

Also just to chip into those who will make the mistake of trying to copy the file first - it is going to very slow for some devices and users on the other end may notice lagging especially if the file is large. If reading the file is a repetitive and on-demand thing, then that approach is highly inefficient.

On the other hand, reading is as a Uri resource directly is pretty smooth. I build an app around other android users consuming media content hosted on android media hubs and the experience was very smooth.

[blue-peacock]

@aGreatDay

I'm facing the same error using release 4.4. Has anybody a solution?

This results in:

mobile-ffmpeg: [NULL @ 0xf465a000] Unable to find a suitable output format for 'content://media/external_primary/video/media/2908'
mobile-ffmpeg: content://media/external_primary/video/media/2908: Invalid argument

[alexcohn]

@aGreatDay, @blue-peacock you can not simply use a content Uri on "command line". You must wrap it as shown in the test app example SafTabFragment.encodeVideo():

String videoPath = Config.getSafParameterForWrite(requireContext(), outUri);

[blue-peacock]

@alexcohn thank you, but when I pass the Uri content://media/external/images/media/2854 to Config.getSafParameterForWrite() I'm getting saf:173/image1617797380694.jpg as output for videoPath. The ffmpeg command then fails with the error:

[image2 @ 0x71493a6600] Could not open file : saf:173/image1617797380694.jpg av_interleaved_write_frame(): I/O error

The content: Uri is created by the following code:

 val filePrefix = "image"
        val fileExtn = ".jpg"
        val contentValues = ContentValues()
        contentValues.put(MediaStore.Images.Media.RELATIVE_PATH, "Pictures/" + "Test")
        contentValues.put(MediaStore.Images.Media.TITLE, filePrefix + System.currentTimeMillis())
        contentValues.put(MediaStore.Images.Media.DISPLAY_NAME, filePrefix + System.currentTimeMillis() + fileExtn)
        contentValues.put(MediaStore.Images.Media.MIME_TYPE, "image/jpeg")
        val uri = context.contentResolver.insert(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, contentValues)