[Enhancement] [Android] In WebView on Android, respect <input type="file" capture="camera" />

[jgold6]

Summary

In Xamarin.Forms, a WebView on Android does not respect the capture="camera" attribute when an <input type="file" capture="camera" /> is clicked on. The default FormsWebChromeClient.OnShowFileChooser implementation presents the Intent provided by the OS to the OnShowFileChooser method, so the user can only select a file or picture from the photos app, there is no option to open the camera and take a picture.

This is expected behavior when using the intent provided by Android when an <input type="file" /> is selected. Android docs note that if you want to capture live media, then you do have to create your own intent or intent chooser. See android docs for CreateIntent on the FileChooserParams: https://developer.android.com/reference/android/webkit/WebChromeClient.FileChooserParams#createIntent()

Creates an intent that would start a file picker for file selection. The Intent supports choosing files from simple file sources available on the device. Some advanced sources (for example, live media capture) may not be supported and applications wishing to support these sources or more advanced file operations should build their own Intent."

UIWebVIew on iOS respects the capture attribute and opens the camera as expected.

API Changes

This suggestion would require having the FormsWebChromeClient.OnShowFileChooser method check for the existence of the capture element and its value. There are three possible values: camera, user, and environment, though camera value is deprecated. user should open the front camera, and environment should open the back camera. See: https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/capture

If the capture attribute is not present, then the Intent supplied via the FileChooserParams fileChooserParams parameter that is passed into the WebChromeClient.OnShowFileChooser method can be used, but if the capture attribute is present, then an Intent would need to be created to open the Camera activity and supply the file path to the picture taken to the filePathCallback.OnRecieve(Uri uri) method.

Intended Use Case

This would be used anytime a WebView on Android goes to a web page that has an <input type="file" capture="[environment | user | camera]" /> element. It would make the experience with a WebView on Android line up with the experience with a WebView on iOS

[ITCBB]

This would be quite relevant for me

[AndrzejRusztowicz]

+1. This is preventing our multiplatform Blazor MAUI app from capturing photos on Android. It opens the file picker instead of camera capture, just like on Windows but unlike normal HTML or Blazor on mobile devices. We use .net 7 but I just checked and it's still not fixed in .net 8 RC 2.

[gerneio]

@AndrzejRusztowicz I recently spent a lot of time digging into this myself for our cross-platform MAUI project, and Android def required the most dev work to get working correctly.

By default the BlazorWebView control uses the FilePicker API to get a file picker to pull up. So they tie into the OnShowFileChooser handler and just translate the request to something that works with that FilePicker API, which doesn't really have the concept of an Android camera/video intent. Sure it can filter by accept-type, but that's about it and I highly doubt they will ever add this into the FilePicker API spec, since it's kind of platform specific (iOS works out of the box). On the other hand, the default WebView has a slightly different approach which better mirrors how to handle the request within an Android native app, however again it falls short of the camera/video intent part.

Ultimately, I ended up completely crafting my own intent logic within a custom WebChomeClient handler. The tricky part was that when using the ActionImageCapture intent, it doesn't pass back the full size image back to OnActivityResult, but rather a thumbnail sized image, so you have to pass along some metadata ExtraOutput with a file path that both the camera app can write to and your app can read from. Turns out that MAUI automatically add a FileProvider which we can use for this, but it gets trimmed out in release (see notes in below code for how we deal with this). In comparison, the ActionVideoCapture intent will pass back the full video data, which is much easier to work with (not sure why it had to be so complicated with the image direction).

CustomMauiWebChromeClient.cs:

using Android.App;
using Android.Content;
using Android.Provider;
using Android.Webkit;
using Android.OS;
using Android.Content.PM;
using Uri = Android.Net.Uri;
using WebView = Android.Webkit.WebView;

namespace MyApp.MAUI.Platforms.Android;

// Microsoft.Maui.Storage.FileProvider is trimmed out in Release and not sure how to keep it in place on Android, so copied it and added our own custom one, which works in Debug & Release
// TODO: Figure out how to do this properly w/o hardcoding a custom file provider (see: https://stackoverflow.com/questions/75380344/)
[ContentProvider(new[] { "${applicationId}.custom.fileProvider" }, Name = "microsoft.maui.essentials.custom.fileProvider", Exported = false, GrantUriPermissions = true)]
[MetaData("android.support.FILE_PROVIDER_PATHS", Resource = "@xml/microsoft_maui_essentials_fileprovider_file_paths")]
internal class CustomFileProvider : AndroidX.Core.Content.FileProvider { }

public class CustomMauiWebChromeClient : WebChromeClient
{
    public override bool OnShowFileChooser(WebView webView, IValueCallback filePathCallback, FileChooserParams fileChooserParams)
    {
        if (filePathCallback is null)
            return base.OnShowFileChooser(webView, filePathCallback, fileChooserParams);

        OnShowFileChooserAsync(filePathCallback, fileChooserParams).FireAndForget();

        return true;
    }

    private async Task OnShowFileChooserAsync(IValueCallback filePathCallback, FileChooserParams fileChooserParams)
    {
        // If capture attribute is used, then don't show file chooser, but decide whether to take picture or video using accept-type below
        var initIntent = fileChooserParams.IsCaptureEnabled ? new Intent() : fileChooserParams.CreateIntent();

        var chooserIntent = Intent.CreateChooser(initIntent, "File Chooser");

        Uri photoContainerUri = default;

        // Add camera/video intents only IF we have permission to use the camera
        if (await MauiAppServices.CheckAndRequestCameraPermission())
        {
            var extraInitialIntents = new List<Intent>();

            var acceptTypes = ParseAcceptTypes(fileChooserParams.GetAcceptTypes());

            if (acceptTypes.HasFlag(FileContentType.Image))
            {
                var cameraIntent = new Intent(MediaStore.ActionImageCapture);

                // We setup a container for the captured image to be written to so we can get the full resolution image, otherwise it will be a low resolution thumbnail
                // https://developer.android.com/reference/android/provider/MediaStore#ACTION_IMAGE_CAPTURE
                photoContainerUri = SetupPhotoCaptureContainer();

                cameraIntent.PutExtra(MediaStore.ExtraOutput, photoContainerUri);

                extraInitialIntents.Add(cameraIntent);
            }

            if (acceptTypes.HasFlag(FileContentType.Video))
            {
                var videoIntent = new Intent(MediaStore.ActionVideoCapture);
                extraInitialIntents.Add(videoIntent);
            }

            if (extraInitialIntents.Count > 0)
                chooserIntent.PutExtra(Intent.ExtraInitialIntents, extraInitialIntents.ToArray());
        }

        var requestCode = default(int);

        requestCode = CustomActivityResultCallbackRegistry.RegisterActivityResultCallback((result, data) =>
        {
            CustomActivityResultCallbackRegistry.UnregisterActivityResultCallback(requestCode);

            OnActivityResult(filePathCallback, result, data, photoContainerUri);
        });

        Platform.CurrentActivity.StartActivityForResult(chooserIntent, requestCode);
    }

    private FileContentType ParseAcceptTypes(string[] acceptTypes)
    {
        FileContentType type = default;

        // When the accept attribute isn't provided GetAcceptTypes returns array with single element of empty string, indicating "*": [ "" ]
        if (acceptTypes switch { [""] => true, _ => false })
        {
            type ^= FileContentType.Any;
            return type;
        }

        // TODO: Do we need to identifiy specific extensions (i.e. jpg, png, etc)?
        if (acceptTypes.Any(e => e.StartsWith("image/", StringComparison.OrdinalIgnoreCase)))
            type ^= FileContentType.Image;

        if (acceptTypes.Any(e => e.StartsWith("video/", StringComparison.OrdinalIgnoreCase)))
            type ^= FileContentType.Video;

        // TODO: Check for doc?

        return type;
    }

    [Flags]
    private enum FileContentType
    {
        Any = Image | Video,
        Image = 1 << 1,
        Video = 1 << 2,
    }

    private Uri SetupPhotoCaptureContainer()
    {
        var imagePrefix = string.Format("img_{0:yyyyMMdd_HHmmss}", DateTime.Now);
        var imageFileName = imagePrefix + ".jpg";

        // When using IMAGE_CAPTURE and passing in uri as "file://", even though we can determine the file exists, we don't have permission to read it's contents later for some reason. Additionally, we are only able to
        // get the camera app to write to the public external picture directory, but won't work for any app directories. Using FileProvider solves both of these issues, but still using public external picture directory
        // so that the user can see and re-upload these images later if they decide to (local app directories would be hidden to them).
        var dir = global::Android.OS.Environment.GetExternalStoragePublicDirectory(global::Android.OS.Environment.DirectoryPictures);

        var imageFile = new Java.IO.File(dir, imageFileName);

        return FileProvider.GetUriForFile(global::Android.App.Application.Context, global::Android.App.Application.Context.PackageName + ".custom.fileProvider", imageFile);
    }

    private void OnActivityResult(IValueCallback filePathCallback, Result resultCode, Intent data, Uri mediaExtraOutputUri)
    {
        filePathCallback?.OnReceiveValue(ParseResult(resultCode, data, mediaExtraOutputUri));
    }

    private Uri[] ParseResult(Result resultCode, Intent data, Uri mediaExtraOutputUri)
    {
        // return FileChooserParams.ParseResult((int)resultCode, data);
        // `FileChooserParams.ParseResult` doesn't support returning multiple files, so we decide to parse the result manually below, which also adds support for `ACTION_IMAGE_CAPTURE` results

        var results = new List<Uri>();

        if (resultCode == Result.Ok)
        {
            if (data?.Extras is not null)
                foreach (var k in data.Extras.KeySet())
                {
                    var itm = data.Extras.Get(k);

                    /* Leaving for historical reasons, but when using `ACTION_IMAGE_CAPTURE` w/o `EXTRA_OUTPUT`, the image data is passed back in the extra data as a Bitmap, but it is a low resolution thumbnail, 
                     * and we'd much rather have the fullsize image
                     * Note: this code was not completed, just explorative
                    if (itm is Bitmap b)
                    {
                        var imageFileName = string.Format("img_{0:yyyyMMdd_HHmmss}_", DateTime.Now);
                        var storageDir = Environment.GetExternalStoragePublicDirectory(Environment.DirectoryPictures);

                        int size = b.RowBytes * b.Height;
                        var byteBuffer = ByteBuffer.Allocate(size);
                        b.CopyPixelsToBuffer(byteBuffer);

                        // 76800
                        var bts = new byte[byteBuffer.Capacity()];

                        byteBuffer.Rewind();
                        byteBuffer.Get(bts);

                        var fp = System.IO.Path.Combine(storageDir.AbsolutePath, imageFileName + $"1.png");
                        File.WriteAllBytes(fp, bts);

                        using (var stream = new MemoryStream())
                        {
                            b.Compress(Bitmap.CompressFormat.WebpLossless, 0, stream);

                            // 12847
                            var bytes = stream.ToArray();

                            fp = System.IO.Path.Combine(storageDir.AbsolutePath, imageFileName + $"2.webp");
                            File.WriteAllBytes(fp, bytes);
                        }
                    }
                    */
                }

            if (data?.Data is not null)
            {
                results.Add(data.Data);
            }
            else if (data?.ClipData?.ItemCount > 0)
            {
                for (var i = 0; i < data.ClipData.ItemCount; i++)
                {
                    var item = data.ClipData.GetItemAt(i);
                    results.Add(item.Uri);
                }
            }
            else if (mediaExtraOutputUri is not null && DoesUriContentExists(mediaExtraOutputUri))
            {
                results.Add(mediaExtraOutputUri);
            }
        }

        return results.ToArray();
    }

    private bool DoesUriContentExists(Uri contentUri)
    {
        if (contentUri is null) return false;

        // https://stackoverflow.com/a/16336319/10388359
        var cursor = Platform.CurrentActivity.ContentResolver.Query(contentUri, new[] { MediaStore.IMediaColumns.Size }, null, null, null);

        if (cursor != null)
            while (cursor.MoveToNext())
            {
                var colIndex = cursor.GetColumnIndex(MediaStore.IMediaColumns.Size);
                var size = cursor.GetInt(colIndex);

                if (size > 0)
                    return true;
            }

        return false;
    }
}

CustomActivityResultCallbackRegistry.cs:

using System.Collections.Concurrent;
using Android.App;
using Android.Content;

namespace MyApp.MAUI.Platforms.Android;

// Copied from: `Microsoft.Maui.Platform.ActivityResultCallbackRegistry` (https://github.com/dotnet/maui/blob/main/src/Core/src/Platform/Android/ActivityResultCallbackRegistry.cs)
public static class CustomActivityResultCallbackRegistry
{
    static readonly ConcurrentDictionary<int, Action<Result, Intent>> ActivityResultCallbacks =
        new ConcurrentDictionary<int, Action<Result, Intent>>();

    static int s_nextActivityResultCallbackKey = Random.Shared.Next();

    public static void InvokeCallback(int requestCode, Result resultCode, Intent data)
    {
        Action<Result, Intent> callback;

        if (ActivityResultCallbacks.TryGetValue(requestCode, out callback))
        {
            callback(resultCode, data);
        }
    }

    internal static int RegisterActivityResultCallback(Action<Result, Intent> callback)
    {
        int requestCode = s_nextActivityResultCallbackKey;

        while (!ActivityResultCallbacks.TryAdd(requestCode, callback))
        {
            s_nextActivityResultCallbackKey += 1;
            requestCode = s_nextActivityResultCallbackKey;
        }

        s_nextActivityResultCallbackKey += 1;

        return requestCode;
    }

    internal static void UnregisterActivityResultCallback(int requestCode)
    {
        Action<Result, Intent> callback;
        ActivityResultCallbacks.TryRemove(requestCode, out callback);
    }
}

MainActivity.cs:

public class MainActivity : MauiAppCompatActivity
{
    ...
    protected override void OnActivityResult(int requestCode, Result resultCode, Intent data)
    {
        base.OnActivityResult(requestCode, resultCode, data);

        CustomActivityResultCallbackRegistry.InvokeCallback(requestCode, resultCode, data);
    }
}

And finally make sure to call webView.SetWebChromeClient(new CustomMauiWebChromeClient()) for each android web view. I use this same logic for both my BlazorWebView and MAUI.Controls.WebView.

iOS comparatively, all I needed to do was add a couple permissions :) ....

Hope this helps!

[AndrzejRusztowicz]

Thanks @gerneio, In my MAUI app targeted for Android, I ended up using Microsoft.Maui.Media.IMediaPicker (from Microsoft.Maui.Essentials, part of MAUI framework), and I use a custom button to trigger it (instead of FileInput). I also kept FileInput next to it for gallery/file attachments that don't need camera.

        public async Task<byte[]> CapturePhotoAsync()
        {
            var photo = await MediaPicker.Default.CapturePhotoAsync(null);
            if (photo == null) return null;
            using Stream sourceStream = await photo.OpenReadAsync();
            using var memoryStream = new MemoryStream();
            await sourceStream.CopyToAsync(memoryStream);
            return memoryStream.ToArray();
        }

And I had to add these permissions to AndroidManifest.xml:

    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

    <queries>
            <intent>
                <action android:name="android.media.action.IMAGE_CAPTURE" />
            </intent>
        </queries>