Cannot use CapturePhotoAsync() if camera causes app to background and restore?

[JRE-6783]

I understand that during the lifecycle of the app, an activity may be put into the background, and essentially be 'put to sleep'. I have code during a normal activity restore, that will recover the data etc.

When I use a method like:

private async void TakePhoto()
    {
        // Photo requested

        var photo = await MediaPicker.CapturePhotoAsync();

       // Photo taken

    }

If when calling TakePhoto(), the Android device does not fully background the app, the method successfully hits "Photo taken". If the app is fully backgrounded (as soon as the camera app loads in front it decides to free up resources), once the photo is taken, the activity restores itself, but the original method is no longer executing, so I am unable to continue to get the photo response, thus "Photo taken" is never hit.

So my question is, how am I implementing this wrong, given how the Android activity lifecycle works?

[JRE-6783]

I appreciate this might not be a bug, but does someone at least know whether I am using it right, and or at least making sense? Is this a limitation that means the lib can only be used if your app does not ever succumb to activity lifecycle recycling once the camera launches, which is meant to be an expected and real life architecture and scenario, that should be accounted for...

[fl-eric]

I'm also facing this issue, but the activity does not get restored. I didn't notice a consistent behavior on the activities lifecycle, but sometimes when the user gets back to the app the Main Page is shown, and sometimes the Splash Screen is shown before loading the Main Page. The log shows that the activities gets their "OnSaveInstanceState" called.

Should Xamarin.Forms be able to automatically restore the app's state? If yes, then the issue is in Xamarin.Forms. Otherwise MediaPicker must provide an alternate interface so it is possible to resume on the application level.

(I had this issue specially on Samsung Galaxy Tab A with low specs).

[JRE-6783]

@fl-eric I am using Xamarin.Android, not forms, so can't comment too much on your additional issues. Even when you resolve the activity lifecycle issue restoring properly, that wouldn't sort the scenario I am facing. If OnResume() and OnCreate() fire properly (as mine does - as my activity process is fully killed (as per Android design https://developer.android.com/guide/components/activities/activity-lifecycle) the code will never go back to your method where you called "var photo = await MediaPicker.CapturePhotoAsync();", so it is game over! Am i missing something? I feel like I am gong mad!

[fl-eric]

Well, if the activity is fully destroyed, then I presume the async call gets disposed too. It seems MediaPicker does not account for this scenario when the activity gets recreated, and the "picking" needs to be resumed.

[JRE-6783]

Except that "picking need to be resumed" is a vicious circle. I relaunch the camera, and the activity is killed again...and again! well about 80% of the time, and then I get lucky. So you can only use this method, if you can guarantee every device and user will never have to worry about Android lifecycles doing their thing in the background, which is impossible??? How is that meant to be a viable option for anyone?

[fl-eric]

I meant this bug could be changed to a feature request, so the MediaPicker provides a way to tell the file path to save the image, or a way to read the file path after returning to the app (even if the activity and the async call get destroyed). If one of these gets implemented, we would be able to resume the operation.

[ConqueringLionSoftware]

I was facing a similar issue in the Android version a Xamarin.Forms app. I solved it by removing the property NoHistory=true inside the [Activity] attribute of the main activity.

[pspeybro]

This is also an issue in the old library (MediaPlugin): https://github.com/jamesmontemagno/MediaPlugin/issues/286

I already tried many of the suggested workarounds, but no luck so far. @jamesmontemagno , is there a way to get to the image captured after the activity is restored? Otherwise we end up with an endless cycle like JRE-6783 mentioned above.

[nbevans]

This is such a fundamental thing which Xamarin Forms totally doesn't handle well, or at all.

Android relies very much on this idea of serializing all your application state into a "bundle" when the app is being closed/suspended by the Android OS, and then deserialized to restore the app to its prior state when it is re-hydrated by the OS. Xamarin Forms doesn't expose any sensical API to allow your app to make use of this mechanism. And - even if it did - only the very simplest of apps can be dumbed-down to make use of that mechanism.

As far as I can see there is simply no real workaround here.

With all that said, I think the Android OS design is really stupid. If an app spawns another app for the purpose of getting an "input" e.g. file chooser, photo capture - it is really silly that the OS thinks it is okay to mess with that transaction which is actively occurring and may have all sorts of intricate state in flux which cannot be easily serialized.

[boris-df]

i'm very frustrated currently: (Android 13, Pixel 4; using Xamarin Forms 5)

i can call await MediaPicker.CapturePhotoAsync() i get a result after i took a picture the Xamarin.Forms 5 App is running after that (i create objects, views, i initialize the view and i do a navigation.PushModal(view) this is all running (you cannot see it because there is still the black camera-app-screen visible

And then, after the app is running just fine, the MainActivity is running into the "OnActivityResult" with a Result "Canceled"

What the heck?! Why? The result is: The App is not capable of taking pictures - because it is cancelled and restarted as soon as the camera-screen/view/intent/whateverItIs finishes :(

Is there ANY workaround for that? I need to offer a camera with Zoom/Flashlight Features and metadata in the images in high resolution... so build my own camera-view with capturing from the live-view (like sooo many examples out there) is no solution (because no zoom, just HD resolution etc.)

[boris-df]

BTW: It works just fine as long i was able to use:

• Xamarin.Forms 4.6
• old Android Nuget Packages like v4 / v7 (no AndroidX)
• Building an APK instead of a Bundle
• targeting Android 9

But now for the new app i had to target Android 12 to be able to create and upload a bundle to the playstore ... and for that i need AndroidX Packages and so on ...

[konradzuse]

Users of our app are also complaining of this issue. It makes the app unusable since it's pot-luck if the app will be closed while the photo is being taken.

[JRE-6783]

It's a joke. Well the whole of Xamarin will be unsupported in about 12 months time, so do you think they care about supporting this useless library? I think they will find the same issue with their 'baked-in' version for MAUI, when Android does it's thing (good point btw @nbevans ), as they have implemented using it the same way from our perspective. The difference is, it's not some James Montemagno bungled project, and the whole of MS will be responsible for making it work.

[nbevans]

Probably the best thing to do is nag them over in MAUI Issues to provide a fix then we can back-port it to legacy Essentials if we haven't all migrated to MAUI in the mean time.

[nbevans]

With that said - I don't think a fix is technically feasible. The MAUI architecture is not much different from XamForms - and on this particular issue of serializing application state into the Bundle it's going to face the same issues.

[nbevans]

I think the only possible avenue to fix this is maybe some "hidden" feature or flag(s) on the Activity Intent in Android that needs to be set to prevent it killing the parent activity.

[konradzuse]

I think a solution lies in registering a callback somewhere in a constructor, so when the parent Activity receives a call to OnActivityResult() it can call your registered callback to handle the photo.

The problem today is they are using an in-memory Task, and then completing it with the result when OnActivityResult is called. But in the scenario we're facing the Task no longer exists.

[konradzuse]

Flutter apps exhibit the same problem, and their workaround....

When under high memory pressure the Android system may kill the MainActivity of the application using the image_picker. On Android the image_picker makes use of the default Intent.ACTION_GET_CONTENT or MediaStore.ACTION_IMAGE_CAPTURE intents. This means that while the intent is executing the source application is moved to the background and becomes eligable for cleanup when the system is low on memory. When the intent finishes executing, Android will restart the application. Since the data is never returned to the original call use the ImagePicker.retrieveLostData() method to retrieve the lost data.

I see Xamarin essentials use an intermediate activity to launch the intent, so they just need to persist the uri to the image in the saveBundleState and restore it when the activity is created. Then expose a similar retrieveLostData() method that you can call when the app is restored.

[pauldiston]

I am facing the issue with the old library as detailed here :-

his is also an issue in the old library (MediaPlugin): jamesmontemagno/MediaPlugin#286

I am looking at switching to use Xamarin.Essentials however after reading this thread I am concerned that the issue will still be present.

Can anyone provide some additional guidance as to how to handle this situation? @konradzuse You mention a saveBundleState and retrieveLostData method approach, have you any more information as to how to go about implementing this approach?

[konradzuse]

Can anyone provide some additional guidance as to how to handle this situation? @konradzuse You mention a saveBundleState and retrieveLostData method approach, have you any more information as to how to go about implementing this approach?

I never resolved this issue. I think you'll have to take the code from Xamarin essentials, and implement your own solution using it.

In the Activity that launches the camera intent, you have the file path where the image will be stored, so you must keep track of this file path, and provide a way for your app to recover it after the process gets restored.

[nbevans]

The problem is that restoring a XamForms app (and I guess MAUI too) to its previous state upon load is incredibly hard to do.

[pauldiston]

@konradzuse Thanks for the reply. @nbevans Yes that is very true, especially if you are going through a number of screens to then take a photo and then you are "kicked" back to the Login page of your application.

This (https://learn.microsoft.com/en-us/xamarin/android/app-fundamentals/activity-lifecycle/saving-state) has some details regarding the overriding of OnSaveInstanceState however as I mention above this is going to be tricky as do I then perform some programmatic navigation back to the Page I left so that the user can resume their workflow or do I do something else?

[pauldiston]

@konradzuse @nbevans What are your thoughts on Xamarin.Essential's usage of IntermediateActivity in the PlatformCaptureAsync method? Does that provide any workarounds to the restoring state/restoring navigation issue?

[konradzuse]

@konradzuse @nbevans What are your thoughts on Xamarin.Essential's usage of IntermediateActivity in the PlatformCaptureAsync method? Does that provide any workarounds to the restoring state/restoring navigation issue?

I think you can use this activity to handle when the app resumes, you can detect that the camera was started. I think OnActivityResult will still be called, but of cause the caller will no longer exist. Your own app code will have to detect somehow it has been restored, and ask the media plugin for the lost image path. You can see that's how they handle it with Flutter.

The whole app lifecycle handling in Xamarin forms is pretty lacking really. I've had a real hard time restoring the navigation state on app resume.

[boris-df]

This is all really frustrating (it starts with the fact that you have to code a lot of code to show / use a camera in the first place - yes, essentials wraps this for you but has problems in itself)

I have no chance to save the state (navigation, objects that are created thru the navigation and all the things around using the app) - at least not in a usefull way or time. So if the OS means to kick our app from memory it means that users are not able to take pictures in our app ... this way. (really a bummer in OS Design as you cannot tell the OS that you are the caller and need to stay in memory to receive the picture - but anyway, we cannot change that)

WHAT NOW? WORKAROUNDS?

Now i'm trying to build my own Camera-Page - showing the live-stream of the camera and capture the picture. There are some examples out there and there is the XamarinCommunity Toolkit which has a View for that. see: https://learn.microsoft.com/en-us/xamarin/community-toolkit/views/cameraview

Downside of that:

• 1000+ lines of code needed to make it good
• The CommunityToolkit has it for you - but... the Android side doesn't rotate the taken picture correctly and it's worse for captures videos which could end up upside down
• So using the Nuget is propable not an option - but using the code to - again - fix it myself (if i can find solutions for that)
• pictures are not high resolution but a snapshot of the video frame (so 1920x1080 max)
• switching cameras is not implemented and would be extra code (if you have a device with more than 1x front and back cam)
• so zooming is digital only
• no exif data, gps-position, ...
• ...

But at least it's in our own app so OS will not kick you out of the memory and users are able to take a picture

... when i read in the issue-tracker of the communityTookit where the Orientation/Rotation Problem on Android are discussed for a long time and still 2.0.5 has this problem - which is so obvious to see - i wonder, if anyone is using Camera with Xamarin at all in the end...

frustrating ... really really frustrating

[pauldiston]

@boris-df Thank you very much for your reply. This is very interesting. I would be very interested in your proposed workaround of rolling your own camera view as my use case for the camera is quite simple (just take a photo) so the additional short comings in functionality might not be a problem for me however I do understand there is a real issue for others who do require this additional functionality. I do have some experience in reading data from the live camera view in another app so this might come in handy. I am currently in the testing phase to see whether I can do anything with the OnSaveInstanceState and OnCreate where the Bundle is not NULL.

[nbevans]

I still think it's an Android design flaw. If App A delegates a task (e.g. take a photo) to App B, that transaction should mean that App A (and App B) excluded from termination.

Imagine if Windows terminated your AppA.exe because it dared to Shell Execute an AppB.exe and read from its StandardOutput?

[boris-df]

@nbevans I absolutely agree - but i don't have the Android Source-Code at hand i cannot fix that and deploy my own Android-OS to our clients :D so my only chance is to not call the Camera App but instead show my own camera page ... with all limitations and hard work to do :/

[konradzuse]

You may be able to solve the camera issue, but what happens if the user receives a phone call while using your app? The app will potentially be killed and restarted.

The app i'm working on using FreshMVVM to handle the navigation stack. I've rolled our own state persistence mechanism, by serializing the navigation stack as json, then restoring it on resume. It's not great though, since the user sees the app transition through the pages as it rebuilds the stack.

If I were to start again, I'd using MvvmCross which has built in support for what they call "Tombstoning". It will handle the process restart and restore the navigation stack for you.

I'd personally go down the path of borrowing the camera code from Xamarin essentials and fixing that, rather than rolling my own camera app.

[boris-df]

I'd personally go down the path of borrowing the camera code from Xamarin essentials and fixing that, rather than rolling my own camera app.

well - AFAIK there is nothing you can fix about this behaviour. If you - or someone - knows a fix to avoid Android kicking you app while you call the camera, i'm interested.

my way now is to use the code from the toolkit (see my link above) where they have a CameraView wich shows a live-stream into a ContentView that you can put anywhere on your own Page. It has a lot of downsides but i really have no better solution at hand

[JRE-6783]

I get the frustration, but I don't get the workarounds. At least for Xamarin.Android, the good old way of just using an OnActivityResult the way Android intended, does still work. I don't understand why it doesn't succumb to the same activity lifecycle issue, but it has never let me down yet - at a guess, maybe doing it direct does cause Android to exclude it from termination. It is shame we can't use this simplified approach, but the intended approach is still easy and works (for me anyway). If it aint broke...! My bigger concern is MAUI, which I have not used (nor Xamarin.Forms), but not sure how easy it is to wire up OnActivityResult and other platform specific lifecycles, especially if you are going cross-platform. Could be more of a fight at best.

[JRE-6783]

just thinking aloud. Maybe the direct approach does still have the lifecysle issue, but behind the scene restores it own OnActivtyResult result, so we never realise or need to worry. Doesn't change that Android have looked after their own approach, so stuck with using their approach.

[konradzuse]

OnActivityResult is called by the platform when the app is restarted. So if you can access the Activity lifecycle methods then you have the opportunity to handle it.

In Xamarin forms / Xamarin.essentials that Activity lifecycle stuff is buried in their code. That's why you'd need to refactor it to support this scenario.

Preventing the app from closing isn't a solution, since you have no control over that. You just need to handle it in every place where the app state must be persisted.

It's not an easy task in native Android code either, SavedStateHandle, Parcelize and all of that stuff to handle a fairly rare edge case when the app might get closed and restarted by the system.

[nbevans]

Basically the Android activity lifecycle design is either broken seemingly by design or highly developer hostile at the least. There is no simple solution. And by the time you realise that there is a problem it could be too late to massively re-architect your program to a different design that could possibly accommodate the flawed Android lifecycle design.

Many people deliberately avoid writing "native" Android code to avoid all the Intent/Bundle/Parcelable bull crap (and yes - it is bull crap - in 2010, let alone 2022, nobody should be having to deal with that level of complexity in passing state around inside your app). They would reach for a wrapper framework like XamForms/MAUI/Flutter/et al simply because these frameworks are "meant to" take care of all that boilerplate code. But imagine the shock when they realise actually these frameworks do not actually do that or even attempt to. And if they do, it's just hacks to workaround things on an case-by-case basis. It took years for Flutter to try to solve this "camera app results in main activity termination" problem and their solution is just a hack.

And if you target other platforms, you will feel aggrieved by all the additional complexity that's needed to support Android's broken activity lifecycle. It's no wonder really that cross-plat frameworks like those mentioned just don't bother to solve this problem for Android and Android alone. It's the only platform with such a messed up architecture. Is everyone waiting for Android to someday overhaul its activity lifecycle design to fix these problems, perhaps?

[dimonovdd]

Hi, Can you try that plugin?

If the problem is not solved, create new issue in that repository. I'll try to fix it.

Don't forget to provide a sample project with this error and the reproduction steps.

[nnikos123]

No It does not work. same behavior. tested on Samsung A21

[nnikos123]

The very same behavior exists on MAUI too , tested on Samsung S21 FE.
App crashes when "Don't keep activities" is set. in MAUI however crashing does not happen on every attempt to take a photo , compared to xamarin where app crashes in a very consisted way.

Now, if "Don't keep activities" hasn't been set crashing becomes less frequent but still happens especially when the user is taken photos very frequently.

bottom line, MAUI and xamarin are not able to cop with android lifecycle process. we need a framework for that since MS will do nothing about it as in the xamarin case.

Is MS really able to provide a real support for android 12+ on MAUI? I do not see any light. if the design principle is "works well under certain assumptions and preconditions" I strongly suggest to move AWAY from this trap and go native.

[dimonovdd]

@nnikos123 please provide a sample project with that plugin

https://user-images.githubusercontent.com/59065470/221382654-a70de344-fa86-4be5-a546-7cf9d762b6d2.mov

[nnikos123]

Hi @dimonovdd , I greatly appreciate your effort to resolve this undefeatable problem. Unfortunately I cannot provide my use case since camera is an integral part of a very large app.

After a close examination on all plugins in the net it came clear that the mechanism StartActivityForResult cannot serve the purpose , and all solutions based on this will fail simply because the callback of a destroyed app will not survive. Google has marked the "StartActivityForResult" obsolete/deprecated and strongly advise devs to use the registerForActivityResult mechanism.

The solution to the problem is to follow the steps bellow:

1. include RegisterForActivityResult in your relevant activity using the correct contract (activityResultContracts.TakePicture())
2. launch the result of the RegisterForActivityResult by passing an Android.Net.Uri which will point to the image file taken from the camera app.
3. save this uri into shared preferences in case that the app gets destroyed , you can override the OnSaveInstanceState for that.
4. now if the the callback that is passed to the RegisterForActivityResult is eventually called then we can retrieve the file from the uri we had passed in. However if the app is Destroyed/Restarted we can conditionally retrieve the uri from the preferences and finally access the image taken by the camera app.

This approach will work for all possible scenarios and is applicable to all frameworks , xamarin for android/ forms/maui etc. other frameworks like flutter much likely use the same methodology.

I'm also attaching some references:

https://developer.android.com/training/basics/intents/result https://github.com/xamarin/AndroidX/issues/648 https://github.com/gmck/ActivityResultApp

[dimonovdd]

@nnikos123 Why don't you make a Pull Request?

https://github.com/dotnet/maui/pull/13564 in this request I want to make android impl using registerForActivityResult

[boris-df]

The solution to the problem is to follow the steps bellow:

1. include RegisterForActivityResult in your relevant activity using the correct contract (activityResultContracts.TakePicture())
2. launch the result of the RegisterForActivityResult by passing an Android.Net.Uri which will point to the image file taken from the camera app.
3. save this uri into shared preferences in case that the app gets destroyed , you can override the OnSaveInstanceState for that.
4. now if the the callback that is passed to the RegisterForActivityResult is eventually called then we can retrieve the file from the uri we had passed in. However if the app is Destroyed/Restarted we can conditionally retrieve the uri from the preferences and finally access the image taken by the camera app.

The problem then is: What if the User should be able to take multiple pictures?

If all works without Android kicking our App from memory, it's easy:

• Take a picture, receive it to store and show it in the own UI and then call the camera again - until a user closes the Camera without taking a picture

But when the app is closed, this doesn't work

• when the picture was taken, Android restarts the App (which is a problem in itself because can take some time)
• Now the Xamarin Forms app doesn't know where it came from, no Navigation stack etc.
• Sure, we could save the uri and after a fresh app-start we see "oh, there's a picture waiting, fine" - but the app doesn't know where to put it (from which mask was the camera started? for which dataset?)
• and sure, we could save the Mask and the dataset IDs to ... but restoring a complex navigation stack after a fresh start... i don't know - this can be very slow and if this happens after every single camera-call, a user would propably call our support hotline anyway ...

Our App can receive photos from other apps (share with). When our app is started and the user navigates to a UI where a picture can be inserted, we show the "clipboard" with a pulsing + sign - with the above method we could do this after taking a picture too (by reading the stored uri for example) ... but then the user would have to navigate thru the data every time for every picture - not so smart :(

We "gave up" and implemented our own camera-view ... it can take only FullHD Photos as we don't get more from the Live-Stream as far as i can see ... but that has to be enough for our use case as nothing else is predictable

I really don't understand why such "basic" things like photo/video on a smart-device are such a nightmare to use :(

[nnikos123]

rebuild of the navigation stack is a must. we cannot escape from this. any app can be destroyed simply by send it to background. So the idea is when the mainActivity calls the Oncreate we have the ability to pick up the uri from the bundle , and inject the full path to the forms Application via LoadApplication call.

Samsung devices which I use, are very aggressive on memory management so lifecycle events and page page/activity reconstruction must be handled at all times.

[nbevans]

rebuild of the navigation stack is a must. we cannot escape from this. any app can be destroyed simply by send it to background. So the idea is when the mainActivity calls the Oncreate we have the ability to pick up the uri from the bundle , and inject the full path to the forms Application via LoadApplication call.

Samsung devices which I use, are very aggressive on memory management so lifecycle events and page page/activity reconstruction must be handled at all times.

Yes - so it's curious then that none on the main cross-plat frameworks, whether it be Xamarin Forms, MAUI, Flutter etc provide for this kind of navigation stack rehydration as part of their core offering. It's not really something that can be tacked on at a later date either.

But as already noted, even if rehydrating the nav stack were a standard thing - in many apps this could be extremely slow to do. What if the user had just executed a long running query? Merely swapping to another app to take a photo, then coming back to your app, could mean the results of that query are lost and would then need to be re-run. The fact is, mobile platforms (especially Android) have got broken multi-tasking models. If App A delegates a minor task to App B (e.g. to take a photo as a transactional operation) then App A should not be eligible for termination under even the most stressing of memory pressure. App A is the boss and hasn't given up its foreground rights as a process. If memory pressure is so bad then the operation to launch App B should simply fail and App A can then deal with that however it wants.

Why are these platforms not hibernating processes to disk when it needs to haul them out of memory? Then it could at least resume the process without any loss of state. I can't believe in 2023 we are having these type of conversations. It's archaic stuff.

[dimonovdd]

You can save navigation stack: https://github.com/dimonovdd/Xamarin.MediaGallery/blob/d5d261e17e7e4205ec86f394268a41073224f28d/Sample/Xamarin/Sample.Android/MainActivity.cs#L23

public class MainActivity : global::Xamarin.Forms.Platform.Android.FormsAppCompatActivity
{
     static App app;

     protected override void OnCreate(Bundle savedInstanceState)
     {
          //........
          app ??= new App();
          LoadApplication(app);
        }
}

[nbevans]

That's Android specific and there's no way from the XamForms API to contribute your own state into that anyway. And even if there were, that's still only a tiny part of the problem. You could build your own state storage mechanism any time the user does anything on your app - so that you're always ready to "rehydrate" from that stored state, if you need to. The problem is that none of this is handled by the cross-plat frameworks. And the amount of complexity it adds to an app would be huge. It would also be extremely costly to tack it on later, once you realise potentially months or years down the line that this type of design may be needed.