Catch crashes using catcher with manual email report mode

[jhass]

https://pub.dev/packages/catcher

[tclaus]

Is it expected under iOS to see a 'Send Error' Dialog? By using the Catcher.sendTestException(); there is no Dialog under iOS and not under Android simulator.

[jhass]

In debug mode (or always?) Flutter catches the exceptions thrown in widget callbacks, so those then don't bubble up to the global error handler. An easy way around this for testing is to crash a new coroutine:

final crash = () async => Catcher.sendTestException();
crash();

Thanks for updating the Podfile! Didn't remember that :)

[jhass]

Future improvement for this is a custom error page maybe. The shipped one looks... not good and broken in the dark theme. But for now the dialog should do fine.

[tclaus]

This has some drawbacks, on my perspective (at least on iOS no advantages) First: On App start, I see immediately a "Inspiration crashed" - Message. (On existing App, and on fresh installed App)

Local log shows this: 2020-11-24 08:00:16.133824+0100 Runner[18358:7300809] Metal API Validation Enabled. 2020-11-24 08:00:16.497377+0100 Runner[18358:7300809] Init App Auth Handler. 2020-11-24 08:00:16.684274+0100 Runner[18358:7301033] flutter: [2020-11-24 08:00:16.681925 | Catcher | FINE] Using release config. 2020-11-24 08:00:16.900464+0100 Runner[18358:7301033] flutter: [2020-11-24 08:00:16.900334 | Catcher | FINE] Catcher configured successfully. 2020-11-24 08:00:16.954202+0100 Runner[18358:7301035] fopen failed for data file: errno = 2 (No such file or directory) 2020-11-24 08:00:16.954347+0100 Runner[18358:7301035] Errors found! Invalidating cache... 2020-11-24 08:00:16.982775+0100 Runner[18358:7301033] flutter: No session to restore found 2020-11-24 08:00:16.984375+0100 Runner[18358:7301033] flutter: [2020-11-24 08:00:16.984341 | Catcher | INFO] Setup localization lazily! 2020-11-24 08:00:17.232896+0100 Runner[18358:7301034] fopen failed for data file: errno = 2 (No such file or directory) 2020-11-24 08:00:17.233003+0100 Runner[18358:7301034] Errors found! Invalidating cache...

On normal App workflow and a crash - dialog appears, I can send a report, and it seems that App runs again.

Do you know that (on iOS) a crash report is sent automatically to the App Store which can be received (per Version) directly into Xcode? Maybe this also happens on (real) Android PlayStore. I don't know what value this has on a flutter app.

[jhass]

On App start, I see immediately a "Inspiration crashed" - Message. (On existing App, and on fresh installed App)

Do you see this on Android too? I didn't. This might be an actual bug on iOS, just that it doesn't crash the main isolate and thus the app.

MissingPluginException(No implementation found for method listen on channel insporation/appauth_authorization_events)

Do you know that (on iOS) a crash report is sent automatically to the App Store which can be received (per Version) directly into Xcode? Maybe this also happens on (real) Android PlayStore.

I know. But I don't get any notifications about crashes, do you?

If you have Google Play Services installed and installed the app from the Play Store then Google automatically collects crash reports too, yes. AFAIK there's not even an opt-out to this. But there's no notifications, little device metadata and since the redesign quite terrible UX in my opinion.

So currently for the Android nightlies distributed via F-Droid, there's no way to get at crash reports. I actually have a user there whose experiencing a crash and getting insight into that is the whole motivation for this topic. More importantly I want to keep F-Droid a first class distribution channel even after the first proper release, and Play Services free Android a supported platform.

But I think this has merits beyond:

• It ensures the opt-in generally, even on Android.
• It centralizes crash collection. We have one place where we can look at crashes from all platforms
• It's push not pull. I already got tired of having to login into the Apple console every day again and manually check the reports through the slow interface.
• Opportunity for further automation. This sends us a mail right now, but we reasonable effort there could be build tooling on top of that, for example a ticket system or a bridge that turns those mails into Github issues or whatever.
• Fine grained user control over the data. Catcher supports many handlers, I already run a Sentry instance so that would have been an easy and nice choice too. I picked manual email because that gives users the opportunity to edit the mail before sending it to us, if for example they don't want to share certain device metadata.
• It gives us control over the data users share with us. Once we don't need it anymore, we can delete the report fully. This option is not present with any of the platform provided crashlytics.

Maybe users always dismiss the dialog

First of all, let's be straight, we'll never get to a 100k users, or even 10k. So we won't reach scale where it's important to collect all crash reports in order to prioritize issues according to frequency.

I really don't think users will always dismiss it. And if only 1 out of 100 sends it that's all we need. We have the crash report.

Users have to tell us their mail address

Yes, that might reduce the number of crash reports we get. But given it's a manual email users can also opt into getting the content to us in a different way, if they really care about the crash being reported without revealing anything about themselves. Also here again, I don't think it's important to get all crashes, one report per type of crash is already enough.

If the crash is rare enough to not bother the user, it's probably also not important to fix. Put the other way around, a high exposure crash will also increase the chance of it being reported. We get a natural filter onto the things that are important to fix.

Finally and most importantly, this is actually a huge opportunity. In the platform provided crashlytics the reports are anonymous, meaning we have no way to talk to the user experiencing the issue usually. Or if they contact us separately about the crash you have to do much guesswork just what crash report correlates to them. This is not the case here, we gain immediate rapport with the user and can ask for further details on what steps lead to the crash etc. For a small project like this I think this is actually invaluable and much more valuable than having the big numbers about crash frequency, crash free sessions etc.

[jhass]

By the way the above could probably just get a stub implementation on iOS. I see there's also the following:

MissingPluginException(No implementation found for method listen on channel insporation/share_receiver)

which is probably needed for the app to receive shares. Android implementation is here: https://github.com/jhass/insporation/blob/main/android/app/src/main/kotlin/jhass/eu/insporation/ShareEventStream.kt

[jhass]

I think the latest commit gets rid of the extra crashes for now.

[tclaus]

Thanks for pulling some concerns away. I do see some benefits, and still have some concerns.

What do you think to have a kind of switch to turn this off some day in future on turn on if we suspect any special errors?

Lets start with this and see what we get.

[jhass]

Yes exactly, we can always easily iterate or remove this in case it turns out to be better in another way, it's not very invasive, code wise.

Regarding the dialog in the background you're seeing, that's very weird, I've never noticed anything like this. I guess this gesture is some kind of back navigation on iOS? Anyways, might very well be a Flutter bug, and as you say should only happen in extreme conditions anyhow already.

In the mid to long-term we can replace the report mode with something custom build and that should also give us an opportunity to properly exit the app or even fully restart it maybe. But for now I think done is better than perfect :)