Session Replay support for Flutter

[bruno-garcia]

Add support for Sentry's Session Replay: https://sentry.io/for/session-replay/

[Rafik-Belkadi-malou]

from how many thumbs up will you start to consider its implementation?

[marandaneto]

from how many thumbs up will you start to consider its implementation?

This is blocked by https://github.com/flutter/flutter/issues/117382

[YasserDRIF]

We are using Smartlook currently, Adding this to Sentry would be amazing

[bruno-garcia]

We're working on it! Wanna join the early adopter release? Join the waitlist and discussion about the feature:

• https://github.com/getsentry/sentry/discussions/63138

[vaind]

Replay alpha version now available for Android in 8.6.0-alpha.2 - please share any and all feedback.

To try out replay, you can set following options:

  await SentryFlutter.init(
    (options) {
      ...
      options.experimental.replay.sessionSampleRate = 1.0;
      options.experimental.replay.errorSampleRate = 1.0;
    },
    appRunner: () => runApp(MyApp()),
  );

Access is limited to early access orgs on Sentry. If you're interested, sign up for the waitlist

[mcosti]

Where should we post the issues we find? In this topic or each in a separate issue?

Here's what I found so far:

[sentry] [debug] WidgetFilter obscuring: Text("Editor")
[sentry] [debug] WidgetFilter obscuring: Text("My Festivals")
[sentry] [debug] WidgetFilter obscuring: Text("Messages")
[sentry] [debug] WidgetFilter obscuring: Text("Profile")
[sentry] [error] Replay: failed to capture screenshot.
         'dart:ui/painting.dart': Failed assertion: line 26 pos 10: '<optimized out>': Rect argument contained a NaN value.
         #0      _AssertionError._doThrowNew (dart:core-patch/errors_patch.dart:51:61)
         #1      _AssertionError._throwNew (dart:core-patch/errors_patch.dart:40:5)
         #2      _rectIsValid (dart:ui/painting.dart:26:10)
         #3      _NativeCanvas.drawRect (dart:ui/painting.dart:5936:12)
         #4      ScreenshotRecorder._obscureWidgets (package:sentry_flutter/src/replay/recorder.dart:137:14)
         #5      ScreenshotRecorder._capture (package:sentry_flutter/src/replay/recorder.dart:103:9)
         <asynchronous suspension>
         #6      Scheduler._run.<anonymous closure> (package:sentry_flutter/src/replay/scheduler.dart:53:41)
         <asynchronous suspension>

Also, quite low performance, especially on Android. I am developing a chat app and I can see a ton of logs for obscuring data, including the bottom navigation which gets obscured on every scroll. (Although not technically re-rendered).

Edit: I do not see any recordings showing up for iOS and for Android, the whole 9 minute recording is stuck on the initial first tab (not the loading screen, but the first screen from the navigation bar). I can see the network logs as I'm scrolling through my app, but the recording itself is stuck.

[✓] Flutter (Channel stable, 3.22.3, on macOS 14.1 23B74 darwin-arm64, locale en-NL)

[vaind]

I do not see any recordings showing up

I suspect that's because of the error - it fails during frame creation so the previous frame gets repeated. I'll have a look at the failing assertion.

If the app you're working on is available publicly (and easy to set up), I can have a look which components are causing the issue.

[mcosti]

My app is not available publicly, but I am more than willing to assist with anything I can help with.

I've just had a look and the error gets triggered on every page I have, both including and excluding pages that have the bottom navigation bar (that was a hunch I had, but it's not that)

[mcosti]

Replacing the code with this:

  void _obscureWidgets(Canvas canvas, List<WidgetFilterItem> items) {
    final paint = Paint()..style = PaintingStyle.fill;
    for (var item in items) {
      paint.color = item.color;
      if (item.bounds.hasNaN) {
        _logger(SentryLevel.debug,
            "Replay: skipping widget with NaN bounds: $item. ${item.bounds}");
        continue;
      }
      canvas.drawRect(item.bounds, paint);
    }
  }

Works, of course.

Adding this inside of _obscureIfNeeded:

    if (rect.hasNaN) {
      logger(SentryLevel.debug, "Widget $widget has NaN Bounds. $offset $size");
    }

Results in these logs:

[sentry] [debug] Widget Text("Contact", debugLabel: ((englishLike bodyLarge 2021).merge((((whiteMountainView bodyLarge).apply).apply).merge((((whiteMountainView bodyLarge).apply).apply).merge(unknown)))).apply, inherit: false, color: Color(0xffffffff), family: Poppins_regular, familyFallback: [Poppins], size: 14.0, weight: 400, letterSpacing: 0.5, baseline: alphabetic, height: 1.5x, leadingDistribution: even, decoration: Color(0xffececec) TextDecoration.none) has NaN Bounds. Offset(NaN, NaN) Size(229.4, 21.0)

I should mention that the Contact label is not visible at all in the moment of running this code.

To give some more hints about how my app works,

final PageController navigationPageController = PageController();
...

  Widget build(BuildContext context) {
    WidgetsBinding.instance.addPostFrameCallback((_) => checkIfOnboardingShouldBeShown(context, mounted));
    return Scaffold(
        bottomNavigationBar: ValueListenableBuilder(
            valueListenable: currentNavigationIndex,
            builder: (context, currentIndexValue, child) {
              return Theme(
                data: ThemeData(
                  splashColor: Colors.transparent,
                  highlightColor: Colors.transparent,
                  brightness: Theme.of(context).brightness,
                ),
                child: BottomNavigationBar(
                    onTap: (index) => navigationPageController.jumpToPage(index),
                    currentIndex: currentIndexValue,
                    type: BottomNavigationBarType.fixed,
                    items: [
                      BottomNavigationBarItem(icon: const Icon(Icons.cut), label: tr(LocaleKeys.app_editor)),
                      BottomNavigationBarItem(
                          icon: const Icon(FontAwesomeIcons.calendar), label: tr(LocaleKeys.app_myFestivals)),
                      const BottomNavigationBarItem(icon: Icon(Icons.chat_sharp), label: 'Messages'),
                      BottomNavigationBarItem(icon: const Icon(Icons.manage_accounts), label: tr(LocaleKeys.app_me))
                    ]),
              );
            }),
        body: UpgradeAlert(
          upgrader: upgrader,
          child: PageView(
            controller: navigationPageController,
            onPageChanged: (index) => currentNavigationIndex.value = index,
            children: [VideoEditorHomePage(), MyFestivalsScreen(), ChatScreen(), ProfileScreen()],
          ),
        ));

I am using a PageView

[vaind]

I am developing a chat app and I can see a ton of logs for obscuring data, including the bottom navigation which gets obscured on every scroll. (Although not technically re-rendered).

and

I should mention that the Contact label is not visible at all in the moment of running this code.

Understanding which widgets are visible and which are obscured/overlayed by something else can be tricky. We have some logic that checks out-of-screen widgets as well as some common ways to hide them (visibility/opacity). I'm open to suggestions how this could be done, at least in your scenario - we can see if it can be generalized or there are counter-examples.

That the widget filter evaluates the Contact label means it is part of the widget tree and it hasn't been trimmed with the visibility heuristics. Any chance you can show the widget tree screenshot from developer tools so we can see if there's anything we can rely on (generically) to trim out hidden pages?

[mcosti]

I should mention that I am a python developer, so this kind of thing is completely out of my realm, but I'll do my best.

This is the widget tree:

As you can see, all the pages are there.

When the contact button is not in view

When the contact button is in view

The only difference I could see is "Needs repaint"

[vaind]

The only difference I could see is "Needs repaint"

Yes, I can't see anything else either. I'm looking for some component that decides whether it's visible. Maybe the itself has visibility or sth 🤔

[vaind]

@mcosti I've tried to reproduce what you're seeing with NaN with PageView but couldn't. Any chance you can come up with a repro that you're able to share?

[vaind]

TODOs

• [ ] (blocker) Provide a way to redact specific elements (hook into redaction logic)
• [ ] (blocker) Measure overhead
• [ ] (blocker) Mask custom graphics
• [ ] (blocker) Mask webviews
• [ ] Pause/resume API
• [ ] support orientation change
• [ ] Clock drift fix: use UTC, or single time clock read, or stopwatch
• [ ] Add quality setting low/medium/high

[rhinck]

I apologize for my ignorance, but does this issue being closed indicate that there is now Flutter support for session replay? If so, how would I go about trying it out in my project? (I couldn't seem to find any docs offering additional information)

[vaind]

Session Replay beta for iOS and Android should come out in the next release

[buenaflor]

It's available starting from Sentry Flutter 8.9.0

[bruno-garcia]

@vaind could you help get it on our docs? https://docs.sentry.io/platforms/flutter/

If we add a note that it's experimental, like we did for the other SDKs, it should be OK. Now is the time since we expect to GA this in a a month or two.