Repaint happening when rendering circle over svg backround

[FlutterFlyer]

Hello Bill, thanks for providing the Jovial svg library. I’m working on a PoC project for plotting things on a floorplan. It seemed like a good way to display the floorplan was to convert it to an svg file from a CAD drawing and render it using your library. I was able to use your demo program to quickly demonstrate rendering of the floorplan, it worked perfectly! Next step was the ability to plot a filled circle over the floorplan. I found this article: https://www.woolha.com/tutorials/flutter-using-repaintboundary-examples. This does exactly what I need. It renders a complex background then renders a filled circle over the background. The article discusses using the RepaintBoundary widget to avoid repainting the background image when the circle is moved (in the example, they use mouse clicks and mouse movements to reposition the circle). I built this code and tried it without and with RepaintBoundary to see how the repaint is avoided. It works perfectly, you can move the circle and see that the repaint of the background is not occurring. Next I tried replacing the call to generate the background to one that uses your svg library to generate the floorplan as a background, a single svg image preprocessed into a .si asset. This does work, but I noticed that repainting of the background is happening every time the circle is moved. The repainting is problematic as it takes ~125 ms. I have spent some time trying to find a way to prevent the repaint without luck. The original example from Woolha that does work correctly uses the CustomPainter widget where isComplex is set to true. From my reading it seems like isComplex tells CustomPainter that the image is complex and should be cached rather than repainted. It seems like the svg library uses CustomPainter, but doesn’t set isComplex to true. Do you have any thoughts on how to prevent the repaint from occurring? Can you look into this and see if there is something that can be done in the svg library to prevent the repaint? I can share my code if you want to see this in action.

[zathras]

Just to be sure: You're using ScalableImageWidget, right?

Probably all that's required is for ScalableImageWidget to have an isComplex parameter that it passes on to the CustomPainter that it creates in its build method. That's a straightforward extension -- I would have put that parameter in there to begin with, if I had known about CustomPainter.isComplex! So when you confirm that you are using ScalableImageWidget, I'll spin a release candidate with that extension, so we can check that it addresses the issue.

[FlutterFlyer]

Yes, using ScalableImageWidget. Here is the code (_buildBackground() gets called, which in turn calls _buildFloorplan()):

Widget _buildFloorplan() { print('build floorplan'); return ScalableImageWidget( si: si!, key: _siWidgetKey, alignment: Alignment.center, scale: _fitToScreen ? double.infinity : _multiplier, background: Colors.white); }

Widget _buildBackground() { print('build background'); return RepaintBoundary( child: _buildFloorplan(), ); }

[zathras]

OK, try version 1.1.10-rc.1, with the new parameter ScalableImageWidget.isComplex set true.

[FlutterFlyer]

Unfortunately, adding isComplex did not change the behavior. To look a little deeper, I made a change in your widget.dart file in the paint() function around line 253. I did this:

void paint(Canvas canvas, Size size) {
    print('Running extensive painting');
    final bounds = Rect.fromLTWH(0, 0, size.width, size.height);

The print statement lets me see how often the repaint is happening. What I'm seeing is that I can click on the drawing to position the filled circle and paint() is always called. If I hold the mouse button and move the circle around, paint() gets called multiple times for a while then paint() stops being called. If you release the mouse button and press again, you get the same result again. isComplex doesn't change this.

So wondering what is different between the unmodified Woolha app and when I introduce ScalableImageWidget to render the background. One thing I noticed is that the Woolha app does this:

bool shouldRepaint(MyExpensiveBackground1 oldDelegate) => false;

So I made this change around line 315 in widget.dart:

  bool shouldRepaint(_SIPainter oldDelegate) => false;
  // bool shouldRepaint(_SIPainter oldDelegate) =>
  //     _preparing != oldDelegate._preparing ||
  //     _si != oldDelegate._si ||
  //     _fit != oldDelegate._fit ||
  //     _alignment != oldDelegate._alignment ||
  //     _clip != oldDelegate._clip;

When I do this, everything works perfectly! There is only one call to paint() and you can mouse click and drag the circle around endlessly with no additional calls to paint(). So I'm thinking there is an issue with the testing you do in shouldRepaint(). It must be producing true some of the time when it should be producing false. Can you look into this? If you determine that the logic is correct and needed, might it be possible to provide an override I can activate in my code to force shouldRepaint() to return false?

[zathras]

Interesting. I have a guess... Can you try this?

  bool shouldRepaint(_SIPainter oldDelegate) =>
      _preparing != oldDelegate._preparing ||
      _si != oldDelegate._si ||
      _fit != oldDelegate._fit ||
      _alignment.x != oldDelegate._alignment.x ||
      _alignment.y != oldDelegate._alignment.y ||
      _clip != oldDelegate._clip;

If that doesn't fix it, can we narrow this down a bit, to find out why. shouldRepaint is giving true? Say:

  bool shouldRepaint(_SIPainter oldDelegate) {
      final result = _preparing != oldDelegate._preparing ||
      _si != oldDelegate._si ||
      _fit != oldDelegate._fit ||
      _alignment != oldDelegate._alignment ||
      _clip != oldDelegate._clip;
     if (result) {
        print("shouldRepaint true...");
        print(_preparing != oldDelegate._preparing);
        print(_si != oldDelegate._si);
        print(_fit != oldDelegate._fit);
        print(_alignment != oldDelegate._alignment);
        print(_clip != oldDelegate._clip);
     }
     return result;
  }

[FlutterFlyer]

Your fix with .x and .y did not improve things. Here is the debug output from starting the app (no circle drawn yet):

flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: Running extensive painting

Now I click once on the screen to draw the first circle:

flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: Running extensive painting flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: Running extensive painting

Now I drag the circle around with depressed mouse button until the paint() operation stops happening:

flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: Running extensive painting flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: Running extensive painting flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: Running extensive painting flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: shouldRepaint true... flutter: true 4 flutter: false flutter: Running extensive painting

So it seems that _preparing is the culprit.

[zathras]

Oh, I see what's happening. Thanks for the great detective work on this!

SVGs can have embedded images, and image decoding is (sensibly) done asynchronously in Flutter. ScalableImage uses ref counting to only decode images once. When a new instance of the widget is created on an SI that has no images, or where all the images have already been decoded, it briefly puts itself into the "preparing images" state, and creates a small async task to check the reference counts. It shouldn't do that; in the case where to preparing is necessary, it shouldn't put itself in that state.

It either needs to either synchronously adjust the reference counts, or it needs to more intelligently hand off the SI from the old delegate to the new. I'm leaning toward the latter, since that wouldn't involve any addition to the visible API, but I want to double-check my thinking on this after lunch :-)

[FlutterFlyer]

I can try out your fix when yo have it ready. I'm on the east coast and about to head home from work. If you have something today I'll test it in the morning.

I do have another enhancement request. As I mentioned, I'm rendering a floorplan and plotting locations on it. When I render it on Windows 10, the app shows the floorplan at maximum dimension vertically, but centered horizontally with white space on either side. Its hard to know where to plot a "location dot" without knowing where the image is on the screen relative to a 0,0 origin. I think it also changes when you maximize the window. It would be helpful if ScalableImageWidget could provide the maximum dimensions of the image and even better the offset to the image location on the screen.

[zathras]

OK, the changes are pretty self-contained, but it's probably easiest to just bring over the change if you have a git repo, or if not, just copy https://github.com/zathras/jovial_svg/blob/main/lib/src/widget.dart over.

About giving the real image dimensions and offset -- that's straightforward enough, assuming you can give me a Size object telling me the widget's size. Given that, I could produce a Rect whose x and y values are the offset, and whose width and height are the (scaled) width and height the SI would be drawn to. So, for example, for BoxFit.fill, you'd get back a Rect whose x and y values are 0, and whose width and height are the same as Size you provide.

Would that work? I glanced at GestureDetector, and it looks like it gives you what you'd need to come up with that Size object.

[FlutterFlyer]

The changes you made fixed the issue. No unnecessary repaints occurring! Thanks for fixing this and the fast response to my issue post. Let me digest your comments on dimensions and offset and I'll get back to you later today.

[FlutterFlyer]

I had a chance to think about my request for dimensions and offset. Your comment about GestureDetector puzzled me a bit, but I finally realized that I haven't described the real need well enough. Let me do that now. Here is the code to build the background image:

Widget _buildBackground() {
    return RepaintBoundary(
      child: ScalableImageWidget(
        si: si!,
        key: _siWidgetKey,
        fit: BoxFit.contain,
        alignment: Alignment.topLeft,
        background: Colors.white,
        isComplex: true,
      ),
    );
  }

When run, this yields:

This shows a floorplan that is top left justified (0,0). You can see there is white space to the right. I assume that the aspect ratio of the image is 1 and will remain so if the window size is changed. I can do this to learn about the window size:

    size = MediaQuery.of(context).size;
    height = size.height;
    width = size.width;
    orientation = MediaQuery.of(context).orientation;
    height_app_bar = AppBar().preferredSize.height;
    print(
        "Media height = $height, width = $width, orientation = $orientation, height_app_bar = $height_app_bar");

on my system this produces:

Media height = 681.0, width = 1264.0, orientation = Orientation.landscape, height_app_bar = 56.0

If I maximize the window, I see the same display, but the numbers change:

Media height = 1017.0, width = 1920.0, orientation = Orientation.landscape, height_app_bar = 56.0

I plan to have a mariadb database on a separate server that keeps a list of items being tracked and the location in X,Y coordinates, maybe in meters. I will periodically reach into the db and get the latest positions and draw them over the floorplan as small filled circles. The issue is what offset to use for the circle drawing to get them on the floorplan at the right position. I can know the full width and height of the floorplan in meters from out CAD drawing. But I don't know how to translate that into screen pixels, I don't know where on the screen the image pixels are drawn. What I need to know is the pixels per meter for both X & Y so that I can calculate the pixel offset for drawing the circles. I'm thinking you could provide the full image width and height in pixels and I should be able to calculate the pixels/meter knowing the actual dimensions of the floorplan in meters.

Another challenge is when the window is resized. When I plot a red circle in one of the offices on the right side:

then I maximize the window:

You can see that the dot is now in the wrong position. Updated pixels/meter values need to be calculated and the circle redrawn at a different location.

So I think the ask is to provide a means for me to get the number of pixels used in both X and Y directions to draw the entire image and be able to get updated values when the window is resized. Maybe the values need to be in floating point to prevent an error from being introduced by round-off. Does this make sense? Another ScalableImageWidget variable could be passed that provides a pointer for the destination of the height/width. If included, the height/width are determined during rendering and passed back.

[zathras]

OK, I see the issue here.

To get the needed pixel positions, you need to know the size of the of the drawing area into which the SI was drawn. What you're asking for is: a) A callback to give you that size, and b) A way to translate that size into the offset and scaled size of the SI The library can do (b), but not (a), but it's straightforward for you to get (a) yourself.

Why can't the library do (a)? It could try, by remembering the drawing area's size the last time the SI was painted. But there's no guarantee that's the current size! It probably is, most of the time, or even all of the time on some platforms, but that's not a guaranteed behavior.

I can even imagine a circumstance where it might not be. Whether or not any real platforms do what I'm about to describe is irrelevant, because future platforms might. So, imagine a platform with video RAM to burn, and where they decided to optimize the case of a phone frequently switching between portrait and landscape. On such a platform, RepaintBoundary might cache the rendering result for both orientations. If it were to do that, the one that happened to be painted last might not be the one currently showing.

But, how do you draw your circles over the floor plan? I don't know what you're planning (or what you've done so far), but it seems like you'd want to have a widget overlapping the ScalableImageWidget. That widget would, I assume, be a CustomPaint that's constrained to exactly overlap the ScalableImageWidget. And, your CustomPainter subclass has a paint method, which is called with a Size argument. That's exactly (a), at exactly the time you need it.

I guess you could achieve the same effect by building up a scene with a bunch of widgets, and getting the geometry constraints right (including the BoxFit and Alignment at the root of that widget tree), but that sounds to me like more work, just so you can get something with more overhead :-)

So, the only thing that's left is to figure out how ScalableImageWidget scales and positions the underlying SI. It's a straightforward enough calculation -- it could be a static method on ScalableImageWidget like:

    static Rect scaledBounds(ScalableImage si, Size drawingAreaSize, Alignment a, BoxFit f)

Now, this calculation is totally standard, and not all that difficult, but exposing it would be a nice convenience and would avoid making programs duplicate code that's in the library. It's basically the code in _SIPainter.paint() that figures out the translate and scale.

[zathras]

It just occurred to me... I could let client code to ScalableImageWidget pass a painting function, where the default is:

void paint(ScalableImage si, Canvas c) => si.paint(c);

That way, you could do any painting you wanted above or below the SI. The canvas would already be scaled and translated, and you could consult ScalableImage.viewport to know the width and height.

That's a two-line change for me to add; would that be easier?

[zathras]

Oh, hang on - if you swapped in a paint function, there would be no way to interpose the RepaintBoundary, so the SI would need to be be re-rendered every time you changed the overlay.

[FlutterFlyer]

Here is the test app I'm using:

import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:jovial_svg/jovial_svg.dart';
import 'package:measured_size/measured_size.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  String si = 'assets/floorplan2.si';
  await rootBundle.load(si);
  final firstSI = await ScalableImage.fromSIAsset(rootBundle, si);
  await (firstSI.prepareImages());

  runApp(MyApp(firstSI));
}

class MyApp extends StatelessWidget {
  final ScalableImage firstSI;

  const MyApp(this.firstSI, {Key? key}) : super(key: key);
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Floorplan',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: HomePage(title: 'Floorplan', firstSI: firstSI),
    );
  }
}

class HomePage extends StatefulWidget {
  const HomePage({
    Key? key,
    required this.title,
    required this.firstSI,
  }) : super(key: key);

  final String title;
  final ScalableImage firstSI;

  @override
  State createState() => _HomePageState();
}

class _HomePageState extends State<HomePage> {
  final GlobalKey _paintKey = GlobalKey();

  Offset _offset = Offset.zero;

  ScalableImage? si;
  final _siWidgetKey = GlobalKey<State<HomePage>>();

  var orientation, size, height, width, height_app_bar;

  Widget _buildBackground() {
    return RepaintBoundary(
      child: MeasuredSize(
        onChange: (Size size) {
          setState(() {
            print(size);
          });
        },
        child: ScalableImageWidget(
          si: si!,
          key: _siWidgetKey,
          fit: BoxFit.contain,
          alignment: Alignment.topLeft,
          background: Colors.white,
          isComplex: true,
        ),
      ),
    );
  }

  Widget _buildCursor() {
    return Listener(
      onPointerDown: _updateOffset,
      onPointerMove: _updateOffset,
      child: CustomPaint(
        key: _paintKey,
        painter: MyPointer(_offset),
        child: ConstrainedBox(
          constraints: const BoxConstraints.expand(),
        ),
      ),
    );
  }

  @override
  void initState() {
    super.initState();
    si = widget.firstSI;
  }

  @override
  Widget build(BuildContext context) {
    // getting the size of the window
    size = MediaQuery.of(context).size;
    height = size.height;
    width = size.width;
    orientation = MediaQuery.of(context).orientation;
    height_app_bar = AppBar().preferredSize.height;
    print(
        "Media height = $height, width = $width, orientation = $orientation, height_app_bar = $height_app_bar");
    return Scaffold(
      appBar: AppBar(
        title: const Text('Floorplan'),
      ),
      body: Stack(
        fit: StackFit.expand,
        children: <Widget>[
          _buildBackground(),
          _buildCursor(),
        ],
      ),
    );
  }

  _updateOffset(PointerEvent event) {
    RenderBox? referenceBox =
        _paintKey.currentContext?.findRenderObject() as RenderBox;

    Offset offset = referenceBox.globalToLocal(event.position);

    setState(() {
      _offset = offset;
    });
  }
}

class MyPointer extends CustomPainter {
  final Offset _offset;

  MyPointer(this._offset);

  @override
  void paint(Canvas canvas, Size size) {
    print("ptr offset = $_offset, size = $size");
    canvas.drawCircle(
      _offset,
      4.0,
      Paint()..color = Colors.red,
    );
  }

  @override
  bool shouldRepaint(MyPointer oldDelegate) => oldDelegate._offset != _offset;
}

I've added a couple print() statements, also one in your svg library in the paint() function. When I start the app, I get:

flutter: Media height = 681.0, width = 1264.0, orientation = Orientation.landscape, height_app_bar = 56.0 flutter: background size = Size(1264.0, 625.0) flutter: ptr offset = Offset(0.0, 0.0), size = Size(1264.0, 625.0) flutter: Size(1264.0, 625.0) flutter: Media height = 681.0, width = 1264.0, orientation = Orientation.landscape, height_app_bar = 56.0

The "Media height =" line is from the app build() function (around line 99) where MediaQuery is used to get information. This represents the size of the window in pixels as well as the app bar size. The "background size =" line comes from the svg library, printing the size variable passed to paint(). You can see it is essentially the full size of the window minus the app bar. The "ptr offset =" line comes form the paint() function (line 139) for drawing the filled circle. You can see this is also the full size of the window minus app bar. In both cases this is the full size of the canvas that can be drawn on, not the size that was drawn. I tried adding the MeasuredSize widget wrapped around ScalableImageWidget, hoping that this will return the drawn image size. This is the "Size(1264.0, 625.0)", which again is the full canvas size not the drawn image size. If I use the mouse button to draw the red circle at the bottom right extreme edge of the layout drawing:

got this:

flutter: ptr offset = Offset(999.0, 624.0), size = Size(1264.0, 625.0)

The Offset indicates the extreme edge of the drawn image, 999.0 pixels. But I can't find a way to get this size back programmatically. In your post from yesterday at the end, you suggested adding scaledBounds(). This might be worth trying. You suggested something else in another post then backed away from it. Not sure what you are thinking. If I know the X,Y offset to the bottom right edge of the drawing, I can calculate the offset required to paint the circles in the right position.

[zathras]

It looks like you're trying to do two different things in one step.

The first thing you need to do is when the user clicks the mouse, you need to figure out where the mouse click is on your floor plan, IN THE FLOOR PLAN'S COORDINATE SYSTEM. You do that by applying the reverse of the scale and translation of the floor plan, when it was drawn at the screen size at the time of the mouse event. You want to get that screen size during the processing of the mouse event. I've never had to do that, so I'm not totally certain of the best way to go about it, but it might be as easy as RenderBox.size.

Then, when you draw your overlay, you need to transform that position from the floor plan's coordinate system to the coordinate system of your painter. You do that by taking the Size you're passed in the paint method, figuring out the needed scale and translation, and applying it. That's the size you're getting in

When you get the mouse event, the screen size isn't necessarily the same as when the scene was last rendered. It probably is most of the time, but that's not guaranteed.

In both cases, to calculate the translation and scale factors, you use the size of the SVG, the size of the screen, the BoxFit, and the alignment. It's a fairly standard computation; here's the code from _SIPainter's paint method:

      final double sx;
      final double sy;
      switch (_fit) {
        case BoxFit.fill:
          sx = size.width / vp.width;
          sy = size.height / vp.height;
          break;
        case BoxFit.contain:
          sx = sy = min(size.width / vp.width, size.height / vp.height);
          break;
        case BoxFit.cover:
          sx = sy = max(size.width / vp.width, size.height / vp.height);
          break;
        case BoxFit.fitWidth:
          sx = sy = size.width / vp.width;
          break;
        case BoxFit.fitHeight:
          sx = sy = size.height / vp.height;
          break;
        case BoxFit.none:
          sx = sy = 1;
          break;
        case BoxFit.scaleDown:
          sx = sy = min(1, min(size.width / vp.width, size.height / vp.height));
          break;
      }
      final extraX = size.width - vp.width * sx;
      final extraY = size.height - vp.height * sy;
      final tx = (1 + _alignment.x) * extraX / 2;
      final ty = (1 + _alignment.y) * extraY / 2;
      canvas.translate(tx, ty);
      canvas.scale(sx, sy);

The only thing the library can reasonably do is expose that calculation. There's some value in that, but it adds a bit of API clutter.

[zathras]

I decided it's not that much clutter -- see the (untested) ScalingTransform.

[FlutterFlyer]

I played with this change a bit today, but can't find a way to make it do what I need. Here is a better explanation of what I want to do:

The floorplan svg I have shown in previous posts is 39.78 meters high by 61.72 meters wide (as directly measured in the building). I have a database that contains a list of assets that are measured from the 0,0 position (bottom left corner) of the floorplan. I want to plot an indicator over the top of the floorplan to show exactly where the asset is located. For instance, one of the assets could be a printer that is located at X = 33.84 meters, Y = 2.13 meters (close to bottom center in the floorplan). I can translate this to a percentage of the svg dimensions, 2.13 / 39.78 = 5.35% from the bottom and 33.84 / 61.72 = 54.83%. Now I want to plot an icon (or something) over the floorplan so that I can see exactly where the print is located. If I knew the dimensions of the svg image in screen pixels, I can easily calculate the offset from the top-left that I need to use to paint the printer icon. But I haven't found a way to get these dimensions out of your latest changes.

I think I may have confused you with the sample app program where you could click on the image to draw a filled circle. This is not the the real use case. It was just and easy way to demonstrate drawing a filled circle on the screen over the floorplan image. I wanted to test overlaying two images. The real application will specify an offset in meters obtained from a database.

In one of your posts above, you mentioned the size of the SVG. I haven't found a way to get this in all the things I have tried. If I could get it, would the size be in screen pixels? I tried to do this:

  Widget _buildBackground() {
    Rect rect = si!.viewport;
    Offset off = Offset(rect.width, rect.height);
    return RepaintBoundary(
      child: MeasuredSize(
        onChange: (Size size) {
          final pos = ScalingTransform(
                  containerSize: size,
                  siViewport: si!.viewport,
                  fit: BoxFit.contain,
                  alignment: Alignment.topLeft)
              .toSICoordinate(off);
          print('Image pos: $pos');
          setState(() {
            print(size);
            print(rect);
          });
        },
        child: ScalableImageWidget(
          si: si!,
          key: _siWidgetKey,
          fit: BoxFit.contain,
          alignment: Alignment.topLeft,
          background: Colors.white,
          isComplex: true,
        ),
      ),
    );
  }

and I get this:

flutter: Media height = 681.0, width = 1264.0, orientation = Orientation.landscape, height_app_bar = 56.0 flutter: ptr offset = Offset(0.0, 0.0), size = Size(1264.0, 625.0) flutter: Size: 1264.0, 625.0 flutter: Offset: 0.0, 56.0 flutter: Position: 632.0, 340.5 flutter: Image pos: Offset(121.0, 75.4) flutter: Size(1264.0, 625.0) flutter: Rect.fromLTRB(0.0, 0.0, 348.2, 217.1)

It seems like the svg size should be 1001.0, 624.0 screen pixels, but I don't get anything near that. What am I doing wrong?

[zathras]

Look more closely at the example in example/lib/scale_translate.dart, particularly the part just after this comment:

// We store the clicks in the SVG's coordinate space, so we need to
// transform them back out to the container's.

The size of the SVG, in the SVG's notion of pixels, is given by ScalableImage.height and ScalableImage.width.

[FlutterFlyer]

With a little more study, I figured out how to use ScalingTransform. Everything is now working perfectly! Thanks for implementing this and responding to my posts. I think the changes to fix the repaint bug and adding ScalingTransform to the SVG library are great improvements to your work. You could certainly close this issue now and do a release at some point. I jammed the ScalingTransform enhancement request onto the end of repaint bug issue. Would you like me to open a separate issue for this and delete the contents from this issue? This would allow you to have the repaint bug and ScalingTransform issues tracked separately.

[zathras]

Glad it worked out! I'm glad that we got isComplex figured out. No need for another issue.

Marking as fixed in 1.1.10-rc.3