How to write custom ImageFilters?

[matthew-carroll]

I'm interested in writing custom ImageFilters as a possible approach to a photo filter application, as well as other possible use-cases.

I see that the current options for ImageFilter are bounded and that they're implemented in native land.

Is it even possible to write new ImageFilters, either in Dart or on the native side? If so, are there any resources to help figure out how to do that?

[nathansbird]

I'm surprised that 8 months later, this still hasn't been implemented. This is actually incredibly feasible. The only thing that needs to be done is for someone to add the dart wrappers to the flutter engine around here: painting.dart, and C++ methods here: _imagefilter.cc. The existing blur logic that's being used, as well as a list of all possible effects that could be implemented can be found here: skia effects

This would be a really cool feature that I myself would use all the time for text color inversion animations and similar visual effects.

[matthew-carroll]

@theEpicality I still agree with you that opening up these effects would be great. One aspect that I hadn't considered initially is the Flutter binary size. Any increase requires substantial justification because it needs to be as small as possible. So in all likelihood, exposing swathes of skia will first require some kind of modular Flutter engine that allows certain aspects to be conditionally included on a per-app basis.

FYI @chinmaygarde @jason-simmons @xster @gspencergoog @goderbauer

[Adrian-Samoticha]

Flutter 3.7's SDK now ships with a shader compiler that allows you to define hot-reloadable shaders in your pubspec.yaml file, which can be used by the Canvas API. I wonder if that means that support for custom ImageFilters is on the way.

I am personally interested in this as ImageFilter.blur doesn't handle transparency properly (I assume the alpha channel simply doesn't get blurred), which can cause issues like this: https://github.com/GroovinChip/macos_ui/discussions/322#discussioncomment-4686058

[Adrian-Samoticha]

I’ve been trying to find a workaround for this (given that it’s been five years since this issue was opened), but I just cannot figure out a way to capture the current screen into an Image that’s efficient enough to use in a RenderBox’s build method.

Has anybody tried anything like this? Any help’s appreciated.

[tattyd]

+1 - am tying myself in knots here trying to get a hold of an Image object from the Canvas given to me in RenderProxyBox.paint()...

Adrian, I'm curious what non-efficient solutions you found...

[Adrian-Samoticha]

+1 - am tying myself in knots here trying to get a hold of an Image object from the Canvas given to me in RenderProxyBox.paint()...

Adrian, I'm curious what non-efficient solutions you found...

Check out the captureAsUiImage method from the screenshot package.

Unfortunately, you need to add a delay of ~20 ms for it to work (see https://github.com/flutter/flutter/issues/22308). I guess that makes this approach infeasible in situations where you cannot afford to have such a delay.

[tattyd]

Ah, thanks! Yeah, that might be a little slow for me.

For the sake of the FR - here's what I'm trying to achieve:

• Compose several image filters together
• Some of them are already available (e.g. blurring)
• Others would be custom (thresholding an image so only pixels above a certain intensity are kept, and recolouring by using pixel value to index into a gradient).
• My plan was to write some custom shaders for these and wrap some CustomPaint widgets to make them reusable.

In my case, since the source image ultimately comes from draw calls, I think I can create a canvas off-screen, draw into that, then render that to an image and pass it as a sampler2D into a shader inside a CustomPainter. A little long winded but OK. The disadvantage is the filters can only be applied to procedurally drawn content, not a tree of widgets.

[jonahwilliams]

There is a mechanism for doing this in the flutter_shaders package. I'm also experimenting with a (Impeller only) API for specifying a fragment shader as an image filter: https://github.com/flutter/engine/pull/53490

[Adrian-Samoticha]

@jonahwilliams If it’s Impeller-only, then I guess it won’t work on macOS? I believe Flutter still uses Skia on Desktop platforms.

[iapicca]

@jonahwilliams If it’s Impeller-only, then I guess it won’t work on macOS? I believe Flutter still uses Skia on Desktop platforms.

let alone web