Sprite rendered with 'renderRect' with Fragment shader is not used.

[JuhBoyBet]

Current bug behavior

I use the SpriteAnimation with a custom Fragment shader written in GLSL. To render it I use the 'renderRect' method but the fragment have no effect on the image. I tried to directly use the 'canvas.drawImage' api but the result is the same, the fragment shader is not used. However, it works well on api like 'canvas.drawCircle'.

Expected behavior

The fragment shader should be used to apply transformation on the sprite.

Steps to reproduce

FutureOr<void> onLoad() async {
    program =  await ui.FragmentProgram.fromAsset('shaders/your-flame-fragment.frag');
    image = await Flame.images.load('your-image.jpeg');
    animation = SpriteAnimation.fromFrameData(...);
}

  @override
  void render(Canvas canvas) {
    var p = Paint()..shader = _program.fragmentShader();
    canvas.drawCircle(Offset.zero, 100, p); // works nicely

     // Just in case it would works that way, but it has no effect as well.
    // animation.getSprite().paint = Paint()..shader = _program.fragmentShader();
    animation.getSprite().renderRect(
        canvas,
        Rect.fromCenter(
            center: Offset(mSize.x / 2, mSize.y / 2),
            width: mSize.x,
            height: mSize.y),
        overridePaint: Paint()..shader = _program.fragmentShader());
  }

Flutter doctor output

[✓] Flutter (Channel stable, 3.7.7, on macOS 12.2.1 21D62 darwin-arm64, locale en-FR)
    • Flutter version 3.7.7 on channel stable at /Users/j.boyer/development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 2ad6cd72c0 (13 days ago), 2023-03-08 09:41:59 -0800
    • Engine revision 1837b5be5f
    • Dart version 2.19.4
    • DevTools version 2.20.1

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
    • Android SDK at /Users/j.boyer/Library/Android/sdk
    • Platform android-33, build-tools 33.0.1
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13F100
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)

[✓] VS Code (version 1.76.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.60.0

[✓] Connected device (3 available)
    • sdk gphone64 arm64 (mobile) • emulator-5556 • android-arm64  • Android 13 (API 33)
      (emulator)
    • macOS (desktop)             • macos         • darwin-arm64   • macOS 12.2.1 21D62
      darwin-arm64
    • Chrome (web)                • chrome        • web-javascript • Google Chrome
      111.0.5563.64

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

More environment information

• Flame version: 1.6.0
• Platform affected: android, ios, web

Thanks & have a good day. PS: I could work on this, but i need some insight on where to dig first.

[renancaraujo]

From the fragment shader docs: "Some APIs, such as Canvas.drawImage, ignore the value of the shader."

[JuhBoyBet]

ouch, i didn't catch that line, sorry. Well, as far as i know renderRect and stuff works with drawImage so .. I suppose it will be complicated to apply FShader on it. Maybe my solution is to use drawRect and set uniform & sampler2D to get the fragment working. But it reduce my opportunity to works with most of the library in place - like animations.

Thanks for your fast answer anyway, i see that it's not planned for now so i'll find another way to do it.

[spydon]

Thanks for your fast answer anyway, i see that it's not planned for now so i'll find another way to do it.

Not planned from our end, but Flutter will most definitely improve on this in the long run, it's not much we can do from the Flame side except waiting.

[JuhBoyBet]

Np, understood ! Thanks for your time 🙏