flame-engine / flame

A Flutter based game engine.
https://flame-engine.org
MIT License
9.27k stars 910 forks source link

Unable to display a simple SpriteWidget #999

Closed ghost closed 3 years ago

ghost commented 3 years ago

Hi all - I'm trying a very simple task to see if I can use Flame as a widget for my RPG made with Flutter. I try to display a SpriteWidget showing 1 of the many images from a large sprite sheet.

I'm using Flame 1.0.0 RC 15.

Any help you can provide is greatly appreciated. Thanks!

Current bug behaviour

The SpriteWidget is drawn in the Widget Tree but the screen is empty - image is not displayed.

Expected behaviour

Display the image.

Steps to reproduce

// main.dart

import 'package:flame/extensions.dart';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:flame/src/spritesheet.dart';
import 'package:flame/src/widgets/sprite_widget.dart';
import 'package:flame/flame.dart';
import 'dart:async';
import 'dart:ui' as images;

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    SystemChrome.setPreferredOrientations([DeviceOrientation.portraitUp, DeviceOrientation.portraitDown]);
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);

  final String title;
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  SpriteSheet? spriteSheet;

  @override
  void dispose() {
    super.dispose();
  }

  @override
  void initState() {
    super.initState();
    asyncInitState();
  }

  Future<void> asyncInitState() async {
    await Flame.images.load('loot.png');
    setState(() {
      spriteSheet = SpriteSheet.fromColumnsAndRows(image: Flame.images.fromCache('loot.png'), columns: 5, rows: 14);
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
        appBar: null,
        body: Center(
          child: SingleChildScrollView(
            child: Column(
              mainAxisAlignment: MainAxisAlignment.center,
              children: <Widget>[
                Padding(
                  padding: const EdgeInsets.symmetric(vertical: 32.0),
                  child: spriteSheet == null
                      ? null
                      : SpriteWidget(sprite: spriteSheet!.getSprite(2, 2)),
                ),
              ],
            ),
          ),
        ));
  }
}

Flutter doctor output

[✓] Flutter (Channel stable, 2.5.2, on macOS 11.1 20C69 darwin-x64, locale en-CA)
    • Flutter version 2.5.2 at /Users/moonshotquest/Developer/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 3595343e20 (6 days ago), 2021-09-30 12:58:18 -0700
    • Engine revision 6ac856380f
    • Dart version 2.14.3

[✗] Android toolchain - develop for Android devices
    ✗ Unable to locate Android SDK.
      Install Android Studio from: https://developer.android.com/studio/index.html
      On first launch it will assist you in installing the Android SDK components.
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).
      If the Android SDK has been installed to a custom location, please use
      `flutter config --android-sdk` to update to that location.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.4, Build version 12D4e
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).

[✓] VS Code (version 1.60.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.27.0

[✓] Connected device (2 available)
    • iPhone X (mobile) • XXXXXXXXXXXXXXXXXXXXXXXXXXX • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-4 (simulator)
    • Chrome (web)      • chrome                               • web-javascript • Google Chrome 92.0.4515.107

More environment information

Log information

No error displayed.

erickzanardo commented 3 years ago

Without errors, is hard to know what is wrong.

Have you tried using the .asset constructor? Check this example: https://github.com/flame-engine/flame/blob/main/examples/lib/stories/widgets/sprite_widget.dart

It handles loading for you so you don't need to worry about that

ghost commented 3 years ago

Thanks for your reply! Yes, I tried and it works but it doesn't fit my purpose as it only display the full image. I need to use a spritesheet with many images and display only one e.g. 2nd row, 3rd column for instance.

No need for an error, I gave the full code - you can copy paste the above code in the main.dart pick any image you want, call it loot.png and put it in assets/images folder. Run the code and the screen is blank.

erickzanardo commented 3 years ago

The asset constructor supports selecting just a portion of the image, see our other example: https://github.com/flame-engine/flame/blob/main/examples/lib/stories/widgets/sprite_widget_section.dart

erickzanardo commented 3 years ago

Also this could be a better way of loading if you prefer loading with a SpriteSheet

    final image = await Flame.images.load('loot.png');
    setState(() {
      spriteSheet = SpriteSheet.fromColumnsAndRows(image: image, columns: 5, rows: 14);
    });
QiXi commented 3 years ago

Hi all - I'm trying a very simple task to see if I can use Flame as a widget for my RPG made with Flutter. I try to display a SpriteWidget showing 1 of the many images from a large sprite shee

The fastest way to draw sprites in Flutter is to use the Canvas.drawRawAtlas. Unfortunately Flame is not built on this method, maybe Flame 2.0 will be built on this ;). My performance measurements show that you can simultaneously draw 500 sprites on the screen while maintaining 60 fps, with 1000 sprites the fps will drop to 30. When drawing 1000 sprites simultaneously using drawRawAtlas, problems start on the GPU Flutter Engine, while the CPU retains performance headroom. Now I'm looking for a way to improve performance and draw 1000 stripes at 60 fps. With this information and the performance ceiling, you can roughly imagine whether this is enough for you to use Flutter Canvas, or if you should consider using GLES natively.

erickzanardo commented 3 years ago

@QiXi Flame support drawAtlas through the sprite batch API.

This is unrelated to the OP issue though. But feel free to open a discussion on the repository discussions to continue on this 😉

QiXi commented 3 years ago

I would be glad if my answer to a person saves him hundreds of man hours, it would also be interesting to compare the speed of drawRawAtlas and drawAtlas, but I do not have much free time to devote it to testing losing options in advance. Unfortunately, I don’t know what OP is.

erickzanardo commented 3 years ago

OP is original post, the original subject of this issue. Which isn't related to perfomance, but rather a Flame Widget that isn't rendering as it is supposed to.

QiXi commented 3 years ago

but I replied to ROP - root original post :P

to see if I can use Flame as a widget for my RPG made with Flutter.

ghost commented 3 years ago

Thank you for your quick reply! I tried but still get the same result, the widget is drawn but empty. I tried changing the image file to jpeg and random other pictures to see if maybe the issue is with the test file. No change.

See the bare minimum code - I removed all the widgets except a SpriteWidget under the Scaffold.

import 'package:flame/extensions.dart';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:flame/src/widgets/sprite_widget.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    SystemChrome.setPreferredOrientations([DeviceOrientation.portraitUp, DeviceOrientation.portraitDown]);
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);

  final String title;
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
        appBar: null,
        body: SpriteWidget.asset(
          path: 'loot.png',
          srcPosition: Vector2(256, 256),
          srcSize: Vector2(256, 256),
        ));
  }
}
erickzanardo commented 3 years ago

Just one addition thing I noticed in your code. You are calling this SystemChrome.setPreferredOrientations([DeviceOrientation.portraitUp, DeviceOrientation.portraitDown]); on your build method. Flutter widgets build methods should be pure, meaning no operations that cause side effects should be called inside then, and that operation does causes side effects. You will probably want to move it to your main method.

Either way, I will give the reproducible code a try later and report back.

ghost commented 3 years ago

I would be glad if my answer to a person saves him hundreds of man hours, it would also be interesting to compare the speed of drawRawAtlas and drawAtlas, but I do not have much free time to devote it to testing losing options in advance. Unfortunately, I don’t know what OP is.

Thank you so much mate! Definitely saving me tons of time :p

I think I will go with this solution as it only depends on flutter original libs and I feel I will be able to tweak more while getting great performance. If I need a game loop with render, then for sure Flame makes great sense. For now, I need to draw tons of sprites, like you would with Image.asset.

ghost commented 3 years ago

Just one addition thing I noticed in your code. You are calling this SystemChrome.setPreferredOrientations([DeviceOrientation.portraitUp, DeviceOrientation.portraitDown]); on your build method. Flutter widgets build methods should be pure, meaning no operations that cause side effects should be called inside then, and that operation does causes side effects. You will probably want to move it to your main method.

Either way, I will give the reproducible code a try later and report back.

Same issue when I remove this line. :(

QiXi commented 3 years ago

If I need a game loop with render, then for sure Flame makes great sense.

You can combine the solution, Using the drawRawAtlas capabilities inside the Flame Engine, perhaps it saves the time at the beginning of the path, so as not to write everything from 0. Later you will figure out where your bottlenecks are, 500 sprites is enough to create many small to medium games.

erickzanardo commented 3 years ago

@MoonshotQuest I found the problem. Flame SpriteWidget will use the size of its parent to render itself, when you add it directly to the Scaffold body, it will receive an infinity height, so the it the size calculate gets confused.

this behaviour is clearly causing some confusion as neither I remembered that, so we will think on a way to improve that, in the mean time, you can just wrap your SpriteWidget into a sized box, or any other widget that has defined dimensions and it will work.

ghost commented 3 years ago

@MoonshotQuest I found the problem. Flame SpriteWidget will use the size of its parent to render itself, when you add it directly to the Scaffold body, it will receive an infinity height, so the it the size calculate gets confused.

this behaviour is clearly causing some confusion as neither I remembered that, so we will think on a way to improve that, in the mean time, you can just wrap your SpriteWidget into a sized box, or any other widget that has defined dimensions and it will work.

Thanks so much for your help!! It does work great now 👍

Sharing the full code for future reader.

import 'package:flame/extensions.dart';
import 'package:flutter/material.dart';
import 'package:flame/src/widgets/sprite_widget.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);

  final String title;
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
        appBar: null,
        body: Center(
          child: Container(
            width: 256,
            height: 256,
            child: SpriteWidget.asset(
              path: 'loot.png',
              srcPosition: Vector2(768, 512),
              srcSize: Vector2(256, 256),
            ),
          ),
        ));
  }
}
ghost commented 3 years ago

If I need a game loop with render, then for sure Flame makes great sense.

You can combine the solution, Using the drawRawAtlas capabilities inside the Flame Engine, perhaps it saves the time at the beginning of the path, so as not to write everything from 0. Later you will figure out where your bottlenecks are, 500 sprites is enough to create many small to medium games.

I also found a way using 100% Flutter libs, no Flame. I believe this is also good to show this here in case someone needs. It also plays into the narrative that ultimately Flame functions and methods make it quicker to get to the same result.

Please feel free to rectify me! I will make a widget out of it, it's just the first draft here 👍 If you guys are on Twitter let's be friends @LeoLovesAi :)

import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:flutter/painting.dart';
import 'package:flutter/rendering.dart';
import 'package:flutter/widgets.dart';
import 'dart:ui' as ui;

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class SquarePainter extends CustomPainter {
  final ui.Image theImage;
  SquarePainter({required this.theImage});
  Paint highPaint = Paint()..filterQuality = FilterQuality.high;

  @override
  void paint(Canvas canvas, Size size) {
    canvas.drawAtlas(
      theImage,
      [RSTransform.fromComponents(rotation: 0.0, scale: 0.5, anchorX: 0.0, anchorY: 0.0, translateX: 0.0, translateY: 0.0)],
      [Rect.fromLTWH(256, 256, 256, 256)],
      [],
      BlendMode.src,
      null,
      highPaint,
    );
  }

  @override
  bool shouldRepaint(SquarePainter oldDelegate) => false;
  @override
  bool shouldRebuildSemantics(SquarePainter oldDelegate) => false;
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);

  final String title;
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  var _lootImage;

  @override
  void initState() {
    super.initState();
    asyncInitState();
  }

  Future<void> asyncInitState() async {
    ui.Image img = await imageLoad('assets/images/loot.png');
    setState(() => _lootImage = img);
  }

  Future<ui.Image> imageLoad(String asset) async {
    ByteData _data = await rootBundle.load(asset);
    ui.Codec _codec = await ui.instantiateImageCodec(_data.buffer.asUint8List());
    ui.FrameInfo _fi = await _codec.getNextFrame();
    return _fi.image;
  }

  @override
  void dispose() {
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
        appBar: null,
        body: Center(
          child: SingleChildScrollView(
            child: Column(
              mainAxisAlignment: MainAxisAlignment.center,
              children: <Widget>[
                Padding(
                  padding: const EdgeInsets.symmetric(vertical: 10),
                  child: Container(
                    height: 128,
                    width: 128,
                    child: _lootImage == null ? null : CustomPaint(painter: SquarePainter(theImage: _lootImage)),
                  ),
                ),
              ],
            ),
          ),
        ));
  }
}
QiXi commented 3 years ago

If I need a game loop with render, then for sure Flame makes great sense. @MoonshotQuest

small GameLoop

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  var deviceTransform = Float64List(16)
    ..[0] = 2.0 // window.devicePixelRatio
    ..[5] = 2.0 // window.devicePixelRatio
    ..[10] = 1.0
    ..[15] = 1.0;
  var previous = Duration.zero;
  var game = ExampleEcsBenchmark();
  game.onAttach();
  window.onBeginFrame = (now) {
    var recorder = PictureRecorder();
    var canvas = Canvas(
        recorder, Rect.fromLTWH(0.0, 0.0, window.physicalSize.width, window.physicalSize.height));
    Duration delta = now - previous;
    if (previous == Duration.zero) {
      delta = Duration.zero;
    }
    previous = now;
    var deltaTime = delta.inMicroseconds / Duration.microsecondsPerSecond;
    game.update(deltaTime);
    game.render(canvas);
    var builder = SceneBuilder()
      ..pushTransform(deviceTransform)
      ..addPicture(Offset.zero, recorder.endRecording())
      ..pop();
    window.render(builder.build());
    window.scheduleFrame();
  };
  window.scheduleFrame();
}
QiXi commented 3 years ago

... for my RPG made with Flutter.

This video will show you the target limits for your flutter game. Flutter performance test