Design a new vector file format

krisgiesing commented 8 years ago

Latest update: https://github.com/flutter/flutter/issues/1831#issuecomment-1012654670

From earlier discussion, some important considerations are:

We don't want full SVG support. There is too much in the spec that is expensive, heavyweight, and/or duplicates what we already have in Flutter.
We should emphasize in our supported format those operations are effective/efficient/optimized in our graphics engine (Skia).
We may want to process the format at build time rather than support a full run-time interpreter.

Hixie commented 8 years ago

Other random thoughts (not all mine):

We really want to avoid supporting a format where there's a preexisting expectation of implementing features that are not going to be performant. For example, implementing SVG will lead people to expect full SVG support including scripting, HTML, SMIL, etc.
We should design our format explicitly very closely aligned to performance features in Skia, e.g. drawAtlas.

appsforartists commented 8 years ago

Vector support seems crucial for an app framework that supports various screen sizes and densities. Otherwise you end up with icons that look like this:

flutter-send-icon

Instead of this:

material-send-icon

appsforartists commented 8 years ago

@HansMuller

HansMuller commented 8 years ago

Kris mentioned this as well: we could do some limited SVG processing at build time. For example we might generate MD icon Widgets based on the SVG descriptions. If done carefully that might save space and increase fidelity to the original designs.

abarth commented 8 years ago

For material design icons specifically, we should investigate using the material design icon font.

@appsforartists For what it's worth, neither Cocoa nor android.view use vector formats for icons.

appsforartists commented 8 years ago

@abarth Doesn't surprise me about iOS - seems like they started with a hardcoded 3.5" screen and have tried pretty hard to keep that mental model/simplicity of supported resolutions ever since (see: the width of each iPhone being the same, or the original iPad and iPad mini having the same resolution). I don't know much about mobile development, but that's slightly surprising about Android not being vector-based.

In any case, the icons should not look aliased like they do in the above screenshot, and vectors are a nice solution for resolution independence. How you guys choose to implement to solve for pretty icons is up to you. :smiley:

abarth commented 8 years ago

How you guys choose to implement to solve for pretty icons is up to you.

Would you be willing to file a separate issue about the resolution issue you were seeing with icons? @krisgiesing implemented a resolution-aware asset resolver that handles many cases. It would be valuable to learn about the cases that it doesn't handle.

krisgiesing commented 8 years ago

I'm looking at adding some tests for the current resolution awareness support, so now would be a great time to understand if something's not working right.

abarth commented 8 years ago

It looks like the 3x has a device pixel ratio of 2.6, which we probably don't have an asset for.

krisgiesing commented 8 years ago

I'm looking at this specific case now. The device pixel ratio is 2.625; we're choosing the 3.0x asset, which is expected. However, something seems to be going on during the rendering that is causing some kind of extra aliasing artifacts. So although @abarth fixed the case of the Material Design icons by switching to a font, I'm still investigating on behalf of other types of image assets that need resolution dependent scaling.

krisgiesing commented 8 years ago

I'll open another bug to track this since it's is tangential to support of a vector format.

krisgiesing commented 8 years ago

See #2337

themacguffinman commented 7 years ago

Android supports the "VectorDrawable" format which is just a well-performing subset of SVG. I hope you consider using the same format, given that it already has some community support and the Android team has apparently deemed its performance adequate.

Hixie commented 7 years ago

VectorDrawable is certainly influenced by SVG but it's not a subset. Also it uses XML, which pretty much guarantees that it's not going to be the most performant vector format one could come up with. :-)

themacguffinman commented 7 years ago

It was wrong to describe VectorDrawable as a strict subset, but it still seems to be a balanced compromise between ensuring good render performance and ease of use. The VectorDrawable specification frequently defers to the SVG specification for key areas like path data syntax, making it much easier to convert SVG assets that many developers already use and share VectorDrawables that existing Android apps already contain.

I can't imagine that XML is a significant performance blocker because it only affects parsing. It seems to me that render performance - the kind of performance that affects Flutter's goal of 60fps apps - is more dependent on the set of vector operations that Skia will have to draw, not a parsing step that can be done once and cached.

I don't want to belabour the point but I can't help feeling that the work for this has already been done and therefore doesn't need to be shelved in the "after 2.0 release" milestone.

eseidelGoogle commented 7 years ago

We ended up using the Material Design font for the Material Design Icons. I'm not aware of any obstacles stopping someone from building this on top of Flutter today using direct dart:ui calls or a CustomPaint widget: https://docs.flutter.io/flutter/widgets/CustomPaint-class.html. We have no immediate plans to build such at this time.

deborah-ufw commented 7 years ago

$.02 - even janky browsers (~~and competitor framework React Native~~ never mind, their solution is hacky) can render SVG. The image export option requires creating a minimum of 3 files for each icon or art asset for my team. That's a poor practice compared to one SVG file can be displayed at any resolution.

For the above comment by Hixie, pre-existing expectations shouldn't inhibit implementing a worthwhile feature. I'd much rather be able to have basic support for SVG display (which solves many problems for me and my team) and find out extended options aren't available than rely on an ever-increasing number of PNGs to accommodate the ever-increasing number of screen resolutions.

I'm falling back to making a font for my team but that's not as good as native SVG support because the font format adds extra formatting for line-height and letter spacing that has to be adjusted in the app. Also, fonts want to be displayed at fixed resolutions (12px, 16px, 20px, 36px...) which is also limiting.

Thank you for cool Flutter framework. :)

Hixie commented 7 years ago

I would encourage anyone who wants SVG support to implement SVG support as a Dart package for Flutter.

sethladd commented 7 years ago

For those interested, I'd point to https://docs.flutter.io/flutter/dart-ui/dart-ui-library.html which is the low-level library for drawing.

deborah-ufw commented 7 years ago

From team: Dart library not good enough solution. Just passing it on.

Hixie commented 7 years ago

Can you elaborate on that? The whole of Flutter's framework is a Dart library; anything we implement here will likely be a Dart library. What are the requirements for "good enough"?

matanlurey commented 7 years ago

Case in point, I implemented a (simple, non-exhaustive) SVG parser in pure-Dart a while back:

https://github.com/matanlurey/svg

It would likely be somewhat trivial to either:

a. Create a runtime SVG -> SvgRenderElement, which uses dart:ui under-the-hood. b. Create a build-time compile step that converts an SVG into something like:

// compiled from star.svg
void drawStarSvg(canvas, width, height) { ... }

sethladd commented 7 years ago

@deborah-ufw if you're interested, we'd love to chat to learn more. Can you email me?

deborah-ufw commented 7 years ago

Hi Seth, matanlurey and Hixie - thank you very much for the response. I'm probably not the right person to talk to in depth, since it was our lead engineers who rejected the Dart plugin solution (thus leading to generation of a huge library of png's for our app, which makes most of us cringe also).

I would like to pass the link for this Github issue on to them and ask them to contribute technical reasons for their comment (which was made rather more colorfully than I passed on) - I will do that just as soon as I finish typing.

eseidelGoogle commented 7 years ago

They're also able to chat with our engineers directly via http://gitter.im/flutter/flutter as well as other contact methods listed https://flutter.io/support/ if needed. :)

deborah-ufw commented 7 years ago

I will add that in. Thank you! :D

gavindoughtie commented 7 years ago

Another approach is to integrate something like http://svgpp.org/ at the native level and use the Flutter plugin system to communicate with it. This would probably give you the best size/performance characteristics for full SVG support, but at the expense of maintaining a dependency.

dnfield commented 6 years ago

What about looking for SVG support at the engine level? I don't think it's reasonable or desirable to expect full support of the standard, but part of the appeal is support for cross functional teams and existing tooling.

zoechi commented 6 years ago

I got the impression Skia provides some basic SVG support. Is there no way to make this available to Flutter?

dnfield commented 6 years ago

From what I can tell, it's only been experimental and partial. There's an issue open here https://bugs.chromium.org/p/skia/issues/detail?id=5596 bit it's over a year old. They're also listing it in their roadmap but again not clear what priority.

Hixie commented 6 years ago

It wouldn't make sense to support SVG at the engine level because that wouldn't expose the SVG document model, which is a core part of SVG support.

If Flutter supports SVG, it's going to be with the eventual goal of actually truly supporting real SVG. It's against Flutter's philosophy to have half-hearted implementations of features.

dnfield commented 6 years ago

I don't see exposing the SVG document model as a core part of SVG support. That'd be a nice to have but not a must have for the use cases that I have in mind.

Yes, library level support would enable more features - but if the engine itself was capable of rendering (a significant subset of) SVG data, that'd presumably be enough to allow rendering vector image assets defined by that subset of SVG.

This issue is clearly bigger than that, but I'd be happy to take existing SVG assets and render them properly at different resolutions without converting them all to bulky raster images.

cbazza commented 6 years ago

Yeah, SVG asset file import would be really nice but the spec for SVG is over 800 pages long. Having said that to just import static files to render on Skia (which is a vector graphics library) shouldn't be a problem. So I am taking it on; as always I can only do work part time on weekends so you won't see stuff right away. Also I have a tonne of SVG files but they all use the same stuff (icon collections from flaticon) so I would appreciate sample SVG files from people that want this feature.

Carlos.

dnfield commented 6 years ago

I've been noodling on this a bit over time as well. Here's what I've come up with:

Engine Level implementation

This seems attractive for the following reasons:

In theory supports something like image: new AssetImage('graphics/background.svg') to have some C/C++ code parse and translate the SVG data to Skia rendering commands
At least one good candidate out there that should be able to plug into Skia fairly easily (https://github.com/svgpp/svgpp) at the engine level
WebKit's SVG logic would be awesome but seems to have far too many dependencies
I don't think librsvg would port too well (tightly coupled to Cairo, transitioning to Rust as a primary language - would need to work out way to get to Skia as a backend at the least)

But it has at least a few difficulties:

Properly handling text
Properly handling CSS (if at all? but adding it is no simple task)
Extending support for additional features (or allowing users to more gracefully add support for features) is challenging.
Seems like it's be very difficult to handle animations

Dart level implementation

Flutter's dart:ui offers a lot of the primitives from Skia that SVG needs already
I suspect text rendering would be much more easily handled at this level than at the engine level
Should be easier for more of community to maintain/extend
Should be able to more cheaply handle direct interaction with the parsing logic (perhaps to handle exceptions and/or animations?)

But has a few difficulties:

May not do much better supporting CSS
May not do much better supporting animations
Would likely be slower (but perhaps not by enough to matter, and could potentially be offset by AOT compiled SVG to Dart code)

Test data

As far as sample SVGs go, it'd probably be most useful to identify what SVGs are supported from the 1.1 test suite (or, if partially supported, what about them is partially supported). Those can be found here: https://github.com/w3c/web-platform-tests/tree/master/svg/import

Or here with some PNG comparisons: https://www.w3.org/Graphics/SVG/Test/20110816/

@cbazza, which way are you thinking of implementing this? At this point, I'm leaning more towards a Dart package level implementation.

dnfield commented 6 years ago

Also, I think icon type support is probably ly easily handled with something like IconData. It might be best to treat that separately, as it'd likely only have to support a single path per codepoint

cbazza commented 6 years ago

I am thinking of 'Dart level implementation' on top of dart:ui.

2 basic use cases:

(1) load static icon svg files with something like your "image: new AssetImage('graphics/background.svg')", where the file could come from the file system or over the network dynamically.

(2) Widget based svg support with DSX (JSX like construct) which would look very similar to: https://github.com/react-native-community/react-native-svg

Regarding 'Test data' I need real files from people that need this and not the w3 test suite. SVG support would be dependent upon what dart:ui supports so if a resired svg file contains stuff that can't be done with dart:ui, I would be able to find out fast.

Carlos.

dnfield commented 6 years ago

I do not think we should go for the react native SVG model. I get wanting to support real life cases, bit I think covering the w3 suite would give us a better basline of what works and what doesn't and how challenging it would be to implement)

cbazza commented 6 years ago

OK, there seems to be some misunderstanding... When I do (1) it will take standard SVG files and I'll try to support as much as it can given dart:ui (Skia). When I do (2), which is create widgets to handle SVG like constructs it will look very similar to that react-native-svg library. These are 2 very different things.

deborah-ufw commented 6 years ago

@cbazza the use case for svgs in our app is mainly for one-color svgs we use as icons. Here is a link to a zip file on Dropbox with a selection of recent svgs that are representative. https://www.dropbox.com/s/dp38wxc22625cvc/icons.zip?dl=0

amirh commented 6 years ago

Thanks for the interest in helping here, I would be happy to see an SVG Flutter plugin coming to life!

I wrote a tool that does some minimal parsing of a sequence of somewhat-SVG files and dumps out Dart files that we later use to show icon animations: vitool.

I used this tool to create the AnimatedIcons data, you can check the result by running the manual test app (cd dev/manual_tests; flutter run -t lib/animated_icons.dart).

It is far from being optimal or complete, but works for the current set of animated icons we have, and someone should feel empowered to fork/extend it.

amirh commented 6 years ago

@deborah-ufw for these single color icons something you can do is create an icons font, and show the icons using the Icon widget.

zoechi commented 6 years ago

@amirh doesn't seem to be easy to create color vector fonts

deborah-ufw commented 6 years ago

@amirh if you read the whole thread above, creating and defining a font with glyphs as icons is an extra series of steps that has to be taken because SVG is not supported. SVG support is much more efficient than a font. Also, zoechi is right - any vector art that has several colors is nearly impossible to make into a font.

amirh commented 6 years ago

Sorry - I looked at the sample icons and thought that you only needed single color icons.

cbazza commented 6 years ago

@deborah-ufw, thank you for the files, exactly what I was looking for and already found stuff that my files don't have.

@amirh, yes I have seen your great work already ;)

KalpeshVedak commented 6 years ago

Do we finally have a straight way to source an svg or vector asset xml into the image for flutter

VincentH-Net commented 6 years ago

@Hixie: please (re)consider that for multi-platform native mobile teams this is a real pain point, but that only partial, relatively simple SVG support is needed to solve this. Towards the SVG standard maintainers this could be considered half-hearted, but Flutter is for mobile developers, not for general standards authors. Flutter would win the hearts of mobile developers if it alleviates their very real pain, even if only in a pragmatic manner.

Please also consider that despite this long-existing need, the community support approach has failed to produce workable solutions in Flutter competitor Xamarin (in my 5 years of Xamarin experience - libs can't handle design tool exports, are too limited, not stable, paid and/or closed source).

Because of this I am not optimistic that the Flutter community will succeed where the Xamarin community failed. I do estimate that a practical. partial support for SVG icons is something that the Flutter team could deliver, using existing Skia primitives.

This would fit the Flutter goal well: enable mobile developers to deliver beautiful, high performance UX in record time. It would improve productivity, fidility and performance (an SVG loads a lot faster than all those PNG's; icon SVG's typically have modest computational cost because a requirement for visual clarity quite often leads to a shape of modest complexity).

The core need is for a productive workflow from commonly used creative designer tools (such as Adobe) to scalable native app icons.

The current PNG approach is very wasteful in terms of productivity and app size; it is just so primitive to generate and ship every icon in up to 9 fixed bitmaps (6 Android, 3 iOS) that are then post-processed at runtime. The problem increases steadily as resolutions increase in size and variation. It sticks out as a sore thumb in the otherwise delightful innovative and productive Flutter developer experience. I think Flutter has a real opportunity to win devs hearts & minds here,

Also @dnfield the above mentioned SVG features that are mentioned as obstacles for implementing in Flutter are in my experience either not relevant or are rare enough that falling back to PNGs for those design assets is a minor inconvenience. Don't cares: CSS, HTML, SMIL, Document Model, scripting Animation, Text: rare enough to fall back to other technologies for that

I hope the length of this comment conveys how much value a solution would have. Hth!

cbazza commented 6 years ago

@VincentH-Net I am 100% with you because I feel the pain too !!! Please send me some of your SVG files. Perhaps Xamarin struggled to deliver SVG support due to lack of vector graphics engine that could handle it; this is not the case for Flutter with Skia so I am very optimistic that a community solution will happen.

dnfield commented 6 years ago

FWIW, Xamarin also has Skia available in SkiaSharp. SVG can be tricky even for minimal implementation.

But @cbazza I'd be very interested in collaborating with you on this

dnfield commented 6 years ago

And on the animation front, I think that enabling Lottie would satisfy a lot of use cases (and probably do it with better tooling for designers).

flutter / flutter