Speculation: standalone binary format that converts to/from svg

[rsheeter]

A ton of work goes into binary image formats but afaik not as much into binary vector formats. Compressed text svgs are often recommended. Strikes me one could build a COLR v1 inspired standalone binary vector format that converts to/from svg nicely while offering a compact representation that is easy to rasterize because the tree deliberately mirrors how you flow through to actually draw the thing.

A COLR style binary format with a tree of graphic constructs offers two benefits:

1. More compact in transfer, in memory, and on disk
   • expected outcome for svg based on COLR v1 experiments:
     • svgz is compact at rest but doesn't compress [brotli standalone or woff2 in font form] all that well
     • svg is large at rest but compresses well
2. Rasterizer is efficient and relatively simple to implement
   • COLR v1 deliberately mirrors widely available 2d graphic constructs; the impedance mismatch between format and graphics constructs is low
   • binary format loads faster, uses less memory
   • For example, https://github.com/BlackFoundryCom/black-renderer took a couple of weeks

It could be interesting to play with this idea. A simple test @drott and @anthrotype suggested on IM might be to try encoding single SVGs.

There are use cases today like icons where fonts represent the graphic nicely but the loading experience is bad, we want something close to img where the size can be known in advance. Ofc that could still be a font file if you could declare load an asset from a font here, don't do normal text fallback, and know the size in advance. Bonus points for then incrementally transferring so a page can declare a bunch of them and it Just Works.

A standalone format that works as an image could work. A way to say <img [load this glyph from this font]> (no text fallback, treated like an image) would also work.

EDITED: tried to clarify benefits 7% less confusingly

[Lorp]

Interesting. Could be worth talking to Google Maps people, such as @antinharasymiv, author of Prototyping a Smoother Map, to see how efficient their vector tiles are.

[antinharasymiv]

I can't speak to the efficiency of the vector format (I could try find someone who knows enough who can) but you can kind of get a sense by just using the network inspect mode of Chrome while using Google Maps. Basically all the tile requests are the ones that look like "https://www.google.com/maps/vt/pb=!1m8!3m7!1m2!1u....." and each will be loading the information to render a 256x256 tile. If you scroll around you can get a sense for the average byte size (and observe how much it changes at zoom levels, or when you're in highly detailed areas of cities vs nature).

That said, you can actually get a pretty long way just with SVG. There are a lot of tricks to minimize SVG size (by combining the path commands etc) and the format gzips pretty happily in transit. We played around with rendering map tiles in SVG too and could make it comparable sizes for a standard DPI image, and show significant savings for HDPI/retina/2x tiles.

I confess to not fully understanding your use case here, but a big advantage of just making do with SVG is that it's handled natively by most platforms and so requires no extra processing – when internet connections were slower or more data constrained you saw big savings in reducing byte size, but now that they're much improved, it may be that the cost of custom rendering or processing exceeds the gain.

[nigeltao]

A couple of drive-by comments:

• https://github.com/flutter/flutter/issues/1831 "Design a new vector file format" is a similar discussion for the Flutter UI toolkit. "We don't want full SVG support. There is too much in the spec that is expensive, heavyweight, and/or duplicates what we already have in Flutter". But they want something.

• https://github.com/google/iconvg is my attempt at a new binary vector file format that y'all might find interesting. From an older announcement, "The [2016 era] Material Design icon set consists of 961 vector icons. As SVG files, this totals 312,887 bytes. As 24 24 pixel PNGs, 190,840 bytes. As 48 48 pixel PNGs, 318,219 bytes. As IconVGs, 122,012 bytes". You can write an IconVG decoder in under 4,000 lines of C code, plus a dependency on Cairo and/or Skia. I'd certainly like to see <img src="foo.ivg"> support in HTML, but that obviously requires buy-in from the web browser community.

[behdad]

Thanks @nigeltao. That's very interesting. Unfortunate that we are almost ready to ship COLRv1; hopefully in the future, when iconvg lands widely, we can embrace the existing sbix table to ship those. COLRv1 allows for sharing across glyphs as well as variations though, which are hard to achieve using a standalone vector-graphics format.

[nigeltao]

Unfortunate that we are almost ready to ship COLRv1

Yeah, COLRv1 didn't exist when IconVG was invented in 2016. At the time, I was hopeful that IconVG might eventually be part of a color-emoji-in-fonts solution. As you know, there wasn't consensus then around COLR (as opposed to stuffing PNGs or SVGs into OTFs), and COLRv0 wasn't perfect (which is why COLRv1 exists, of course).

In 2021, though, shipping OTF/COLRv1 is definitely the thing to do, given buy-in from the Big Four orgs (Adobe, Apple, Google and Microsoft) as well as software like FreeType.

Outside of OTF, this issue is about a stand-alone format. And I think that that's a separate and interesting discussion.

COLRv1 allows for sharing across glyphs as well as variations though, which are hard to achieve using a standalone vector-graphics format.

IconVG is an extensible format and I recently posted some rambling thoughts about future extensions at https://github.com/google/iconvg/issues/4#issuecomment-853643903

The feature list includes "Compound Graphics", which would allow sharing across glyphs. Other features ("Custom Parameters", "Arithmetic Opcodes", "Tweening") would allow something like variations.

[Hixie]

I'd be very interested in feedback on the format proposed at the bottom of https://flutter.dev/go/vector-graphics, especially in terms of how it would address the use cases here.

[rsheeter]

feedback on the format

After reading the first ~20 pages this broadly lgtm. Before worrying about details I'd be interested in understanding 1) how serious we are about this being Flutter-only (why not general purpose?) and 2) what the path from the doc to a shipping implementation looks like.

Might be worth a VC to better understand your problem space, how you expect to progress, and how it might relate to our projects. For example, we want to deliver icons to Flutter [today done via fonts], we want GPU rendering of fonts, icons, and emoji, etc etc.

[Hixie]

My goal would very much be for it to be general purpose, in particular I would hope to see implementations in browsers and support for the format as an output/export format from common editing tools. Obviously getting there from here involves getting lots more people on board. I don't have an answer to #2 yet (but presumably Flutter would be where we would start with shipping a renderer, and we'd offer some conversion tools from SVG to get started). Happy to chat more!

[rsheeter]

https://github.com/TinyVG/specification appears to be another effort in this area.

EDIT: https://github.com/RazrFalcon/resvg/blob/master/docs/rendering.adoc is also informative [from https://news.ycombinator.com/item?id=29629792]

[rsheeter]

https://github.com/Tencent/libpag, if efficient for stills, might serve this purpose as well.

[yisibl]

As far as I know libpag seems to be more focused on animation and is used as an alternative to lottie.