Support SVG image rendering

[mpaperno]

OK, here we go...

I got everything to match Chrome perfectly. There are a ton of visual tests including various usages of SVGs I could think of, including for pattern generation (probably need to split that into separate pages soon).

As discussed, I'm using a Picture to store the rendered SVG DOM, and that remains nicely scalable.

SVGs with no specified size

It would all be rather simple actually except for one complication... SVGs with no intrinsic size, meaning no width or height attributes in the root element. It wouldn't be fun w/out a little challenge, right? So I looked at what browsers do, esp. Chrome.

Both FF and Chrome will assign a default size of 150x150 (arbitrary it seems, yes). After that they somewhat diverge in what happens when you use these images in a canvas w/out specifying a size, as in:

• drawImage(img, x, y);
• the 9-argument variant, which has no way to specify the source size, only which part of it you want to clip;
• when creating a pattern from an SVG there's also no way to specify a size;

So, what Chrome does in these situations is scale the SVG to the current canvas' size. So if you drawImage(img, 0, 0) with one of these, it will always stretch to the full canvas. Is that strange? Yes... a bit.

Notably is also does this when using such an SVG for a pattern.

FF doesn't scale anything and just tries to draw the SVG at 150 x 150. But it's pretty wonky and for some reason the image ends up translated part way across the canvas and also cut off.... kinda bizarre actually. And when used in a pattern it's completely FUBAR. You can see what I mean in the visual tests page.

So, I chose to do what Chrome does, even though it adds some code complexity. It's debatable if we want to do that, but it'll be easier to cut it out than add it later.

This is all commented in the code in the relevant parts.

To make this work for creating patterns, I had to get the current canvas' size into CanvasPattern's from_image() API function. So I passed the whole canvas as an argument through the JS layer. I could just be a size, but not sure that's any better/simpler/faster. Or if there's some other way to get the current size that I couldn't see...

SVGs with fonts

The one thing that doesn't seem to work quite right is SVGs with fonts. Or more specifically lists of font family names. If it can't find the first name in the list, the whole search fails and it falls back to some default font. I haven't researched this further (besides trying a bunch of stuff) and I'm not sure we can do anything about it since that's all presumably handled somewhere behind the scenes.

But the SVG DOM parser does require a FontMgr to be passed to it when reading in data, so maybe there's some trick here I'm not aware of yet. All the (2 or 3) examples I've seen just use the default manager, which is how I have it now. I have no idea if this manager has anything to do with choosing the next font in the list, though it does affect which fonts it seems to know about (more on that below).

The whole thing is rather vague really (surprise!)... Skia docs are a year old and don't mention fonts at all until you find the Builder internal class docs. But I'm not sure what the skia-safe bindings are even doing since they seem to invoke some non-existing methods... 🤷🏼 Maybe it'll make more sense to you.

The last point on fonts is I wonder if we could inject any loaded custom fonts into the SVG parser somehow... like maybe via this FontMgr that we need to give it. I hacked around with the FontLibrary and at one point got the SVGs to use only a custom font I loaded (regardless of the font family specified in the SVG)... which was amusing but not very practical. But it did suggest that whatever FontMgr we give the parser may make a difference.

But I couldn't get Chrome to use custom fonts when rendering SVG to canvas either (even though same font worked directly in eg. fillText()), so I guess it's not that important.

Related: #172 Related: #182

[mpaperno]

I just realized I was testing the non-native-sized image stretching with square canvases. Turns out Chrome will preserve a square aspect ratio of the image, not just stretch both dimensions. Which makes sense. All the images I've come across that don't specify a size use a square aspect ratio.

It's really like what happens when one opens an un-sized SVG in a browser... it will stretch to fill the smallest window dimension while preserving aspect ratio.

So I'll fix that and re-open when ready. Shouldn't be a major change. And I can deal with the "draw canvas as image" question once that's settled.

[mpaperno]

OK, that was an easy fix. Looking good now.

Not sure why this still shows changes that you just merged.... I'll try to re-open this and see if that clears it up. If not I'll rebase and force.

[mpaperno]

Hmm, not sure... still showing the clipping fix as changes, but says no conflicts... I guess I'll leave it alone for now?

[samizdatco]

Actually, do you think you could open a PR with just the non-SVG-related Image changes? It would be nice to be able to work through merging that uncontroversial part first before focusing on the less settled SVG stuff (with its open questions about font handling, sizing, etc.)

[mpaperno]

OK done, but now I expect no controversy about that one... 😃 j/k, no need to rush it.

I'll rebase and force this one once that's merged, then.

[mpaperno]

Well with the fonts I've verified that the FontMgr I'm giving to the SVG parser does have all my system fonts in there (as I'd expect). But a list in font-family still doesn't work (as an attribute, nor as CSS/style). If it can't find the first font in the list then it goes to default.

I've tried some searches but so far nothing on this particular subject (hard to filter down to anything relevant though).

It also doesn't seem to understand generic ones like monospace, serif, etc. I see neither does the ctx.font property in skia-canvas. So I guess those need to be registered somehow... interesting.

As a naive test I added this to FontManager and fed the resulting FontMgr to the SVG parser. This is when the only font the SVG renderer would use is the one custom one I had loaded.

  pub fn get_font_manager(&self) -> FontMgr {
    let mut assets = TypefaceFontProvider::new();
    for (font, alias) in &self.fonts {
      assets.register_typeface(font.clone(), alias.as_deref());
    }
    assets.into()
  }

If I can figure out how to "build" a FontMgr from the default one plus custom fonts stored in FontLibrary then I can get them into the SVG renderer, though I don't see how to control what happens after that. So not really sure it's worth it. I mean really any fancy font in an SVG should be outlined anyway.

[samizdatco]

I've got a semi-reasonable handle on the Skia font management mess after recently working on the Typesetter struct again, so I'm happy to take this part on. It'd also be nice to share as much font infrastructure as possible with the text rendering side of things…

Re: generic fonts like serif, monospace, fantasy, etc. Skia doesn't handle those for us, but I'd really like to support them (especially fantasy obvs). The problem is not knowing what fonts will be installed on a given system, especially given the free-for-all of linux distros. If we could come up with a css-style list of fallbacks for the different platforms, I could rig something up to match the generic names against them.

I wonder if anyone has already come up with the canonical serif, sans-serif, and monospace lists at least?

[mpaperno]

I wonder if anyone has already come up with the canonical serif, sans-serif, and monospace lists at least?

Well there are common (?) "CSS font stacks" for web use... lots of web resources for that, but eg. https://www.cssfontstack.com/ (even fantasy ;)

[mpaperno]

It'd also be nice to share as much font infrastructure as possible with the text rendering side of things…

Yea I should just drop it but I couldn't help myself... 😅

You probably already figured it out, but I did manage to get all the fonts collected including custom ones... I'm sure this is a stupidly naive implementation (should be cached, at minimum) but it got the testing done.

  // FontLibrary
  pub fn get_font_manager(&self) -> FontMgr {
    let mut mgr = FontMgr::default();
    let mut assets = TypefaceFontProvider::new();
    // default family must be first
    let default_fam = mgr.legacy_make_typeface(None, FontStyle::default()).unwrap();
    assets.register_typeface(default_fam, None);
    mgr.family_names().for_each(|f| {
      let font = mgr.match_family_style(&f, FontStyle::default()).unwrap();
      assets.register_typeface(font, None);
    });
    for (font, alias) in &self.fonts {
      assets.register_typeface(font.clone(), alias.as_deref());
    }
    assets.into()
  }

image.rs -> load_svg

-  let dom = svg::Dom::from_bytes(&data, FontMgr::default());
+  let mgr = FONT_LIBRARY.lock().unwrap().get_font_manager();
+  let dom = svg::Dom::from_bytes(&data, mgr);

So yea now I can use custom fonts in SVGs as well as system fonts. As long as the font listed first is found, otherwise it just falls back to the default. So that's really the mystery/issue.

<text x="50" y="68" font-family="Monoton, 'Segoe UI', Consolas, Menlo, monospace" font-size="48" fill="#FFF" text-anchor="middle"><![CDATA[410]]></text>`

I don't think it's a major blocker, personally. Would be nice to figure out but I don't want to get hung up on it for now.

But hey I figured out what the default font is on my system, so that's cool (Segoe UI). :) I guess browsers have settings for that, which they seem to respect for SVGs also.

[mpaperno]

Alot of conflicts for an uncontroversial PR... lol.

[samizdatco]

I think I've got a clean rebase working at this point. I'll force push but feel free to overwrite if you're currently working on something you haven't pushed…

[mpaperno]

Looks good I think except that one missing decode bit. And whatever you decide to do with Buffer.

[mpaperno]

Yea the natural size is debatable. Because we actually assign a natural size (150x150) when we create the graphic (the containerSize must be set otherwise it won't draw anything). So in my view FF actually "lies" and there's no way to tell what size it actually assigned to it. Though OTOH the way Chrome does it there's no way to tell that the original wasn't sized.

I considered using the viewbox dimensions as default size but currently we have no way to get to them (w/out parsing the svg ourselves). And that's also a debatable approach (the viewbox isn't really supposed to determine anything about the final image).

[samizdatco]

Ah, so it looks like Chrome doesn't just return a default 150×150 square: it returns 150 for the height but it uses the viewbox aspect ratio to scale the width to some proportion of 150. So there actually is some useful information being communicated; it's not just a number that has no bearing on the size it will be drawn at in the 2-arg version of drawImage()…

[samizdatco]

I feel like this has probably come about as far along as it can given the current in-progress state of SVG in rust-skia. The main things on my todo list for when the next upstream release comes along are:

• [x] find a way to support dynamically loaded fonts in addition to the system font library
• [ ] use the viewbox aspect ratio to calculate a proportional width for un-sized SVG files
• [ ] see what else the upcoming Resource Providers api allows for

For now though, it's exciting to finally have a way to render SVGs. Thanks for your ingenuity in getting it all working and your help thinking through all the design-decisions (controversial and non) along the way 🎉

[mpaperno]

Ah, so it looks like Chrome doesn't just return a default 150×150 square: it returns 150 for the height but it uses the viewbox aspect ratio to scale the width to some proportion of 150.

Hmm, indeed, the viewbox has an effect. Guess we really need that info. It actually gets somewhat complicated especially when the preserveAspectRatio attribute also gets involved. Which Chrome seems to ignore unless it's set to none.

The Skia SVG renderer itself already pays attention to the viewbox properties and the aspect ratio setting. I think if we can set the initial containerSize to be proportional to the viewbox ratio, it should take care of the rest automagically.

But I agree this is probably as far as it's worth going for now (w/out hacking Skia bindings, parsing some of the SVG ourselves, or using a different SVG loader/parser).

Glad you found it helpful!