Allow arbitrary "sourceUrl" for cache checking when processing SVG buffers

[stephenjbell]

A little followup on #40 ...

I've been able to get SVGs turned into PNG files for OG images, which is great.

Here's my new issue: As far as I can tell, Sharp doesn't seem to have any way of loading an external image into an SVG (https://github.com/lovell/sharp/issues/2519 - "sharp does not provide any networking features, you'll have to write your own code to download the data and provide it as a Buffer."

When I try to load an image into my SVG string via an absolute URL (see code below), it just doesn't show up in the output image `<image id="_Image3" width="202px" height="202px" xlink:href="/Users/stephenjbell/Documents/Projects/steedgood/src/forestry-img/poolportrait.jpg" />`

The easiest solution seems to be turning the image into a datauri (through https://www.npmjs.com/package/datauri), and then dropping the datauri into the SVG string.

const datauri = require("datauri");

let photoDataUri = "";

try {
  photoDataUri = await datauri(authorPhoto);
} catch (error) {
   console.log(error); // Leave photo blank if we can't load the file
}

let svgStr = `<svg><image id="_Image3" width="202px" height="202px" xlink:href="${photoDataUri}" /></svg>`;

// Then run svgStr through eleventy-img

I /think/ when you made changes based on #40 , you used the entire SVG string to check if there was a cached image or not.

This is a great idea, except that each time I want to check if there's a cached version of my PNG file, I need to generate a datauri from the external image, which is /slow/ for performance, especially if I use my socialImage shortcode several times in the same page (i.e. dropping the image url into the page for both Twitter and Facebook meta):

<meta property="og:image" content="{% socialImage title, author.data.title, author.data.feature_image %}">
<meta name="twitter:image" content="{% socialImage title, author.data.title, author.data.feature_image %}" />

So, I think it might make sense to add the ability to use a different string instead of sourceUrl for checking if the cached image is available? It's possible I'm not understanding correctly how things are currently set up.

[zachleat]

Ah, I’m not sure if I get what you’re saying. This isn’t a valid URL path so it won’t take this branch: https://github.com/11ty/eleventy-img/blob/ecec599f0b592f0c084762f431594ef9cfb5f332/img.js#L347

Are you feeding it a buffer and you’re saying that this line is slow? https://github.com/11ty/eleventy-img/blob/ecec599f0b592f0c084762f431594ef9cfb5f332/img.js#L355

[zachleat]

Related to #70

[stephenjbell]

"Are you feeding it a buffer and you’re saying that this line is slow?" Kind of?

How generating an OG image using an image in an SVG works currently:

1. Get the article title and author photo.
2. Turn the author photo into a dataURI.
3. Add title and dataURI into my SVG string.
4. Pass the SVG string into Image for processing.
5. Cache -- Check if the image has already been generated, using the full SVG string
6. eleventy-img creates a PNG file. (very processor intensive)
7. Done

I’m trying to figure out a way of doing this, moving the cache checking earlier in the process:

1. Get the article title and author photo.
2. Cache -- Check if the image has already been generated, using just the title and photo URL
3. Turn the author photo into a dataURI (very processor intensive)
4. Add title and dataURI in my SVG string.
5. Pass the SVG string into Image for processing.
6. eleventy-img creates a PNG file (very processor intensive)
7. Done

The image caching is designed to prevent the processor-intensive work of converting the SVG into an image (which is nice), but when you're also loading an image into the SVG, it's not avoiding the processor-intensive work of converting an image into a datauri and creating the SVG string.

I thought that if there were a way to check the cache against a different string than sourceUrl (being the full SVG), there might be a way to avoid steps 2-4 in the first group above.

If this is outside the scope of what you feel like doing with this project, it's not a big deal. Just trying to figure out how to optimize the OG creation process a little.

[zachleat]

Ah, hmmmm—appreciate the extra information.

Yeah, that’s a tough one—we use the image contents to make sure that the image hasn’t changed. If the source input changes we know to generate a new image.

I could see us adding a way to override the getHash callback, but that approach is risky—you could very easily get into the situation where your input image changes and it skips generating any output images. I’d consider that scenario a worst-case for a cache and contributes to folks deleting their caches whenever they encounter a problem with their code.