Create small versions of images during build for use in thumbnails

[narthur]

### Desiderata
- [x] Clarify how this gissue relates to #348
- [x] Add thumbnail generation to the build process?
- [x] See notes from Dan Byler, possibly make new gissues?
- [ ] Fix the regression: generate thumbnails again

Currently the new version of the blog does nothing to optimize images. We should bake out smaller versions of blog images during build so we don't serve huge images for list pages, Beeminder homepage, etc.

HT adamwolf (and many users)

Cognata

• 348

Verbata: images, bandwidth, blog loading speed, page load times, clear build cache and deploy, render.com, render,

[narthur]

Astro has an experimental feature for handling this:

https://docs.astro.build/en/guides/assets/

Hopefully that's all we'll need.

[narthur]

Testing what URLs GitHub uses to serve images.

[narthur]

This is done.

[narthur]

Vet's PR adds git-level image compression. The change I made for this PR adds build-level image size variant generation. So the two features should work together fine to provide faster images.

[dreeves]

Notes from Dan Byler that I'm just now adding:

I’m working at the park today, so I’m tethering … and I noticed the images on your blog are loading kind of slowly. If you care about snappy, you could speed that up by optimizing them pre-upload [*], hosting in a generic CDN, or a combination of the above. It looks like Astro has some integrations to that effect: https://astro.build/integrations?search=images

[*] I downloaded a random png image from the blog, threw it into ImageOptim, and got a visually identical image that was ~80% smaller. From my TOOLS- Image compression, hosting.md file, here are some image optimization tools I’ve made note of over the years:

Local options:

• ImageOptim
• Realmac's Squash
• Optimage

Web based:

• compressor.io
• ShortPixel - has APIs, Zapier integrations, and more

Hosting options with dynamic transformations:

• Cloudinary (used this at AFG for years; no complaints)
• ImageKit
  • Now has Google/Amazon-based image tagging and background removal
  • Sample transformations: https://ik.imagekit.io/h7kn17twvu/tr:ot-overlay%20made%20easy,ots-45,otp-20,otbg-FFFFFF/default-image.jpg https://ik.imagekit.io/h7kn17twvu/tr:oi-Lightbulb_-pink_kd_GZpFQy0F.png,ox-45,oy-300/default-image.jpg https://ik.imagekit.io/h7kn17twvu/tr:oi-Lightbulb_-pink_kd_GZpFQy0F.png,ox-45,oy-300:tr:oi-Money_UzoNNXI63.png,ox-100,oy-5

[narthur]

@dreeves As far as future work goes on image optimization it depends on how important build speed is to you. Currently the blog is generating smaller versions of image at build time, which slows down builds. If we switched to something like ImageKit (or cloudflare--I just got an email from them saying they're providing a similar service now), it could speed up our builds significantly, at the complexity cost of integrating with a third-party service.

[narthur]

I guess we could also consider finding a way to include generated thumbnails in git using a github action, similar to strickvl's PR.

[narthur]

@dreeves When did Dan give you that note? I'm curious if it was before or after I deployed my build-level image resizing.

[dreeves]

When did Dan give you that note? I'm curious if it was before or after I deployed my build-level image resizing.

Aug 24, 2023, 7:21 AM pacific

No strong opinions from me on build speed vs image optimization tradeoff. I guess reoptimizing every single image from the whole history of the blog on every build feels wasteful. Sometimes after publishing a post I keep spotting small errors and do build after build after build. But "feels wasteful" can be different from "actually costly".

[narthur]

@dreeves Given I don't think we're paying Render.com anything for the blog (?), definitely wouldn't think our current approach is costly.

[dreeves]

Good point. The wait when making tweaks for the new build to be live is kinda painful, but normally it's worth the build taking longer. Ooh, idea: can we have two kinds of deploys/builds? Draft mode and prod mode? Like if I'm making tweaks or fussing with css or something i want to minimize build time. Other times i want to do a full build with all the image optimization and stuff. I don't know if this makes sense or is even possible but maybe there's a way to have the best of both worlds somehow?

PS: meta: leaving this open until we're sure we have the image optimization stuff covered in other gissues and since we're still discussing ideas for best of both worlds with build time vs optimizing what's built.

[narthur]

@dreeves Hmm. Not easily possible if you're working directly on the master branch. If you were using separate branches, we could think about changing the settings for preview builds to make them speedier.

[dreeves]

What about something in posts.json for site-wide settings? Like {"mode": "prod"} vs {"mode": "draft"} or booleans for whether to do things like optimize images. Or what if there was an explicit list of which posts to build, with a special option for "build all of them"? Then when working on a particular post you could change it to just rebuild that one post.

(Maybe that last idea doesn't make sense -- it would have to put the prebuilt ones somewhere to somehow retrieve them without building them I guess.)

[narthur]

@dreeves Yup, definitely want to avoid introducing options that change build behavior for production builds.

[narthur]

@dreeves With merge of #412 Astro's image cache is now preserved between builds, which should effectively eliminate the image optimization time during most builds.

[dreeves]

How do we bust the cache when an image changes? Or is that automatic?

[narthur]

@dreeves If an image changes that is at the same url it was previously, you'll need to go into Render.com and re-run the deploy without the cache.

[dreeves]

Maybe that won't come up often. But what do you mean specifically? Is that a special Render thing, "deploy without cache"?

[narthur]

@dreeves Yup! And definitely let me know if this happens more than I thought it would and it becomes annoying.

[bsoule]

I noticed that the lastest several blog posts don't have a thumbnail version of the post image. Did something break with generating those? Or like, does it only make thumbnails when you do the "clear build cache & deploy" thing or something?

Also, out of curiosity, is there a way to find the thumbnail version directly? [context: for the beeminder front page, where we show the latest blog posts, i have a scraper that looks at the blog for new posts. right now we use the large image for the latest, and then get the url for the thumbnail version when the next post is published, but we don't ever actually need the large size image, since we're showing little thumbnails on the beeminder frontpage, so if there was a way to get the thumb more directly in the scraper, that'd be cool].