Feat: Dynamic Image Resizing

[nelsonic]

At present we have single size image resizing: 7. Resizing/compressing files This is a very good starting point, 👍 but it sets and artificial constraint that ends up looking "meh" on most devices.

Story

As a person using multiple devices to view the App - that features image content - I want images to conform to the device/screen size So that they always look their best.

As noted in https://github.com/dwyl/imgup/issues/91 our friend has created https://github.com/jupiter/rust-image-worker which appears to be well-documented and tested and runs on Cloudflare with CDN caching. 🏎️ While it currently does not have support for certain formats https://github.com/jupiter/rust-image-worker/issues/3 But the underlying library does support GIF and Webp: https://docs.rs/image/latest/image/enum.ImageFormat.html#variant.Gif

Todo

• [ ] Follow the deployment instructions in the repo: https://github.com/jupiter/rust-image-worker#how-to-use
  • [ ] Create a new file to document our setup/deployment in _even _more detail: dynamic-resizing.md
• [ ] Bonus Level: Create Pull Request extending the image support: https://github.com/jupiter/rust-image-worker/issues/3#issuecomment-1635583690

[ndrean]

@nelsonic I am curious. ,You must probably have read this article. Since (obviously) you are using a server to upload image, what is the reason you did not consider to run your own Task to compress the image before saving to S3? You would save a bucket and a lambda. Something obvious I am missing?

[nelsonic]

@ndrean hadn't seen that fly.io blog post published Mar 13, 2023; thanks for sharing.

It's quite superficial, not really a "tutorial", more of a "you can do this thing" but you have figure out the precise details on your own ... 🙃 That's exactly the kind of post that mega frustrates me.

To answer your question: we want to maintain the original image so that people can view as much detail as possible. We will save money when we switch to using B2 https://github.com/dwyl/imgup/issues/98 I don't have anything against optimising the images in a Task in Elixir. But I would prefer to minimise the Elixir bottleneck and use an existing Rust project to the just-in-time optimisation. From an experience perspective the Elixir approach where you perform the immediate resizing could be better in terms of response time. ⏳ We could test it. For now the S3 buckets + Lambda function works "OK". Have you implemented image compression/resizing in Elixir? how fast was it?

Ultimately using B2 + Cloudflare which is "free" for their CDN would save us money (over S3 + Lambda)

[ndrean]

@nelsonic, Yes superficial indeed, but it gives ideas.

But aren't you serving the client back with a compressed image? https://user-images.githubusercontent.com/17494745/242736344-bd61d716-8a4e-445f-a643-8f5d13a00510.png

that's why I imagined it was an extra step (and I always fear AWS costs....).

I forked the repo but did not run it yet but I plan to as I never did this server-side nor loaded files yet with Phoenix. But I have no intention so far to send something to S3, too much bills with AWS.

I maybe totally wrong but I thought a simple URL.createObjectURL for the preview was enough, although I have no idea if Liveview accepts this.

I will also try the https://hexdocs.pm/image/Image.html and explore

[nelsonic]

Yeah, the current AWS bill is currently Zero because it's all in the "Free Tier". Which is why the priority on #98 is priority-2 Returning the compressed image URL is a stop-gap and will be phased out in due course instead the device will request the size of image that best fits the viewport #91 therefore prospectively resizing becomes wasteful. If the device uploading the image is 600px wide there's no need to have images 200px or 2000px wide. Then when subsequent devices attempt to view the image, if they can request dimensions that best suit their viewport without a significant latency penalty, that's super desirable.

Very curious to see your exploration of the https://github.com/elixir-image/image library. 👌 I didn't find the docs to be particularly beginner-friendly when I tried to read them a few months ago ... 🤷‍♂️

I'm especially not a huge fan of risk of crashing the whole BEAM with one malformed image: https://hexdocs.pm/image/readme.html#security-considerations

If I can offload image processing/resizing to a totally separate Rust micro-app I'm going to chose that every time. 🚀 But if I needed to do Image recognition in Elixir then the image library looks interesting. 💭

[ndrean]

@nelsonic I am exploring 2 ideas. If this does not interest you, just tell me (nicely 😄). I send the image from the browser (not from PHoenix) to a (tiny) companion Node.js server that runs sharp,. The server returns a resized image based on the device. You then PUT directly to S3 via an API Gateway, no round trip to the PHoenix server, and the bucket is closed, except this endpoint that exposes him, of course.

1) The Node.js server (30 LOC) runs independently next to the Phoenix app. It includes sharp. You define an endpoint to which the Phoenix app can POST the image as well as the desired dimensions you collect from the device (eg window.innerHeight * 0.5). You resize the image accordingly. On completion, it will respond back. I am fighting with how to display a preview of the result back.

2) It seems that sending the image to Phoenix is consuming resources but you don't perform anything on the image. You can send the image directly to S3 via an API Gateway, no round trip. The whole thing takes a few LOC (including the "CIDing of the filename and saving the metadata of the file to the Phoenix database). I am fighting with a CORS problem though. If you are interested, I can append some code here.

[nelsonic]

@ndrean these are all perfectly valid suggestions and we have done something similar in the past.

our goal with doing this in Phoenix is to minimise latency. In the best case scenario API Gateway + Lambda is about 300ms round trip. Much worse if the Lambda function has to “cold” boot 🥶 Direct Upload to S3 works and we’ve done that before too.

Perhaps the most important thing we aren’t yet doing in the imgup app is logging metadata and person info so that the people making the uploads can see all their images easily. in other words, try to think of where an image uploading service with privacy focus can go rather than just looking at what we currently already have. to be clear: I agree that we don’t need Phoenix. Heck we could achieve what we currently have with a single PHP file/script copy-pasted from StackOverflow or ChatGPT … But when it comes to a roadmap that heavily features images, having a clear grasp of the stack is mega important.

[ndrean]

@nelsonic Of course, I do not pretend any originality at all. Just that I found the Phoenix code difficult. When you say "300ms round trip", you mean between upload and the display of the URL and preview of their pics?

Perhaps the most important thing we aren’t yet doing in the imgup app is logging metadata and person info so that the people making the uploads can see all their images easily.

I probably misunderstood. You mean a profile to be able to join his URLs and meta, thus retrieve?

[nelsonic]

Yeah, for now this repo is “just” a simple way to upload images. If we had time we would build out the rest of the features. 💭

[ndrean]

I imagine you want to use the "standard" mix .phx.gen.auth and a join table of S3 URLs. The tricky part is probably to handle safely thousands of downloads S3 -> client.

[nelsonic]

Indeed. The “standard”, though woefully incomplete, mix phx.gen.auth is what we are using as the basis for our auth re-write which is ongoing … 👌 Then imgup will have one-tap auth and images will be associated with a person_id so people can easily see & share the images they’ve uploaded. ✅

[ndrean]

You probably know https://imagekit.io/ ? I found interesting the way they did it, and rather inline with your roadmap,

Watch https://www.youtube.com/watch?v=sWcSYG1eifo

however, the price seems high.

• the idea of having a default S3 bucket offered, and then using your own storage, db or your own S3 bucket
• the idea of having a separate transformation process, and using the url for specific transforms.

[ndrean]

@nelsonic I believe I have a use case for a modal in the following situation: once you uploaded files, you have a list or miniatures previews of them. You can put a modal for each file to 1) link to a new page 2) a form to enter a name and download the file. What do you think?

[nelsonic]

Respectfully, I disagree. 🙅 Unless you need to hijack the person's attention for very specific reason, Modals are never the answer to "How do we make excellent UX here?" If you've ever watched a senior citizen use the web they are always confused by them.

Interacting with a bank of images needs to be as "flat" as possible. Inserting a new image into the DOM and applying a highlight (border) around it is enough to show the recently uploaded file. Opening the larger version of that file should be full-screen with a gentle transition and a "back button" to allow them to return to the list.

Apologies if I come across as "harsh" (bordering on militant) but Modals are evil and should be avoided at all costs. 😉

[ndrean]

I used Vix to compress/resize (Vix.Vips.Image and Vix.Vips.Operation). It is easy and super fast.. You can do this in the clientless flow in the handle_progress

A JPEG 5472x3648 of 2.5MB

compressed to 988kB (Q=30):

resized 10%: to 547x365 (43kB):

and a thumb of 5kB with the same ratio 1.5

[ndrean]

A guide: https://imagekit.io/blog/how-to-resize-image-in-html/

[ndrean]

@nelsonic

Let me know what you think of this? it is only SSR because Phoenix transforms the images.

When it reads a picture, I transform it into WEBP format to minimise the size. Only WEBP fomat are saved in S3.

You can let the browser resize the pic for you, or minimise the data over the wire and resize it. Resize to what? To the uploading device for example?

With this 100% Phoenix strategy, we could run a job to compute the resizing for different pre-defined devices, upload this to S3. You will have several versions of the same pic in S3. Then - upon some matching - when you want to display a picture, you "fetch" the right format depending upon your device.

To keep it simple - and given I don't want to pay for - , if you upload a pic from a device, the image will be resized to the dimension of the device meaning that the S3 upload is adapted to your device. I do not keep the original format, just because this is a demo, but it is possible.

https://up-image.fly.dev

This kind of app - with the jungle of the async messaging I used - could benefit of the new assign_async coming in LiveView 0.20: see https://www.youtube.com/watch?v=FADQAnq0RpA

I tried to use it when you upload several files. Each upload will trigger a "writter" and then you invoke a start_async and handle_async. This works for one upload. Indeed, each task is identified by its own unique key. If you upload several files, this will be triggered several times and invoke the same key for this async task, so this will fail. In such case, a "traditionnal" Task with the handle_info handlers.