add passing an URL and query string params to resize an image?

[ndrean]

I added today the following functionality to my toy fork: if a picture is served, you pass a GET request to the endpoint with the URL in the query string and get back a link to a resized WEBP picture from S3. I found some pictures that were served and wanted to use them.

I choose WEBP format to limit traffic on S3 and bandwidth usage on mobile (and canIUse is :ok).

To run in a Livebook:

map= %{url: <the url.png>, w: 900, h: 600})

URI.parse("https://up-image.fly.dev/api")
|> URI.append_query(URI.encode(map)
|> URI.to_string()
|> Finch.get() 
|> Finch.build(MyApp.Finch)

or

curl  -X GET  https://up-image.fly.dev/api?url=<the url.jpeg>&h=600&w=600

It will deliver a json reply with the URL of the new file in S3.

Let me know if you find some interest.

Endpoint: (normally works 😀) https://up-image.fly.dev/api

[nelsonic]

@ndrean nice. 👌

Tried: https://up-image.fly.dev/api?url=https://world-celebs.com/public/media/resize/800x-/2019/8/5/porter-robinson-3.jpg&h=600&w=600

{"h":600,"w":600,"url":"https://dwyl-imgup.s3.eu-west-3.amazonaws.com/6E70A71E.webp",
"h_origin":800,"init_size":56172,"w_origin":800,"new_size":20980}

https://dwyl-imgup.s3.eu-west-3.amazonaws.com/6E70A71E.webp

Each change in width/height results in a new image: https://up-image.fly.dev/api?url=https://world-celebs.com/public/media/resize/800x-/2019/8/5/porter-robinson-3.jpg&h=200&w=200

{"h":200,"w":200,"url":"https://dwyl-imgup.s3.eu-west-3.amazonaws.com/1BACCDFF.webp",
"h_origin":800,"init_size":55149,"w_origin":800,"new_size":3640}

https://dwyl-imgup.s3.eu-west-3.amazonaws.com/1BACCDFF.webp

This is a perfectly valid use case. Especially the use of .webp to optimise storage/bandwidth. ✅

As outlined in https://github.com/dwyl/imgup/issues/91#issuecomment-1635476057 I still prefer the idea of having a single version of an image in storage and then using URL params w=200 to request a smaller size and caching that request on the CDN for speed. i.e. having a distinct/different URL for each different size of image is not desirable for us. 🔗

[ndrean]

Thanks for your evaluation.

I have to better understand how you compute a resized image and transfer the resized image to the CDN, and then how do you call a resized image when used.

EDIT: I think that I can set up a DNS on Cloudfare, and keep the app hosted on Fly.io. Then instead of using S3, I can use R2. Then I think I can just add this domain as Cloudfare will cache the files. Now how to use R2 instead of S3. Probably client -> R2 must not be too difficult (?), but this would be only for the original image. Since I need to transform the images, I need client -> elixir-app -> R2, and able to interact with R2. The whole R2 API to explore.

[ndrean]

I added another functionality: a POST endpoint.

From a client - the browser -, you want to upload multiple files and you get a JSON response with a simple fetch request. Phoenix parser only accepts one file. See build in parsers. You don't have an allow_upload equivalent as in LiveView.

@nelsonic The code below might interest you as this is not standard. The "secret" is to build your own multipart parser that will effectively parse a FormData. The idea is to exchange the key used as the input for a file in the FormData (it will the same, namely the "name" attribute of the input) to a new indexed one that you create. That's it. The cool part is that you are "almsot" independant on how the front-end coded it, just a FormData (see the HTML example below, 2 lines of JS). I still request to use "w" if you want a specific resize.

defmodule Plug.Parsers.FD_MULTIPART do
  @multipart Plug.Parsers.MULTIPART

  def init(opts) do
    opts
  end

  def parse(conn, "multipart", subtype, headers, opts) do
    length = System.fetch_env!("UPLOAD_LIMIT") |> String.to_integer()
    opts = @multipart.init([length: length] ++ opts)
    @multipart.parse(conn, "multipart", subtype, headers, opts)
  end

  def parse(conn, _type, _subtype, _headers, _opts) do
    {:next, conn}
  end

  def multipart_to_params(parts, conn) do
    case filter_content_type(parts) do
      nil ->
        {:ok, %{}, conn}

      new_parts ->
        acc =
          for {name, _headers, body} <- Enum.reverse(new_parts),
              reduce: Plug.Conn.Query.decode_init() do
            acc -> Plug.Conn.Query.decode_each({name, body}, acc)
          end

        {:ok, Plug.Conn.Query.decode_done(acc, []), conn}
    end
  end

  def filter_content_type(parts) do
    filtered =
      parts
      |> Enum.filter(fn
        {_, [{"content-type", _}, {"content-disposition", _}], %Plug.Upload{}} = part ->
          part

        {_, [_], _} ->
          nil
      end)

    l = length(filtered)

    case l do
     # user pressed enter without any files => do nothing
      0 ->
        nil

      _ ->
       # get the none "content-type" inputs
        other = Enum.filter(parts, fn elt -> !Enum.member?(filtered, elt) end)

        # get the key used to name the files
        key = elem(hd(filtered), 0)
       # build a new list of keys
        new_keys = keys = Enum.map(1..l, fn i -> key <> "#{i}" end)

        # and exchange the "old" key to the new indexed one. The keys will be unique this way.
        f =
          Enum.zip_reduce([filtered, new_keys], [], fn elts, acc ->
            [{_, headers, content}, new_key] = elts
            [{new_key, headers, content} | acc]
          end)

       # rebuild the "parts"
        f ++ other
    end
  end
end

To use this beast, "just" add to your API pipeline:

#router
pipeline :api do
    plug :accepts, ["json"]

    plug CORSPlug,
      origin: ["*"]

    plug Plug.Parsers,
      parsers: [:urlencoded, :my_multipart, :json],
      pass: ["image/jpg", "image/png", "image/webp", "iamge/jpeg"],
      json_decoder: Jason,
      multipart_to_params: {Plug.Parsers.FD_MULTIPART, :multipart_to_params, []},
      body_reader: {Plug.Parsers.FD_MULTIPART, :read_body, []}
  end

  scope "/api", UpImgWeb do
    pipe_through :api
    get "/", ApiController, :create
    post "/", ApiController, :handle
  end

To test this quickly, it is easy: create from the code an "index.html" and serve it. I request to use "w" and a checkbox named "thumb" if you want a thumbnail (default is 100px).

<html>
  <body>
    <form
      id="f"
      action="https://up-image.fly.dev/api"
      method="POST"
      enctype="multipart/form-data"
    >
      <input type="file" name="file" multiple />
      <input type="number" name= "w"/>
      <input type="checkbox" name="thumb"/>
      <button form="f">Upload</button>
    </form>

    <script>
      const form = ({ method, action } = document.forms[0]);
      form.onsubmit = async (e) => {
        e.preventDefault();
        return fetch(action, { method, body: new FormData(form) })
          .then((r) => r.json())
          .catch(console.log);
      };
    </script>
  </body>
</html>

You will get a JSON response

All async process to S3, thanks to Elixir. It was a real pleasure to code this.