bdkjones / CodeKit

CodeKit 3 Issue Tracker
https://codekitapp.com
82 stars 5 forks source link

[Feature Request] Intelligent Creation & Optimisation of Images Based on URL Parameters #665

Open iocouto opened 3 years ago

iocouto commented 3 years ago

Having CodeKit automatically convert images to WebP is fantastic - hopefully we'll see conversion to AVIF, soon, too! - #628

But optimising images for the web these days means not just converting them into 'modern' formats, but also providing them in a multitude of responsive sizes and orientations, to help browsers only download the smallest size & resolution needed for the visitor's current device. So, a tag that used to look like this...:

<img src="assets/img/beach.jpg" alt="tropical beach sunset">

...is now likely to look like this...:

<img src="assets/img/beach.jpg"
     srcset="assets/img/beach.jpg 300px,
             assets/img/beach@2.jpg 600px,
             assets/img/beach@3.jpg 900px" 
     sizes="(min-width: 640px) 300px, 100%" 
     alt="tropical beach sunset">

...or even like this - if you're a serious developer trying to optimise your page loading speed and get a good ranking in GTMetrix or PageSpeed:

<picture>
    <source type="image/avif" 
            srcset="assets/img/beach.avif 300px,
                    assets/img/beach@2.avif 600px,
                    assets/img/beach@3.avif 900px"
             sizes="(min-width: 640px) 300px, 100%" >
    <source type="image/webp" 
            srcset="assets/img/beach.webp 300px,
                    assets/img/beach@2.webp 600px,
                    assets/img/beach@3.webp 900px"
             sizes="(min-width: 640px) 300px, 100%" >
    <img src="assets/img/beach.jpg"
         srcset="assets/img/beach.jpg 300px,
                 assets/img/beach@2.jpg 600px,
                 assets/img/beach@3.jpg 900px" 
         sizes="(min-width: 640px) 300px, 100%" 
         alt="tropical beach sunset">
</picture>

That means, that for each image used on the page, the developer very often needs to create multiple copies: not just in different formats, but also in different sizes. In our simple example above, the developer would have to create 9 different images. When we throw in "art direction" - i.e., when the Designer specifies that in different screen sizes or orientations we must use different images (usually images cropped differently, or resized to a different aspect ratio), then we can end up having to generate well over a dozen images...

In order to help the developer out, some pre- and post-processing tools have started to incorporate intelligent image processing: the tool essentially looks for image URLs in HTML, CSS and JS files, which use certain special query parameters. The tool uses those parameters to generate the output images required by the developer. For example, if the image URL has a ?w=100 query parameter, it means we want our output image to be 100px wide. Similarly, we could have parameters of h= (height), preset= (one of CodeKit's quality presets), crop= (to specify the cropping 'centre', eg., 'top-left'), and as= (to specify output format, i.e., 'webp', 'avif', etc.). Of course, multiple queries could be combined to generate exactly the image needed.

Here is a real-world example: the client supplies us with an image to be used in the project. The image is a 5000x4000 pixel JPEG file titled 'beach.jpg'. Using our simple example above, it would be amazing if CodeKit could auto-generate the responsive images needed for the layout, by reading the query parameters in the image URLs, like this:

<img src="assets/img/beach.jpg?w=300"
     srcset="assets/img/beach.jpg?w=300 300px,
             assets/img/beach.jpg?w=600 600px,
             assets/img/beach.jpg?w=900 900px" 
     sizes="(min-width: 640px) 300px, 100%" 
     alt="tropical beach sunset">

For our more complex example, the URLs would look like this:

<picture>
    <source type="image/avif" 
            srcset="assets/img/beach.jpg?w=300&as=avif 300px,
                    assets/img/beach.jpg?w=600&as=avif 600px,
                    assets/img/beach.jpg?w=900&as=avif 900px"
             sizes="(min-width: 640px) 300px, 100%" >
    <source type="image/webp" 
            srcset="assets/img/beach.jpg?w=300&as=webp 300px,
                    assets/img/beach.jpg?w=600&as=webp 600px,
                    assets/img/beach.jpg?w=900&as=webp 900px"
             sizes="(min-width: 640px) 300px, 100%" >
    <img src="assets/img/beach.jpg"
         srcset="assets/img/beach.jpg?w=300 300px,
                 assets/img/beach.jpg?w=600 600px,
                 assets/img/beach.jpg?w=900 900px" 
         sizes="(min-width: 640px) 300px, 100%" 
         alt="tropical beach sunset">
</picture>

When either w or h is specified alone, the image should be resized proportionately. If both are specified together, then the image is resized first, then cropped, if needed. In that case, it would be wonderful to also be able to specify a 'cropping focus': a position in the image that the cropping would try to maintain as 'center'. The 9 possibilities (like in CSS) would be top, bottom, left, right, centre, top-left, top-right, bottom-left, bottom-right:

<img src="assets/img/beach.jpg?w=300&h=300&crop=bottom"
     srcset="assets/img/beach.jpg?w=300&h=300&crop=bottom 300px,
             assets/img/beach.jpg?w=600&h=600&crop=bottom 600px,
             assets/img/beach.jpg?w=900&h=600&crop=bottom 900px" 
     sizes="(min-width: 640px) 300px, 100%" 
     alt="tropical beach sunset">

This would mean that from one single image in the source folder, CodeKit would be able to auto-generate EVERY variant that is needed by the layout - without the developer having to spend hours manually cropping and resizing images in a third-party app...

bdkjones commented 3 years ago

Thanks for the detailed writeup! This is certainly something I’d consider, but it would be a lot of work.

On 15 Sep 2021, at 23:47, Igor Couto @.***> wrote:

Having CodeKit automatically convert images to WebP is fantastic - hopefully we'll see conversion to AVIF, soon, too! - #628 https://github.com/bdkjones/CodeKit/issues/628 But optimising images for the web these days means not just converting them into 'modern' formats, but also providing them in a multitude of responsive sizes and orientations, to help browsers only download the smallest size & resolution needed for the visitor's current device. So, a tag that used to look like this...:

tropical beach sunset

...is now likely to look like this...:

<img src="assets/img/beach.jpg" srcset="assets/img/beach.jpg 300px, @. 600px, @. 900px" sizes="(min-width: 640px) 300px, 100%" alt="tropical beach sunset"> ...or even like this - if you're a serious developer trying to optimise your page loading speed:

tropical beach sunset

That means, that for each image used on the page, the developer very often needs to create multiple copies: not just in different formats, but also in different sizes. In our simple example above, the developer would have to create 9 different images. When we throw in "art direction" - i.e., when the Designer specifies that in different screen sizes or orientations we must use different images (usually images cropped differently, or resized to a different aspect ratio), then we can end up having to generate over a dozen images...

In order to help the developer out, some pre- and post-processing tools have started to incorporate intelligent image processing https://v2.parceljs.org/recipes/image/: the tool looks through image references in HTML, CSS and JS files. If the image uses certain special URL query parameters, then the tool uses those parameters to generate images as required for the developer. If the image URL has a ?w=100 query in the URL, it means we want the processed image to be 100px wide. Similarly, we could have queries with h= (height), preset= (one of CodeKit's quality presets), crop= (to specify the cropping 'centre', eg., 'top-left'), and as= (to specify output format, i.e., 'webp', 'avif', etc.). Of course, multiple queries can be combined to generate exactly the image needed.

Here is a real-world example: the client supplies an image file to be used in the project. The image is a 5000x4000 pixel JPEG file titled 'beach.jpg'. Using our simple example above, it would be amazing if CodeKit could auto-generate the images needed, by reading the queries in the image URLs, like this:

<img src="assets/img/beach.jpg?w=300" srcset="assets/img/beach.jpg?w=300 300px, assets/img/beach.jpg?w=600 600px, assets/img/beach.jpg?w=900 900px" sizes="(min-width: 640px) 300px, 100%" alt="tropical beach sunset"> For our more complex example, the URLs would look like this:

tropical beach sunset

When either w or h is specified alone, the image should be resized proportionately. If both are specified together, then the image is resized first, then cropped. In that case, it would be wonderful to also be able to specify a 'cropping focus': a position in the image that the cropping would try to crop around. The 9 possibilities (like is CSS) would be top, bottom, left, right, centre, top-left, top-right, bottom-left, bottom-right:

<img src="assets/img/beach.jpg?w=300&h=300&crop=bottom" srcset="assets/img/beach.jpg?w=300&h=300&crop=bottom 300px, assets/img/beach.jpg?w=600&h=600&crop=bottom 600px, assets/img/beach.jpg?w=900&h=600&crop=bottom 900px" sizes="(min-width: 640px) 300px, 100%" alt="tropical beach sunset"> This would mean that from one single image in the source folder, CodeKit would be able to generate ALL variants that are needed by the layout - without the developer having to spend hours in a third-party image manipulation app...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bdkjones/CodeKit/issues/665, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHCGZSY3A5EJ2B7BUE3MP3UCGHJ7ANCNFSM5EEA2ZBA.

iocouto commented 3 years ago

@bdkjones it may not be as complicated to implement as it might seem at first glance. The way that Parcel does it is roughly like this:

  1. check the file extension, and if the file is the right type, add it to the parser queue (Parcel seems to parse only .html, .css and .js files, but we could extend it and parse .md, .less, .styl, .pug, etc., if we wanted to)
  2. parse the files in the queue, using a simple regex to find image URLs/paths with the required query parameters - and add those to the 'render queue'.
  3. To render the required images, for each image in the render queue we extract the path/location (so it can be output to the equivalent location in the Build folder), and the parameters, from the query string.
  4. produce the output image as specified by the required query parameters, in the required location within the Build folder

The key is having a parser that is fast and smart enough, to make it easy for us to extract the information we need from the files and image URLs/paths, without too much fiddling.

You could then also list the image files being parsed and generated in the CodeKit interface, in a similar way that you now list imported JS/CSS files — eg., clicking on an image file would enable us to see which html files is 'linked' to, and which images it is being used to generate in the Build folder.