Wish: Add support for focal point

[HRasch]

Hello and thanks for your efforts in this great library. Today i researched on some points in case of responsive images. One interesting feature for a resizing library would be the option to define a focal point. If an image gets resized, this point ensures that the "important" part of a picture is still included. I know there's an option "anchor" that goes in this direction, but support for a focal point would be much better. Here's some info from another Software that supports such an option:

https://docs.imgix.com/tutorials/focal-point-cropping

Thanks.

[lilith]

We call it "gravity" in the JSON API.

https://github.com/imazen/imageflow/blob/main/docs/src/json/constrain.md

I haven't implemented in the URL API yet, because I'm wondering if we want to support multiple focal points or a rectangle instead of just a point

On Sun, May 22, 2022, 1:49 PM Holger Rasch @.***> wrote:

Hello and thanks for your efforts in this great library. Today i researched on some points in case of responsive images. One interesting feature for a resizing library would be the option to define a focal point. If an image gets resized, this point ensures that the "important" part of a picture is still included. I know there's an option "anchor" that goes in this direction, but support for a focal point would be much better. Here's some info from another Software that supports such an option:

https://docs.imgix.com/tutorials/focal-point-cropping

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/imazen/imageflow/issues/602, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2LH75MUWQ75X45K6TWEDVLKFUJANCNFSM5WT2ZQ6A . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[HRasch]

My opinion: Support one point and an optional second point. Two points are the easiest way to span a rectangle if needed ;-)

[lilith]

If automatic crop is also specified in a command query, should padding be used to absolutely prevent cropping into the focus rectangle if too large? What do you think is most intuitive?

[HRasch]

I guess it would be more intuitive not to use padding and just crop the focus-area. The focus area is to mark the most important area of the picture. Maybe it‘s possible to „zoom out“ so the focal area keeps untouched while the requested ratio is still valid?

[lilith]

Well, the problem is that zooming out would add padding if the image isn't large enough, right? Ex, if you focused the whole image, then requested any different aspect ratio, there'd be nothing to show.

[HRasch]

I would expect to use the center of the focal area as reference and crop, even if this violates the focal area.

[lilith]

If so, then there's no reason for a 2nd focal point, right? Because it would be no different than simply providing the center of the desired rectangle?

[HRasch]

Yes, finally it would result in the same behavior. The difference would be that if you change the ratio, the focal area would be the center instead of the center of the image. To be honest, I currently solved this issue by using the object-fit together with object-position and css custom-properties. This finally result in larger images than necessary to be transfered. The typical use-case is that you have e.g. a ratio of 16:9 on desktop-size and a ratio of 4:3 on mobile phones. If the focus is not in the center of the image, the important part moves out of screen. If you define a focal area my expected behavior ( if the native resolution is e.g. 1600x1200, focal-area 200:200, 600:600) is that on desktop, where you need a width of 1600px and height of 900px, the center of this area is calculated (400:400), then the required image-area is calculated (0:-50, 1600:850), the area is moved to the origin (0:0, 1600:900). On mobile, where you need a ratio of 4:3, (width: 640px, height: 480px), the area would be 80;160, 720;640. If you have a ratio of 4:3 with a width: 320px, height: 240px, the result would be the area of 240:280, 560:520, what is cropping the focal area unnecessary. In this case the image should be resized to show the complete focal area (172:200, 628:600). Your edge-case, where the focal area is very large and you can‘t resize without exceeding the native image size, the image should be resized to the max and then cropped. Hope my explaination is not too confusing 🥴

[lilith]

There are cases where users want cropping to be prevented, like #594.

We also probably want to offer integration with other features, like face detection: https://github.com/imazen/imageflow-dotnet/issues/44

[HRasch]

I think, you select mode „crop“ to have an image, covering the image area. Adding padding suddenly will lead to more unexpected results than cropping the image if zooming out is not possible anymore because it exceeds the image size.

[iJungleboy]

Hi @lilith

Even though I'm a developer I actually end up using the resizer as an end user a lot. And I'm always surprised as how difficult certain simple looking scenarios end up being, so I'll try to summarize why one point is ok, but not enough...

Note that I like the https://docs.imgix.com/tutorials/focal-point-cropping explanations, because it does cover a lot of aspects. But again, not quite all. In my world, here's what usually should happen:

1. The designer configures the system to make images for headers very-wide , for normal content 16:9, for tall-side-pics maybe 1:2
2. The user adds content to all these predefined things and determines what's important
   It's important to note that the user is usually not qualified to change settings like compression, aspect ratio (would mess up the design) or stuff like that. So he/she can't actually change this.
3. Based on these two givens, the system should be able to do everything as magically as possible

For simplicity, let's assume we have these predefined formats

1. square
2. tall
3. wide
4. 16:9

Part 1: Point or Area depends on Min/Max control

Now IMHO the user should just say what must certainly be in the image, so that automatic cropping (almost) never takes too much away.

With this, I think the designer should be able to specify rules which try to

1. crop as much as possible
2. crop as little as possible

For example, a SQUARE image MINIMUM would still result in something like this:

Whereas SQUARE MAXIMUM would result in something like this:

The imgix tries to solve this using the zoom factor, but that only works if the user is optimizing the image for exactly one specific use case (so it's a great idea, but often wouldn't be enough).

So to summarize part 1

1. If you only want to crop as little as possible, and the formats are mostly similar to the original image size, then 1 point could work.
2. As soon as it should be more automated, we need an area (2 points / rectangle), and a parameter to minimize / maximize cropping

Part 2: More extremes

For wide banners the previous example would fail quickly. So there is either...

...A need for multiple priority areas where the user can determine different priority areas depending on the extremes (this should probably be handled at the CMS level, but could also be handled at the resizer level

which would then result in a banner like

I believe that Part 1 would best be solved at resizer level. Part 2 is probably more diverse in what can be necessary, so for now I think we shouldn't handle it yet, until it's really clear what the requirements are.

[HRasch]

If I understood it correctly, Lilith was talking about the case where it's not possible to keep the focal area together with the ratio like I would prefer to get the blue rectangle, even if it violates the focal area, instead of a padding

[iJungleboy]

@HRasch Basically what you're describing is certainly correct.

If the resulting bounds would be outside of the possible image, the system should automatically do "the right thing". What that is could be a url parameter but by default it should result in what you're describing.

[iJungleboy]

Any progress on this? @lilith

[wilz05]

@lilith any updates on this ?