imazen / imageflow

High-performance image manipulation for web servers. Includes imageflow_server, imageflow_tool, and libimageflow
https://docs.imageflow.io/
GNU Affero General Public License v3.0
4.19k stars 139 forks source link

Wish: Improve anchor parameter #594

Open iJungleboy opened 2 years ago

iJungleboy commented 2 years ago

At the moment the url allows for ?anchor=topcenter to make sure cropping behaves as we need it to when the image is auto-cropped.

But we would like to give our users much more control - but automatically. The idea is that they could mark an area on the image which is important, and the cropper would use this to limit what must stay inside the result, and what can be cropped.

Something like ?anchor=bottomright&focus=22-67,50-90

The core idea is that the content-editor could select a rectangle on the image determining what's really important - and therefor must stay in the image no matter how we crop it. The example above would have been in %, so the important bits of the above example would be:

  1. Within the width from 22% to 67%
  2. Within the height of 50% to 90%

Based on this, and also because bottom-right is the anchor, the image would try to crop ensuring that this area will be visible, and expand to top-left from the 67%/90% corner.

Of course it would need to do various corrections if the image doesn't fit.

We believe this would be super-useful for many CMS scenarios, and would also help write the code if you see this is a useful addition.

Note: I noticed this is somehow similar to what the gravity in the JSON configuration would do. Maybe it would also be enough to allow a gravity-parameter in the url-syntax?

lilith commented 2 years ago

This is good; I had something similar in the Faces plugin and I would like to implement this feature as you described.

On Mon, Feb 28, 2022, 7:09 AM iJungleboy @.***> wrote:

At the moment the url allows for ?anchor=topcenter to make sure cropping behaves as we need it to when the image is auto-cropped.

But we would like to give our users much more control - but automatically. The idea is that they could mark an area on the image which is important, and the cropper would use this to limit what must stay inside the result, and what can be cropped.

Something like ?anchor=bottomright&focus=22-67,50-90

The core idea is that the content-editor could select a rectangle on the image determining what's really important - and therefor must stay in the image no matter how we crop it. The example above would have been in %, so the important bits of the above example would be:

  1. Within the width from 22% to 67%
  2. Within the height of 50% to 90%

Based on this, and also because bottom-right is the anchor, the image would try to crop ensuring that this area will be visible, and expand to top-left from the 67%/90% corner.

Of course it would need to do various corrections if the image doesn't fit.

We believe this would be super-useful for many CMS scenarios, and would also help write the code if you see this is a useful addition.

โ€” Reply to this email directly, view it on GitHub https://github.com/imazen/imageflow/issues/594, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2LHZJWODSM3RBDX3RDO3U5N6YFANCNFSM5PRGDMQA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

iJungleboy commented 2 years ago

I love you even more ๐Ÿ˜˜

iJungleboy commented 2 years ago

BTW: there has been no activity on this for almost 9 months. Is there a code freeze or something? Any chance such a feature would be implemented any time soon?

lilith commented 2 years ago

No, no code freeze, just a series of unfortunate events. It's still high on the list

iJungleboy commented 2 years ago

Thanks! Anything we could do to accellerate this?

lilith commented 2 years ago

This overlaps with https://github.com/imazen/imageflow/issues/602

iJungleboy commented 10 months ago

ping ? ๐Ÿ˜‰

lilith commented 10 months ago

Unfortunately this is on the Rust side.

I'm currently bogged down in a massive Imageflow Server refactor that will enable a lot of new scenarios (and be usable from .NET 4.8 as well as .NET Standard 2, .NET 6-8, but it also heavily architected. It creates new nuget packages Imazen.Routing and Imazen.Abstractions, with the intent to support serverless/one-shot/AOT mode as well as long-running server processes that can apply a lot more heuristics and optimizations. It introduces a ton of new interfaces and establishes a mini-framework for blobs with optimized routing, resource dependency discovery for smart cache keys, etags, smart buffering, async cache writes, memory caching, blob caching, Pipelines support, cache invalidation, search, audit trails, and purging, better non-image blob handling, intuive route->provider mapping, and can support future real-time resilience features like circuit breaker, multi-cache choice, and possibly offloading jobs to other servers in the cluster or serverless functions. It will autoconfigure s3 and azure containers for maximum throughout and automatic eviction of expired unused entries.

This also lays the foundation for AI salience detection and storage of metadata like this anchor value, so that it could automatically happen if a salience backend/service and cache service are registered.

It's a lot, and I could use some code review when I wrap up the first preview. I think it would be usable directly in your packages that are alternatives to ImageResizer. I could probably do this side quest for anchor support once I have a preview shipped and reviewed.

On Wed, Jan 10, 2024, 1:14โ€ฏAM iJungleboy @.***> wrote:

ping ? ๐Ÿ˜‰

โ€” Reply to this email directly, view it on GitHub https://github.com/imazen/imageflow/issues/594#issuecomment-1884373453, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2LH44CBWA565BY44NB3TYNZEVHAVCNFSM5PRGDMQKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBYGQZTOMZUGUZQ . You are receiving this because you commented.Message ID: @.***>

lilith commented 10 months ago

I think this feature and the logic it should follow needs to be worked through. A single gravity point is straightforward, but a rectangle has implications such as - do you ever zoom? If impossible aspect ratio wise, do you crop through the preserved zone or do you introduce padding? What color/transparent should padding be depending on the image type?

Art direction is a twin concern, because screen size affects how much background we want to show.

If we specify a preservation rect, and also gravity, should gravity be relative to that rect or the image?

How should fractional percentages be expressed? 0-1.0 or 0-100.0?

What if there are two key regions on opposite sides of the landscape image, but we have to sacrifice one due to a portrait mode aspect being requested? What are the failure fallback behaviors, and how do we allow them to be expressed?

How should those correlate with a salience function's output?

On Wed, Jan 10, 2024, 2:08โ€ฏAM Lilith River @.***> wrote:

Unfortunately this is on the Rust side.

I'm currently bogged down in a massive Imageflow Server refactor that will enable a lot of new scenarios (and be usable from .NET 4.8 as well as .NET Standard 2, .NET 6-8, but it also heavily architected. It creates new nuget packages Imazen.Routing and Imazen.Abstractions, with the intent to support serverless/one-shot/AOT mode as well as long-running server processes that can apply a lot more heuristics and optimizations. It introduces a ton of new interfaces and establishes a mini-framework for blobs with optimized routing, resource dependency discovery for smart cache keys, etags, smart buffering, async cache writes, memory caching, blob caching, Pipelines support, cache invalidation, search, audit trails, and purging, better non-image blob handling, intuive route->provider mapping, and can support future real-time resilience features like circuit breaker, multi-cache choice, and possibly offloading jobs to other servers in the cluster or serverless functions. It will autoconfigure s3 and azure containers for maximum throughout and automatic eviction of expired unused entries.

This also lays the foundation for AI salience detection and storage of metadata like this anchor value, so that it could automatically happen if a salience backend/service and cache service are registered.

It's a lot, and I could use some code review when I wrap up the first preview. I think it would be usable directly in your packages that are alternatives to ImageResizer. I could probably do this side quest for anchor support once I have a preview shipped and reviewed.

On Wed, Jan 10, 2024, 1:14โ€ฏAM iJungleboy @.***> wrote:

ping ? ๐Ÿ˜‰

โ€” Reply to this email directly, view it on GitHub https://github.com/imazen/imageflow/issues/594#issuecomment-1884373453, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2LH44CBWA565BY44NB3TYNZEVHAVCNFSM5PRGDMQKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBYGQZTOMZUGUZQ . You are receiving this because you commented.Message ID: @.***>

iJungleboy commented 10 months ago

So 1. what you're doing sounds insane! and awesome ๐Ÿ’ƒ.

Would love to do some code review, but not sure if I'm qualified enough ๐Ÿ˜‰ - sounds pretty hard core.

The direction etc. is an issue. I remember explaining my expected gravity behavior here https://github.com/imazen/imageflow/issues/602 that may help a bit.

I'm pretty sure gravity would have to be based on the selected area.

Percent probably in 33.3 format.

lilith commented 3 months ago

I've sketched out my idea of the interactions and details - can you all request access and leave comments/edits? This is certainly complex to handle in a way that interacts well with all other features.

https://docs.google.com/spreadsheets/d/1RDWcTka6MfMXsd5tJES4O7UYFLn4sKw7EgFj9Hd97hc/edit?usp=sharing