kean / Nuke

Image loading system
https://kean.blog/nuke
MIT License
8.16k stars 529 forks source link

Fascinating: caching a "related calculated value" such as dominant color #303

Closed smhk closed 4 years ago

smhk commented 5 years ago

Here's an interesting thing, I load many images form the net:

Nuke.loadImage(with:data... , options: ILO, into: self)

Fantastic. Eacgh image is loaded many places in the app or on the same screen. No problem for Nuke.

Often these days you want a "dominant" color for use in shadows or borders. (Good online example: lokeshdhakar.com/projects/color-thief/ )

For this you have CIAreaAverage or perhaps CIAreaHistogram. Importantly, CIKMeans is coming soon, too, which will be the go-to choice.

For each of these images, once the image is downloaded (by Nuke), in many places I run some processing:

    if let aac = image?.areaAverageColor {
        shadowLayerColor = aac
    }

(The code for areaAverageColor is below.)

Would there be some way in the Nuke pipeline, that

or the like)

Does this already exist in Nuke or would it be a good idea?

Perhaps the ideas of picking out a built-in color should be built-right-in (much as say blur is immediately available).

Nuke == awesome


    var areaAverageColor: UIColor? {
        guard let inputImage = CIImage(image: self) else { return nil }

        let extentVector = CIVector(
            x: inputImage.extent.origin.x,
            y: inputImage.extent.origin.y,
            z: inputImage.extent.size.width,
            w: inputImage.extent.size.height)

        guard let filter = CIFilter(
            name: "CIAreaAverage",
            parameters: [
                kCIInputImageKey: inputImage,
                kCIInputExtentKey: extentVector]
        ) else { return nil }

        guard let outputImage = filter.outputImage else { return nil }

        var bitmap = [UInt8](repeating: 0, count: 4)
        let context = CIContext(options: [ .workingColorSpace: NSNull() ])
        // consider also , .outputColorSpace: NSNull()

        context.render(
            outputImage,
            toBitmap: &bitmap,
            rowBytes: 4,
            bounds: CGRect(x: 0, y: 0, width: 1, height: 1),
            format: .RGBA8,
            colorSpace: nil)

        return UIColor(red: CGFloat(bitmap[0]) / 255, green: CGFloat(bitmap[1]) / 255, blue: CGFloat(bitmap[2]) / 255, alpha: CGFloat(bitmap[3]) / 255)
    }
kean commented 5 years ago

Hi, @smhk. I think you are onto something, I saw similar suggestions before.

There is already one built-in mechanism to do that which is ImageProcessing. I think what you could do is implement a custom processor (or use the anonymous one with a closure) to perform this operation and then save the result in a dictionary or something else. Then the processors can return the original image without modification (yes, not ideal). Keep in mind, this won't work with the recently added on-disk cache for processed images (isDataCachingForProcessedImagesEnabled option).

and I guess some sort of callback to deliver the calculated value

If you'd like this value to be calculated after you display an image, then I would suggest implementing it separately from Nuke. All you need is cache, which can be simply a dictionary, and a way to re-use tasks for calculating the average color.

As for adding areaAverageColor to Nuke, this might be another good candidate for addition. I'm not sure where to draw the line which ones should and should not be added. This seems like something that many apps need. But if I were using a framework like this, I would've never assumed it had this kind of feature. My first step would've been to look elsewhere. Nuke only has the most basic image processors and a way to create custom ones, as complex as the user needs.