Silence-GitHub / BBMetalImage

A high performance Swift library for GPU-accelerated image/video processing based on Metal.
MIT License
988 stars 126 forks source link

Add multiple consumers at once #54

Closed mnfro closed 4 years ago

mnfro commented 4 years ago

Is there the possibility to add to the BBMetalCamera few consumers at once (maybe as an array) instead of chaining multiple .add(consumer:) function?

    public func add(consumers: [BBMetalImageConsumer]) {
        for consumer in consumers {
            lock.wait()
            _consumers.append(consumer)
            lock.signal()
            consumer.add(source: self)
        }
    }

I wrote this function, but apparently it does not work. I mean, if I init the BBMetalCamera in viewDidLoad() as follows:

let exposureFilter = BBMetalExposureFilter(exposure: -2.0)
let lutFilter = BBMetalLookupFilter(lookupTable: UIImage(named: "test_lookup.png")!.bb_metalTexture!)

bbMetalCamera.add(consumers: [lutFilter, exposureFilter, metalView])

print(bbMetalCamera.consumers) will show that the filters and the metalView are loaded as consumers, but the live view is not updated with the filters. But maybe this is due to the intrisic chain nature of the stack. I mean, each consumer added becomes the source of the next one.

What's more, as far as I understood, once a consumer is loaded in the chain, its intensity cannot be modified (for example the exposure value in a BBMetalExposureFilter), unless of generating again the whole chain with the updated value. Am I right?

mnfro commented 4 years ago

I ended up with the following code:

    func addMultipleFilters(filters: [BBMetalBaseFilter]) {
        let first = bbMetalCamera.add(consumer: filters[0])
        var index = 0
        var lastFilter = first
        while index < filters.count-1 {
            index += 1
            let new = lastFilter.add(consumer: filters[index])
            lastFilter = new
        }
        lastFilter.add(consumer: metalView)
    }

which basically transforms the visual input represented by the array [Filter1, Filter2, Filter3, ...] in the chained representation:

metalCamera
-- filter1
---- filter2
------ filter(i)
-------- metalView
Silence-GitHub commented 4 years ago

About adding multiple consumers

Your code addMultipleFilters(filters:) is right. We can add multiple consumers in a function. Maybe I will add this feature.

About modifying filter intensity

After a filter is loaded in the chain, we can still modify the filter intensity (for example the exposure value of BBMetalExposureFilter). The modified effect works on the new texture after modification. For example, we can slide a slider to modify the exposure value. If we are processing camera or video, we can see the modified effect immediately since camera or video transmits texture one by one. If we are processing image, we need to call transmitTexture function to transmit a new texture to see the modified effect.

var imageView: UIImageView!
var imageSource: BBMetalStaticImageSource!
var filter: BBMetalExposureFilter!

override func viewDidLoad() {
    ...

    imageView = ...

    let slider = UISlider(frame: ...)
    slider.minimumValue = -3
    slider.maximumValue = 3
    slider.value = 0
    slider.addTarget(self, action: #selector(slide(_:)), for: .valueChanged)
    view.addSubview(slider)

    imageSource = BBMetalStaticImageSource(image: image)
    filter = BBMetalExposureFilter(exposure: 0)
    imageSource.add(consumer: filter)
        .addCompletedHandler { [weak self] _ in
            guard let self = self  else { return }
            DispatchQueue.main.async {
                self.imageView.image = self.filter.outputTexture?.bb_image
            }
    }
}

@objc private func slide(_ slider: UISlider) {
    filter.exposure = slider.value
    imageSource.transmitTexture()
}
Silence-GitHub commented 4 years ago

Now I think it is not necessary to add this feature to the library. Because the filter chain may be complex, and it is hard to design a function to set up a complex filter chain. So, design a function for your project. Your addMultipleFilters(filters:) function works.

mnfro commented 4 years ago

Yes, I totally agree with you. Thanks for the help.