Silence-GitHub / BBMetalImage

A high performance Swift library for GPU-accelerated image/video processing based on Metal.
MIT License
988 stars 126 forks source link

BBMetalTransformFilter is NOT working as expected #89

Closed fazal-e-majid closed 3 years ago

fazal-e-majid commented 3 years ago

First, I would like to thank @Silence-GitHub for creating this library. The library is working really good.

I'm using BBMetalTransformFilter in a project. but it's not very smooth and takes huge amount of memory that the app get crashed if multiple gestures read at once. I'm reading user's gestures which is applied to BBMetalView. Pan, Pinch and Rotation gestures. What happens is when only a single gesture is allowed to be read it doesn't work smoothly i.e either goes out of the frame or very slow. And when multiple gestures are allowed to be read the memory jumps up very much (more than 1.5GB) that it crashes the app. here is my transformation code :

@objc func handlePan(_ gesture: UIPanGestureRecognizer) {
        let translation = gesture.translation(in: bbMetalView)

        var transform = transformFilter.transform
        transform = transform.translatedBy(x: translation.x, y: translation.y)

        transformFilter.transform = transform
        imageSource.transmitTexture()
    }

    @objc func handlePinch(_ gesture: UIPinchGestureRecognizer) {
        var transform = transformFilter.transform
        transform = transform.scaledBy(x: gesture.scale, y: gesture.scale)

        transformFilter.transform = transform
        imageSource.transmitTexture()
    }

    @objc func handleRotation(_ gesture: UIRotationGestureRecognizer) {
            var transform = transformFilter.transform
            transform = transform.rotated(by: gesture.rotation * .pi / 180)

            transformFilter.transform = transform
            imageSource.transmitTexture()
    }

Any help will be appreciated. Thank You!

Silence-GitHub commented 3 years ago

Reset gesture after updating transform. For pan

gesture.setTranslation(.zero, in: gesture.view)

For pinch

gesture.scale = 1

For rotation, maybe you should calculate manually.

Set transform filter fitSize false. If it is true, the filter will consume a lot of memory when the image scales large.

transformFilter.fitSize = false
Silence-GitHub commented 3 years ago

To remain the same center while pinching, please try version 1.1.11 and set BBMetalTransformFilter anchorPoint (0.5, 0.5).

fazal-e-majid commented 3 years ago

Thank You very much for your response @Silence-GitHub . The memory problem largely got solved. it is not going to 2GB anymore but still when I first touch the image it goes from 115MB to 409MB right away. though it doesn't increase after that. One more thing I would like to point out and that is if I rotate the image at some rotation angle and then translate, it's not translating in the right direction. I guess the transform matrix is not multiplied in correct order. Thank you @Silence-GitHub for the support you are providing. I really appreciate it.