h2non / bimg

Go package for fast high-level image processing powered by libvips C library
https://pkg.go.dev/github.com/h2non/bimg?tab=doc
MIT License
2.65k stars 337 forks source link

Memory leak in Process(), Convert() #440

Open rvishrm opened 1 year ago

rvishrm commented 1 year ago

Hi, the following code is a simplified version of my code. Here, I'm trying to do some image processing on the image buffer.

package main

import  "github.com/h2non/bimg"

func resizeAndConvertImage(image []byte, height int, width int) ([]byte, error) {
        size, err := bimg.NewImage(image).Size()
        if err != nil {
            return nil, err
        }

        if width == 0 && height == 0 {
            width = size.Width
            height = size.Height
        }

        options := bimg.Options{
            Width:  width,
            Height: height,
        }
        imgType := bimg.JPEG
        image, _ = bimg.NewImage(image).Process(options)
        image, _ = bimg.NewImage(image).Convert(imgType)
        return image, err
}

func main() {
    newImage, err := resizeAndConvertImage(imageBuffer, height, width)
}

I used a script for hitting so many requests to the server.

I started the script at 11:31 and stopped it at 11:38. It can be clearly seen that around 2GB of memory is stucked somewhere.

Screenshot 2023-01-23 at 12 50 55 PM

Then I tried commenting Process() and Convert() functions one by one.

//  image, _ = bimg.NewImage(image).Process(options)
//  image, _ = bimg.NewImage(image).Convert(imgType)

And graph showed around 1GB of memory leakage. (script start time is 11:53)

Screenshot 2023-01-23 at 12 41 56 PM

Then I commented both the function calls.

//  image, _ = bimg.NewImage(image).Process(options)
//  image, _ = bimg.NewImage(image).Convert(imgType)

And the memory graph shows that there is no memory leakage. Memory used is just in MBs.

Screenshot 2023-01-23 at 12 46 28 PM

Am I missing something obvious here?

I have also tried following options for the fix

   os.Setenv("GODEBUG", "madvdontneed=1")
    bimg.VipsCacheSetMax(0)
    bimg.VipsCacheSetMaxMem(0)
    os.Setenv("MALLOC_ARENA_MAX", "2")
piyushRazorpay955 commented 1 year ago

Facing the same issue in process and convert function.

fcuenca4 commented 1 year ago

have you tried setting MALLOC_ARENA_MAX=2 ?

DundieWinner commented 7 months ago

Any solution for this?