Reducing memory usage when processing images

[micahcantor]

I'm using this library in a project of mine, and I realized that resizing was using an incredible amount of RAM, much more than I would have expected to process one image. I decided to run a memory profiler to confirm my suspicions. I tested this image, rescaling from from 2048x1536 to 1920x1080. These were the results [pdf] I got after running caire with pprof. Specifically, here was the code I used:

func resize(in io.Reader, out io.Writer) {
    defer profile.Start(profile.MemProfile).Stop()
    p := &caire.Processor {
        NewWidth:  1920,
        NewHeight: 1080,
        Scale: true,
    }

    err := p.Process(in, out)
    check(err)
}

As you can see, caire used 3.4 GB of RAM to process this one image. I decided to investigate further on what it was using this memory for, and focused on the reduce() function, which is the main function which is called repeatedly when resizing an image:

reduce := func() {
    width, height := img.Bounds().Max.X, img.Bounds().Max.Y
    c = NewCarver(width, height)
    c.ComputeSeams(img, p)
    seams := c.FindLowestEnergySeams()
    img = c.RemoveSeam(img, seams, p.Debug)
    imgs = append(imgs, img)
}

The program appends each iteration it creates to an imgs slice, which from my understanding is only used when the input is a gif, since that slice will be used to reconstruct the frames of the gif. Since I was processing a jpg, I removed the line and ran the program through a profiler again, and here were the results [pdf]. Removing this one line reduces the memory usage from 3.4 GB to only 232 MB.

I tested the change on another image and found a similar reuction, from [1.8 GB] to [231 MB]. More testing could be done, but I would propose adding another parameter to the Processor struct, like 'IsGIF', then only appending the images to the slice if this is given as true, such as

if (p.IsGIF) {
    imgs = append(imgs, img)
}

[esimov]

Thanks for letting me know, I will investigate it.

[esimov]

@micahcantor thank you again for notifying me about this issue. I've fixed the huge memory allocation problem taking a different approach then what you have suggested because on the gif generation we should have faced the same issue and I would have been forced to include another flag which shouldn't be the case. Now the memory allocation dropped significantly. Please test it on your side too.

[micahcantor]

@esimov Looks good to me. I tested it on some images and I'm hitting a similar mark at around 230 mb of memory usage. My fix was more of a bandaid for my use case, which would never involve GIFs, so your solution is definitely better.

[esimov]

Closing this ticket.