davidbyttow / govips

A lightning fast image processing and resizing library for Go
MIT License
1.25k stars 196 forks source link

How to release memory after completing a task of converting PNG to AVIF #358

Closed ficapy closed 1 year ago

ficapy commented 1 year ago

I have written a long-running script that performs PNG to AVIF conversions during periods of low system load. However, I noticed that the memory used is not released after the task is stopped, with memory usage up to 500MB. Can anyone advise how to manually release the memory?

package main

import (
    "fmt"
    "github.com/davidbyttow/govips/v2/vips"
    "os"
    "time"
)

func main() {
    vips.Startup(nil)
    defer vips.Shutdown()

    vipsImage, _ := vips.NewImageFromFile("demo.png")

    imgbuf, _, _ := vipsImage.ExportAvif(&vips.AvifExportParams{
        Quality:       20,
        StripMetadata: false,
        Lossless:      false,
    })
    os.WriteFile("demo.avif", imgbuf, 0644)
    vipsImage.Close()
    fmt.Println("Done")
    time.Sleep(1 * time.Hour)
}
tonimelisma commented 1 year ago

Considering your Close()ing the image properly, that's an issue with Golang memory management. There's nothing we can do in the govips library side to help you, unfortunately.

I suggest you check the golang garbage collector guide: https://tip.golang.org/doc/gc-guide

Here's another useful resource: https://concertio.com/blog/optimizing-the-go-garbage-collector-and-concurrency/

I seem to recall the latest golang versions provided some new control over the gc, in addition to the env variables mentioned in the above posts. I personally handled this by doing heavy image processing in separate goroutines, which are spawned and killed for each task as part of a pool.