search

h2non / bimg

Go package for fast high-level image processing powered by libvips C library
https://pkg.go.dev/github.com/h2non/bimg?tab=doc
MIT License
2.65k stars 337 forks source link

Memory consumption issue when processing images #469

Open Kosadchiy opened 6 months ago

Kosadchiy commented 6 months ago

Hi,

Bug report I faced with a problem that application crashes due to high memory consuming trying to resize multiple images when spawning goroutines. A docker container exits with code 139. I spawn goroutine for each Image.Process method call. The problem does not reproduce in a single thread without goroutines. Also I noticed that it depends on how many cache memory seted via VipsCacheSetMaxMem. If the value is 0 application crashes almost instantly while if I set VipsCacheSetMaxMem(1000) it process several images before error.

Environment (please complete the following information)

Additional context Cant reproduce it on MacOS. The problem exists in Alpine or Ubuntu based docker containers.

Please let me know what other information I can provide.

Thanks!

luhuaei commented 5 months ago

I have the same issue. I doubt it is caused by CGO in goroutines memory isn't being freed.

Winens commented 4 months ago

Im not sure but, once i watched a video about what happends when goroutines are in low memory. It causes the app to crash. Maybe the issue here is not about CGO at all? How much memory did u even give? Looks like its not enough memory.

Winens commented 4 months ago

Also you said you give VipsCacheSetMaxMem, but did you give that much memory to the docker container?