Unable to save image (Not a PNG file) on concurrent requests

[anantadwi13]

Hi, I have an issue when using this library for high concurrent requests. When I apply a watermark with multiple concurrent requests, It will return an error png2vips: unable to read source source or even vips2png: unable to write to target. I don't know whether the problem occurs in this library or libvips.

I have created a repository to reproduce this error. Perhaps you can help me to investigate it. Thank you.

Environment:

• ubuntu 18.04
• 8 core CPUs
• libvips v8.12.2
• bimg v1.1.6

How to reproduce:

• Clone https://github.com/anantadwi13/test-bimg
• Build the image, docker build -t test-bimg .
• Execute the program, docker run --rm -it test-bimg (with concurrent request 4 times of CPU cores)

• It will return an error.

  Example

  ``` # docker run --rm -it test-bimg process is starting queueing #0 done epoch #0 req #24 queueing #1 done epoch #0 req #18 done epoch #0 req #16 done epoch #0 req #19 done epoch #0 req #13 queueing #2 done epoch #0 req #23 done epoch #0 req #27 done epoch #0 req #29 done epoch #0 req # 0 done epoch #0 req #31 done epoch #0 req #28 done epoch #0 req # 7 done epoch #0 req #15 done epoch #0 req # 2 done epoch #0 req #20 done epoch #0 req # 9 done epoch #0 req #14 done epoch #0 req # 8 done epoch #0 req #11 done epoch #0 req # 3 done epoch #0 req #21 queueing #3 done epoch #0 req #12 done epoch #0 req #17 done epoch #0 req # 5 done epoch #0 req #25 done epoch #0 req #26 done epoch #0 req # 6 done epoch #0 req #30 done epoch #0 req #10 done epoch #0 req # 4 done epoch #0 req # 1 done epoch #0 req #22 queueing #4 (process:212): VIPS-WARNING **: 04:59:36.047: Not a PNG file done epoch #1 req # 7 panic: png2vips: unable to read source source goroutine 119 [running]: github.com/h2non/bimg.catchVipsError() /root/go/pkg/mod/github.com/anantadwi13/bimg@v0.0.0-20220405024938-d613e42da6fd/vips.go:785 +0x8c github.com/h2non/bimg.vipsSave(0x7fbdb830f4c0, {0x0, 0x4b, 0x6, 0x1, 0x0, 0x0, 0x0, 0x0, {0x0, ...}, ...}) /root/go/pkg/mod/github.com/anantadwi13/bimg@v0.0.0-20220405024938-d613e42da6fd/vips.go:534 +0x3ee github.com/h2non/bimg.saveImage(_, {0xfa0, 0xfa0, 0x0, 0x0, 0x0, 0x0, 0x4b, 0x6, 0x0, ...}) /root/go/pkg/mod/github.com/anantadwi13/bimg@v0.0.0-20220405024938-d613e42da6fd/resizer.go:193 +0x158 github.com/h2non/bimg.resizer({_, _, _}, {0xfa0, 0xfa0, 0x0, 0x0, 0x0, 0x0, 0x4b, ...}) /root/go/pkg/mod/github.com/anantadwi13/bimg@v0.0.0-20220405024938-d613e42da6fd/resizer.go:145 +0xaf8 github.com/h2non/bimg.Resize({_, _, _}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}) /root/go/pkg/mod/github.com/anantadwi13/bimg@v0.0.0-20220405024938-d613e42da6fd/resize.go:15 +0xfc github.com/h2non/bimg.(*Image).Process(_, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}) /root/go/pkg/mod/github.com/anantadwi13/bimg@v0.0.0-20220405024938-d613e42da6fd/image.go:203 +0x65 main.main.func1(0x0?, 0x0?, {0xc000180000, 0x109e11, 0x0?}, {0xc000132000, 0x4b0, 0x0?}) /root/test-bimg/main.go:54 +0x2a7 created by main.main /root/test-bimg/main.go:41 +0x272 goroutine 119 [running]: main.main.func1(0x0?, 0x0?, {0xc000180000, 0x109e11, 0x0?}, {0xc000132000, 0x4b0, 0x0?}) /root/test-bimg/main.go:63 +0x2f8 created by main.main /root/test-bimg/main.go:41 +0x272 exit status 2 ```

It will not return an error if I'm using one request or as large as the CPU core size at a time.

Example

``` # docker run --rm -it -e CONCURRENT_REQ=8 test-bimg process is starting queueing #0 done epoch #0 req # 1 done epoch #0 req # 7 done epoch #0 req # 2 done epoch #0 req # 0 done epoch #0 req # 6 done epoch #0 req # 4 done epoch #0 req # 5 done epoch #0 req # 3 queueing #1 done epoch #1 req # 5 done epoch #1 req # 0 done epoch #1 req # 7 done epoch #1 req # 6 done epoch #1 req # 1 done epoch #1 req # 4 done epoch #1 req # 3 done epoch #1 req # 2 queueing #2 done epoch #2 req # 1 done epoch #2 req # 4 done epoch #2 req # 5 done epoch #2 req # 3 done epoch #2 req # 6 done epoch #2 req # 7 done epoch #2 req # 2 done epoch #2 req # 0 queueing #3 done epoch #3 req # 3 done epoch #3 req # 0 done epoch #3 req # 4 done epoch #3 req # 7 done epoch #3 req # 1 done epoch #3 req # 6 done epoch #3 req # 5 done epoch #3 req # 2 queueing #4 done epoch #4 req # 5 done epoch #4 req # 6 done epoch #4 req # 0 done epoch #4 req # 2 done epoch #4 req # 4 done epoch #4 req # 1 done epoch #4 req # 7 done epoch #4 req # 3 queueing #5 done epoch #5 req # 3 done epoch #5 req # 1 done epoch #5 req # 0 done epoch #5 req # 4 done epoch #5 req # 5 done epoch #5 req # 6 done epoch #5 req # 2 done epoch #5 req # 7 queueing #6 done epoch #6 req # 1 done epoch #6 req # 0 done epoch #6 req # 4 done epoch #6 req # 3 done epoch #6 req # 6 done epoch #6 req # 5 done epoch #6 req # 7 done epoch #6 req # 2 queueing #7 done epoch #7 req # 3 done epoch #7 req # 5 done epoch #7 req # 0 done epoch #7 req # 6 done epoch #7 req # 1 done epoch #7 req # 4 done epoch #7 req # 2 done epoch #7 req # 7 queueing #8 done epoch #8 req # 1 done epoch #8 req # 6 done epoch #8 req # 5 done epoch #8 req # 4 done epoch #8 req # 3 done epoch #8 req # 0 done epoch #8 req # 2 done epoch #8 req # 7 queueing #9 done epoch #9 req # 6 done epoch #9 req # 2 done epoch #9 req # 0 done epoch #9 req # 3 done epoch #9 req # 1 done epoch #9 req # 5 done epoch #9 req # 4 done epoch #9 req # 7 waiting for all requests process is done ```

[anantadwi13]

Update:

It will be ok if I'm using workers https://github.com/anantadwi13/test-bimg/tree/with-worker

git clone https://github.com/anantadwi13/test-bimg.git
cd test-bimg
git checkout with-worker

docker build -t test-bimg .
docker run --rm -it -e WORKER=8 test-bimg