PNG images read/write time increases with sustained load.

[ajatprabha]

I'm trying to build an image processing service which will manipulate images on the fly based on query parameters in the URL of an image which is stored in an S3 bucket.

I load tested this with vegeta and for around 200rps and 120s duration, the average response time is between 1.8-2.7s for JPG images. But when I tested this on PNG images, the response time starts increasing linearly and I start to face timeout errors in vegeta. I tried to debug and found that reading the original image and writing the processed image to the disk is taking time and it is increasing with subsequent requests. I'm not sure why is this happening only with PNG images and not JPG images. The only processes I'm applying right now are crop, resize, grayscale and compress.

PS: The disk is an SSD and I could've done the processing in memory but when such large requests are to be handled, memory is used up very quickly, so using the disk gave much better results here for sustained loads.

[justinfx]

Which version of ImageMagick are you running? Is it the same problem at #163, which was fixed by using >= 6.9.9?

[ajatprabha]

I'm using ImageMagick v7.0.5

I used an object pool for wands and I'm not creating and destroying them on every request. I just do a mw.Clear() before reusing a wand again.

Also, the slowdown is specific to PNG images only in my case.

[justinfx]

If that's the case then it's going to be ImageMagick and not the bindings. I would raise this on an ImageMagick forum

[justinfx]

Also, is the slowdown still there when you create a new wand for every png conversion? It would be ideal to have the simplest reproduction possible.

[ajatprabha]

Yes, it also happens when I create a new wand for every request.

[ajatprabha]

Can you please reproduce this on your local machine where, for every request, you create a wand, read a byte array into it with ReadImageBlob, do a CropImage, ScaleImage, TransformImageColorspace(imagick.COLORSPACE_GRAY) and then write this data to http.ResponseWriter with GetImageBlob. Repeat this for around 120s at 200rps, you should see response times going beyond 30 seconds.

[justinfx]

I will try if I get the time. But like I said, if it only happens with png then it is going to be an issue with ImageMagick and not the bindings. There is no png specific code in imagick. So even if I reproduce it, I can't fix it. You would want to present this repo to ImageMagick project.

On Sat, Jun 30, 2018, 9:48 PM Ajat Prabha notifications@github.com wrote:

Can you please reproduce this on your local machine where, for every request, you create a wand, read a byte array into it with ReadImageBlob, do a CropImage, ResizeImage, TransformImageColorspace(imagick.COLORSPACE_GRAY) and then write this data to http.ResponseWriter with GetImageBlob. Repeat this for around 120s at 200rps, you should see response times going beyond 30 seconds.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/gographics/imagick/issues/166#issuecomment-401530482, or mute the thread https://github.com/notifications/unsubscribe-auth/AARPxoVmBW29WYntuZ3xQXflQiSEA8rvks5uB0legaJpZM4U9IEm .

[ajatprabha]

@justinfx I looked deeper into this and found that mw.ReadImageBlob() or mw.WriteImage() are taking the most time, and the hundredth iteration is significantly slower than the tenth. Also, if I restrict the size of PNG image to <= 100x100, the process doesn't slow down. Perhaps the resource that is responsible for writing images is limited and not able to keep up with the load. Not sure exactly, but does doing imagick.Initialize() only once is the issue here? I read in the documentation that it should be initialized only once but then do different MagickWands get their own resources or some of them are shared? Can you suggest a possible way to get around this? I'll try that.

[justinfx]

You had said the slow down is specific to png, as opposed to say, jpg. That implies that the problem lives somewhere in the libpng delegate in ImageMagick. It is a documented workflow to only initialize ImageMagic once and to terminate it once when the ImageMagick resources are no longer needed. If you try to do it dynamically you will end up crashing your program when you try to access ImageMagick during or after it is terminating. If this were a problem specific to Imagick bindings, there might be something we could do about it. But if you don't have this problem with formats other than png, it highly suggests it is outside our scope.