Performance stats

[eksoverzero]

Hey,

Just thought I'd share some test results with you quick.

• Server is an instance on AWS
• Source image is huge (10800 × 5400, 14,5 MB) http://upload.wikimedia.org/wikipedia/commons/3/3d/LARGE_elevation.jpg

In case you get bored to death before you get to the bottom of this, somewhat useless piece of information, thanks.

So, I'm here...

... the server is here...

...and the src is here:

Using Siege I started a siege doing a 'resize=640' and then a few minutes later asked someone else to hit a new one, 'resize=1080', and then a few minutes later again, another person using 'resize=720'

I took 2 screenshots and a top -c output copy/paste from the server. Once as the second siege got started and then again while all 3 were running.

• I was running 4 Unicorn workers using the code from my release brach here: https://github.com/eksoverzero/magickly/tree/release
• There does seem to be an issue with S3 or something. I have no idea why, between an EC2 instance and a S3 bucket it would be so slow.
• The 'Longest transaction' times aren't bad considering the image it's processing?

siege -d10 -c100 http://**********.co.za/\?src\=http://upload.wikimedia.org/wikipedia/commons/3/3d/LARGE_elevation.jpg\&resize\=640

Transactions:              13209 hits
Availability:              99.91 %
Elapsed time:             867.17 secs
Data transferred:         635.99 MB
Response time:              1.49 secs
Transaction rate:          15.23 trans/sec
Throughput:             0.73 MB/sec
Concurrency:               22.76
Successful transactions:       13209
Failed transactions:              12
Longest transaction:           29.20
Shortest transaction:           0.91

siege -d10 -c100 http://**********.co.za/\?src\=http://upload.wikimedia.org/wikipedia/commons/3/3d/LARGE_elevation.jpg\&resize\=1080

Transactions:              11821 hits
Availability:              99.90 %
Elapsed time:             902.52 secs
Data transferred:        1567.70 MB
Response time:              2.53 secs
Transaction rate:          13.10 trans/sec
Throughput:             1.74 MB/sec
Concurrency:               33.12
Successful transactions:       11821
Failed transactions:              12
Longest transaction:           30.21
Shortest transaction:           2.00

siege -d10 -c100 http://**********.co.za/\?src\=http://upload.wikimedia.org/wikipedia/commons/3/3d/LARGE_elevation.jpg\&resize\=720

Transactions:               7272 hits
Availability:              99.85 %
Elapsed time:             495.33 secs
Data transferred:         438.29 MB
Response time:              1.66 secs
Transaction rate:          14.68 trans/sec
Throughput:             0.88 MB/sec
Concurrency:               24.31
Successful transactions:        7272
Failed transactions:              11
Longest transaction:           29.15
Shortest transaction:           1.13

Here are the app and monitoring stats:

top - 14:10:33 up 10 days,  1:04,  1 user,  load average: 0.04, 0.24, 0.20
Tasks:  81 total,   1 running,  80 sleeping,   0 stopped,   0 zombie
Cpu(s):  2.9%us,  1.5%sy,  0.0%ni, 94.9%id,  0.0%wa,  0.0%hi,  0.7%si,  0.0%st
Mem:   7629456k total,  5436484k used,  2192972k free,   210020k buffers
Swap:        0k total,        0k used,        0k free,  4415104k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                         
 7810 *******   20   0  395m  96m 4176 S    2  1.3   0:13.78 unicorn worker[1] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 8203 www-data  20   0 64132 3232 1020 S    2  0.0   0:03.08 nginx: worker process                                                                                                                                                            
 7813 *******   20   0  396m  97m 4176 S    2  1.3   0:13.49 unicorn worker[2] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 7816 *******   20   0  407m  94m 4184 S    2  1.3   0:13.75 unicorn worker[3] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 7807 *******   20   0  396m  97m 4176 S    1  1.3   0:14.04 unicorn worker[0] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 5275 mongodb   20   0 6781m  85m  17m S    1  1.1  24:16.05 /usr/bin/mongod --config /etc/mongodb.conf       

top - 14:19:57 up 10 days,  1:14,  1 user,  load average: 0.49, 0.39, 0.32
Tasks:  81 total,   1 running,  80 sleeping,   0 stopped,   0 zombie
Cpu(s):  3.2%us,  2.7%sy,  0.0%ni, 92.8%id,  0.0%wa,  0.0%hi,  1.2%si,  0.2%st
Mem:   7629456k total,  5314188k used,  2315268k free,   210036k buffers
Swap:        0k total,        0k used,        0k free,  4425396k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                         
10884 *******   20   0  289m  64m 3968 S    3  0.9   0:02.08 unicorn worker[2] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 8204 www-data  20   0 64296 3304  932 S    3  0.0   0:06.52 nginx: worker process                                                                                                                                                            
10878 *******   20   0  289m  64m 3968 S    2  0.9   0:02.61 unicorn worker[0] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
10860 *******   20   0  289m  64m 3968 S    1  0.9   0:02.92 unicorn worker[3] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
10846 *******   20   0  289m  64m 3988 S    1  0.9   0:04.05 unicorn worker[1] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 8203 www-data  20   0 63904 3004 1024 S    1  0.0   0:07.71 nginx: worker process           

[eksoverzero]

Here is the start of some normal staging operations. A solid mix of cached vs uncached requests:

Just incase it was question, this is my response time to the magicaly server:

64 bytes from 54.72.87.102: icmp_seq=0 ttl=47 time=222.495 ms
64 bytes from 54.72.87.102: icmp_seq=1 ttl=47 time=227.093 ms
64 bytes from 54.72.87.102: icmp_seq=2 ttl=47 time=330.777 ms
64 bytes from 54.72.87.102: icmp_seq=3 ttl=47 time=236.994 ms
64 bytes from 54.72.87.102: icmp_seq=4 ttl=47 time=281.330 ms
64 bytes from 54.72.87.102: icmp_seq=5 ttl=47 time=223.149 ms
64 bytes from 54.72.87.102: icmp_seq=6 ttl=47 time=269.614 ms
64 bytes from 54.72.87.102: icmp_seq=7 ttl=47 time=223.872 ms
64 bytes from 54.72.87.102: icmp_seq=8 ttl=47 time=311.896 ms

[afeld]

Wow, nice work! Any takeaways for you? Nothing here jumps out as me as a glaring problem or something Magickly can do much about, since it is CPU-bound at the ImageMagick level. I will comb through the info above again in a day or two, though.

Resizing is a fairly simple operation – the big performance bottlenecks happen when multiple/complex effects are applied. https://github.com/afeld/magickly/issues/16 will help greatly.