[WIP] Embedded lua to generate dynamic request data

[samdfonseca]

Hi @stevej, sorry for dropping this on you without a heads up, but I'm really just interested in getting some feedback from you regarding the potential for merging this in at some point.

This PR gives the user the ability to specify a lua script that defines the function slow_cooker.generate_data, which is provided the method, url, host, and request id of each request and called to generate the request's body. I've used the awesome gopher-lua project to handle the communication between the go-side and lua-side.

I was interested in using slow_cooker as a replacement for jMeter since, like jMeter, it gives results as a time-series. However, the endpoint I needed to test validates a certain element on the request body for uniqueness. The changes I made allow me to have a single slow_cooker binary, and a series of lua scripts to generate the request body according to different scenarios. My previous approach was to modify the go source for each scenario and compile separate binaries. The overhead of the lua VM, while I'm sure it is non-zero, seems to be negligible, although I haven't tested this too extensively. Whatever overhead does exist will only be incurred when using the -dataGeneratorScript flag.

The changes I've made are far from final yet, and I haven't settled on certain decisions, such as if the lua-side should be able to modify any part of the request or just generate the body. My interest is mainly just to see if you could envision a version of this being merged in, since that will dictate how diverging some of the changes I make should be. Thanks!

→ time ./slow_cooker_without_lua -qps 100 -concurrency 10 -method POST -totalRequests 10000 http://httpbin.org/post
# sending 1000 POST req/s with concurrency=10 to http://httpbin.org/post ...
#                           good/b/f t   good%   min [p50 p95 p99  p999]  max change
2016-12-19T16:12:45-05:00   4009/0/0 10000  40% 10s  12 [ 18  30  52 2895 ] 2895 +
2016-12-19T16:12:55-05:00   4814/0/0 10000  48% 10s  12 [ 18  30  41  339 ]  339
2016-12-19T16:13:05-05:00   4822/0/0 10000  48% 10s  12 [ 19  31  49  154 ]  154
FROM    TO #REQUESTS
   0     2 0
   2     8 0
   8    32 13087
  32    64 499
  64   128 34
 128   256 11
 256   512 6
 512  1024 1
1024  4096 7
4096 16384 0

real    0m30.062s
user    0m1.915s
sys 0m1.551s

→ time ./slow_cooker_with_lua -qps 100 -concurrency 10 -dataGeneratorScript ./randomstring.lua -method POST -totalRequests 1000 http://httpbin.org/post
# sending 1000 POST req/s with concurrency=10 to http://httpbin.org/post ...
#                           good/b/f t   good%   min [p50 p95 p99  p999]  max change
2016-12-19T16:14:54-05:00   4795/0/0 10000  47% 10s  12 [ 19  30  41  232 ]  232 +
FROM    TO #REQUESTS
   0     2 0
   2     8 0
   8    32 4632
  32    64 152
  64   128 10
 128   256 1
 256   512 0
 512  1024 0
1024  4096 0
4096 16384 0

real    0m10.057s
user    0m1.306s
sys 0m0.577s

[stevej]

Oh wow! I will discuss this with the other engineers on my team and we'll figure out an answer to your questions.

[stevej]

If you run the release.sh script (it builds binaries for target platforms) can you tell me how large the resulting binaries are?

[samdfonseca]

6.0M Dec 19 18:27 slow_cooker_darwin
6.0M Dec 19 18:27 slow_cooker_linux_amd64
5.3M Dec 19 18:27 slow_cooker_linux_arm

Thanks for the quick reply, btw!

[samdfonseca]

gah sorry didn't realize the release script used the import path. changed it to github.com/samdfonseca/slow_cooker and got this

7.8M Dec 19 18:31 slow_cooker_darwin*
7.8M Dec 19 18:31 slow_cooker_linux_amd64*
6.9M Dec 19 18:31 slow_cooker_linux_arm*

[stevej]

Hi Sam,

I need feedback from some team members who are away for the christmas break. Hopefully I'll hear from them soon and I've asked them to comment on this thread.

[samdfonseca]

Hey Steve,

I've decided to go the route of allowing the lua side to modify certain request data (method, url and host), and just pushed those changes. I'd like to be able to test against endpoints that use a resource id in the URL, and I think there are some other worthwhile use cases. Headers aren't modifiable yet since I made these changes before merging in https://github.com/BuoyantIO/slow_cooker/pull/38, but I'll be adding that in the near future.

[samdfonseca]

Hey @stevej @olix0r, have you guys gotten a chance to review this? I plan on doing some work on it in the next few days so it'd be great to hear what your thoughts are. Thanks!

[samdfonseca]

@olix0r @stevej im going to go ahead and close this since i haven't heard back