Be able to cycle through multiple urls

[jtushman]

First off, This utility is awesome! Really incredibly useful.

I would love to slam my app from a handful of urls at the same time, rather than just one. To simulate the actual behavior of my app. Something like the way siege does it with an optional url_file parameter.

Is something like this possible with the BwMG now? If not I would love to see this feature.

I did something like this with ruby/ab by spawning threads and kicking of parallel ab. But I could not get it to scale. Maybe with BwMG, we can send each url in the list to a bee?

I totally would try to add feature to this, but I am lost in Python.

Thanks again for the great utility!

[marfarma]

Why not use multiple swarms, each targeting a different URL? Starting them up in tandem can be scripted with a shell script / batch file -- you don't have to use python.

[ethermeme]

I've been looking for an example of how you would run multiple swarms at once. Care to share an example script?

[ethermeme]

I've had a lot of problems running multiple swarms. One issue has been that the last swarm created is the only swarm bees can see, unless I use the screen command. But that still doesn't work with bees down, only one swarm gets shut down, leaving the others that I have to turn off 1 by 1 in the AWS console.

Another problem with the multi-swarm approach is that you can't effectively emulate a user workflow. Users go through steps A, B & C in a given app. B takes longer than C, which takes longer than A. How do you get the swarms to evenly distribute these requests? I haven't found a way yet. This means I can simulate very unrealistic loads, but I'm still in the dark about what will happen when people start using the app.

Making bees a bit smarter so it can handle a simple click path would avoid a lot of uncertainty and doubt about what it's results are telling me.

[rheist]

I would very much like this feature also. I've been enjoying learning BWMG and using it to test our AWS setup before we switch the DNS. It's been very nice, but I'd like to be able to walk through several of the site's pages to force caching and do a more thorough site test. This is listed as a low priority feature. I'd suggest making it a higher priority and want to add my voice to those requesting it. Has there been any progress on implementing it?

[calebfornari]

I am working on a feature right now that would allow uploading of a Selenium IDE script to be run on the bees. This is for a specific project requirement, but I will submit a PR if there is interest from others in this ability. I may also add the ability to just specify a url config file as specified above while I am at it.

[cosmin]

@rheist the problem is that this is a fundamental limitation of apache bench, on which bees relies for testing. Adding support for stepping through multiple URLs in a sequence requires using something else for driving load (or a lot of bandaid and hacking to simulate something close to this on top of ab).

@calebfornari that would be great pull request if you get around to it.

[calebfornari]

@cosmin Have you seen this? http://code.google.com/p/apachebench-for-multi-url/ Obviously this would require a custom ab build to be present on the AMI, or BWMG would need to upload and rebuild ab when the bee was instantiated. Would you have concerns or preferences regarding this approach? The first would obviously add an extra step for any AMI that was used, while the latter would increase the time it takes to spin up new bees.

[cosmin]

That might work, but patching apache bench seems a little brittle. Are there any efforts to get that merged into ab itself?

[calebfornari]

Not that I can find, the project appears to be somewhat dormant. I agree that the approach is brittle but it's the most direct path to multi-url support I could find. This would be really easy for users to add on their own if they really need it, so unless it's a core feature that you really want in the project it may be better leave it out.

[beefmaster]

i think he means that you give him a list and it goes through every url dosing it and then goes to the next and dos that.

[stevielb]

I would also really like to see multiple url support. I am interested in testing a search engine with a little more than 2000 distinct requests per minute. Unfortunately the solution from @marfarma isn't viable for my situation, because each url only needs to be hit 1-5 times in a test and they need to be distributed throughout the testing period roughly evenly.

[brian-traceme]

So when is this planned? Multiple URL support would make this perfect, but without it, I'm going to need to use something other tool.

[YikSanChan]

@brian-traceme Multiple urls enabled in https://github.com/newsapps/beeswithmachineguns/issues/133