search

dthtvwls / wsmax

1 stars 1 forks source link

Realistic testing #1

Open jessetane opened 9 years ago

jessetane commented 9 years ago

Any thoughts on how to test without spinning up a ton of ec2 instances? Posting a link to the demo on hn or reddit probably works at least once if you've got enough karma, but would be cool if there was some bot net already out there for doing simulations like this..

dthtvwls commented 9 years ago

Explain "realistic".

jessetane commented 9 years ago

Lots of connections coming from different ip's / geographic locations?

dthtvwls commented 9 years ago

The thing to keep in mind is that whatever generates load has to retain the websocket connection indefinitely. This is why I'm using phantomjs to load the html client and then idle, thousands of times. (I found it to work best in batches of about 500 per process.) The development dependency "thor" is supposed to do this at the websocket level but I didn't spend too much time with it.

There are a few cloud tools I know of for generating load, but I don't know if any of them are suited for websockets (I'm pretty sure BlazeMeter is using JMeter, for example, so they're out). Blitz, the one I am most familiar with, sends all of its traffic for a single run from the same region of ec2, so I don't know if that meets your realism requirement.

bees with machine guns and tsung are probably worth a look into.

jessetane commented 9 years ago

Very interesting, thanks for the notes. I'll play around again tomorrow, but hitting this with thor (from one micro instance to another) easily opens 10k+ connections, which doesn't seem representative of a real world scenario.

I didn't try running the phantom test - out of curiosity what kind of numbers were you seeing?

dthtvwls commented 9 years ago

I haven't tried distributing it yet but phantomjs on my macbook will hit the open files limit long before 10k. It seems to be only heroku's routing mesh that breaks first. If thor is working well for you I would say let's go with that because thor has a lot less to do: no HTTP, no DOM, no JS runtime, etc. You should submit a PR with your thor code.

jessetane commented 9 years ago

Cool, check out #3. The trick is to get data moving back and forth. You definitely can get more connections by running the test from > 1 machines at a time (network bottleneck?) but I only had two machines to attack with, my laptop and another micro instance..

jessetane commented 9 years ago

Also, the macbook probably has a tiny open files limit by default, 256 iirc