search

aws-samples / distributed-load-testing-using-aws-fargate

Solution to setup AWS Fargate to run multi-region distributed performance testing.
Other
83 stars 33 forks source link

JMeter Distributed Mode #8

Open cda210 opened 4 years ago

cda210 commented 4 years ago

How can the task containers be configured as hosts when taurus.yml is configured using JMeter in Distributed Mode? I know networking with fargate can be tricky without an ALB. Looking for some insight here, please? Thanks. Great stuff here BTW!

ferdingler commented 4 years ago

Hi @cda210, thanks for giving this tool a try!

Would you even want to run JMeter in distributed mode though? This tool runs an instance of taurus per Fargate task, which is distributed by design already. I'm curious to hear how would you expect/like the distributed mode to work? What would you get that you aren't getting already?

Networking with Fargate is not that much different than networking on EC2. Each task gets it's own Elastic Network Interface (ENI) with it's own IP address from the VPC. Containers within the Fargate task can communicate with each other through localhost. If tasks want to communicate with each other, they can use their corresponding private IP address.

cda210 commented 4 years ago

I have realized that I am already getting the behavior of JMETER distributed mode - but I framed the question using that mode because you need IPs. That is what I want to figure out - I am curious about dynamically getting these private IP addresses. The subnets can be determined pretty easily - it's getting the full IP that's the tricky part of this process. I have a hypothesis that - if I can prove out - could make a big difference with some minor adjustments for my infrastructure constraints.

It's awesome as is, but I am looking into heavily customization of JMeter's execution. This is one critical component I will need - IP addresses of each task before the load test is ran - so at the very latest during ramp up period.

I am halfway to a tool that simulate load on my api and sites that as it grows, where the hitches will be and start to act proactively vs reactive. Thanks to you - I am happy to give back to the project when I get to a point of satisfaction.

For example - there was a request for the below, which I working within the constaints of AWS: tasks > 10 being spawned in groups of 10

ferdingler commented 4 years ago

It's awesome as is, but I am looking into heavily customization of JMeter's execution. This is one critical component I will need - IP addresses of each task before the load test is ran - so at the very latest during ramp up period.

Is the intention of knowing the IP addresses to whitelist them in your target system? If so, you could have an external process (i.e. Lambda function) monitor the running tasks in the cluster and get their IP addresses as they get scheduled in Fargate. The tasks will actually get an ENI with an IP address assigned even while they are in the PENDING status (before RUNNING).

It would be easier if you could Bring your own ENIs to Fargate, and specify them at the RunTask API, but unfortunately that's not a feature Fargate supports at the moment.

For example - there was a request for the below, which I working within the constaints of AWS: tasks > 10 being spawned in groups of 10

Are you working on it? that would be great if you can do a pull request for it. Shouldn't be too complicated to loop in groups of 10 if count is greater than 10.