Readme thoughts

[samwhite]

We’re focusing on a RESTful HTTP API, which for each incoming request, makes a small number of upstream API requests, collects data from the responses and responds to the original request. The majority of the time taken to respond is waiting for responses from upstream requests. Therefore the performance is IO-bound. We believe this is a fairly typical scenario in microservice architectures in large-scale enterprise systems.

upstream should say downstream (See RFC 2616§1.3: https://tools.ietf.org/html/rfc2616#section-1.3)

Therefore the performance is IO-bound.

I think this is a faulty corollary... do you mean network IO? And what do we mean by performance (concurrency / latency)?

startup time and resource usage until ready to handle requests

This seems highly specific to the specific use case I know you're looking at currently 😄 I think generally this is not an issue, or at least one which k8s/container orchestration should remove.

(the diagram)

• There's a line coming out of node1 which goes nowhere?
• The light shaded background makes the node labels difficult to read (imo)
• The font in general is a bit tricky 😞

node for a simulated back-end (e.g. 2-second latency)

If you're looking to emulate legacy systems, I would make this significantly higher, and add some variance (adding variance here can introduce some interesting behaviours with certain GCs). Would recommend 5-10 seconds with some sort of pseudo-random distribution.

You may also want to discuss how many requests you will be sending in parallel at any one time and the ramp up rate (locust uses 'Number of Users' and 'Hatch Rate' to describe these) - as these can make a notable differrence.

Java, Spring Boot with Servlet API (Tomcat, @Async). Thread pool concurrency model (blocking IO) for incoming and outgoing requests

Are the downstream requests made with the same thread pool or a separate one?

We will also test the performance limits of the legacy back-end to ensure that capacity here is not a problem. However, tech choice is much less relevant here, so we choose Go and Rust, for self-indulgent purposes :-)

Probably not relevant given you'll provide a high amount of resources, but I would rewrite this bit to say that you've picked the highest performances ones currently available, see: https://www.techempower.com/benchmarks/. Drogon or Actix are the most performant, and thus least likely to introduce noise or variance to test results.

[StuartHarris]

Yay! Thanks for your comments @samwhite – much appreciated.

• I read that rfc as "messages flowing from upstream to downstream" meaning requests go upstream and responses downstream. So it is correct unless I'm understanding it wrong.
• Yeah, network IO, and I think we just mean overall service performance (in terms of throughput). I'll update the readme.
• startup time and resources required to be ready, is, from our experience a problem for Java Spring Boot services, and IMO critical when running under K8s. Pods should be light and nimble so that they can be scaled quickly and not cause problems on the cluster. When a pod takes a minute, for example, to start up and consumes lots of CPU during that period, it makes setting resource requests and limits really tricky and can trigger cascading failures with multiple crash loop back-off scenarios unless carefully managed (e.g. with pod disruption budgets etc.). We're trying to explore this more in this experiment.
• great suggestion re increasing the latency randomly. My first thoughts would be that it could skew results if measured separately, but I think that would even out if we ran each test for long enough. Will give that a go! Thanks.
• re ramp up and thread pool configurations, we'll evolve this as we go and update the readme. Initially we're trying to get an idea, and we can make it more reliable/repeatable/etc if we need to.
• for the legacy back-end, unless I'm mistaken, this just needs to be performant enough to stay out of the way. So not too bothered TBH about what it is.