Request source vs multiple endpoints

[oschaaf]

Opening this issue as a place to discuss combining request sourcing & multiple endpoints.

description

Currently Nighthawk is able to hit multiple endpoints (feature 1) while using a request source (feature 2).

A convenient way to achieve this is through the file based request source plugin, where one can specify a few requests and then the software will loop over those. For those lacking context, see here for an example.

When combining the two features, I think that if one specifies two endpoints and two request options in the traffic profile, this may then proceed to send request 1 consistently to endpoint one and request two to endpoint two. This may not perfectly hold throughout a full execution, but I think chances are there will be skew.

Therefore I'm wondering if options to randomize selection of endpoints and/or request options would be prudent here. That might be useful both for the independent options, but practically essential when combining the two.

[mum4k]

Thank you for starting this discussion @oschaaf.

To help me understand, can you clarify what do you think could be the cause of the skew, i.e. why wouldn't the assignment of options to targets hold throughout the execution. Or is it more about the fact that the assignment tends to be coincidental rather than guaranteed and behaviors added in the future might break it?

Secondly could you elaborate what options you have in mind when talking about randomizing endpoints / options? If our goal is to ensure that the assignment of options to targets remains the same, maybe we want new options that actually fix the two together rather than randomizing them? Unless I misunderstood the problem of course.

[oschaaf]

So right now the file based request source loops over the provided options in a predictable and orderly way. The same applies to any endpoints we define. Both are well defined and useful in my opinion.

However, when combining these things, it might be that option 1/2 defined in the options always ends up being send to endpoint 1/2, and option 2/2 structurally gets send to endpoint 2/2. That might be the desired outcome, but it doesn't have to be;

To be honest, I haven't really concretely thought about what could be next steps, but either an option for the file based request source, or the multi-path endpoint selection scheme, might help:

• Arbitrarily suggesting a means to solve this: adding an option which affects endpoint selection, one could specify something like --multi-path-endpoint-selection: rotate|random. Or maybe we could wire in & piggy back on load balancing strategies provided by Envoy, not sure what that would take.
• A flag makes the behavioural aspect discussed here explicit, while providing a hook to document it, hopefully forcing users to think about it when designing tests.

[mum4k]

Thank you @oschaaf, I agree with your conclusions, it will be good to add the flag. If for nothing else - it will clearly document the behavior that could also be just coincidental currently. We could also add a test that will help ensure we don't break it going forward.