Loadtest macOS and Windows MDM features

[georgekarrv]

Goal

User story
As a self-hosted Fleet user,
I want Fleet to load test all reference architecture breakpoints w/ macOS and Windows MDM features turned on
so that I know what infrastructure resources I need to handle the load of my fleet's needs.

Context

• Requestor(s): @zayhanlon
• Product designer: @noahtalerman

Changes

Engineering

• [ ] Load testing: Load test all the breakpoints in the Cloud providers > AWS section of the docs.
  • For each breakpoint use 50/50 macOS/Windows hosts with both macOS and Windows MDM turned on for all hosts.
  • If Fleet becomes unusable, (i.e. page load times are over 5 seconds) increase (beef up) the resources if necessary to make Fleet usable.
  • [ ] In addition to the above, load test these scenarios to prepare for prosect-numa:
    1. 8,000 macOS and 30,0000 Window hosts w/ macOS MDM turned on only
    2. Same number of hosts w/ both macOS and Windows MDM turned on
• [ ] Document the results of the load test and best practice resources for each test in a public Google doc
• [ ] Outdated documentation changes: Based on the results, update the best practice resources in the Cloud providers section of the reference architecture docs.

QA

Risk assessment

• Requires load testing: TODO
• Risk level: Low / High TODO
• Risk description: TODO

Manual testing steps

1. Step 1
2. Step 2
3. Step 3

Testing notes

Confirmation

1. [ ] Engineer (@____): Added comment to user story confirming successful completion of QA.
2. [ ] QA (@____): Added comment to user story confirming successful completion of QA.

[noahtalerman]

Hey @zayhanlon and @rfairburn, I updated this story to capture testing/updating the best practice resources for all breakpoints assuming 50/50 macOS/Windows hosts w/ macOS and Windows MDM turned on.

I think this will address the problem that our 25k hosts breakpoint doesn't work for more than 10k hosts w/ Windows MDM turned on (more details here). In addition, I think we'll uncover more updates (to other breakpoints) we want to make to our best practice resources and MDM features.

@georgekarrv I included 2 special tests for prospect-numa. First they'll enroll macOS/Windows but only turn on MDM for macOS. Later they'll turn on MDM for Windows hosts.

I moved the story to "Settled."

Let me know what y'all think.

[zayhanlon]

Looks good to me @noahtalerman - My team can update docs and infra requirements once we have the load test results

[georgekarrv]

Hey team! Please add your planning poker estimate with Zenhub @dantecatalfamo @ghernandez345 @gillespi314 @jahzielv @mna @roperzh