Bring the cost down down down

[h701h]

AKA Tracker Email issue 1

• [ ] - [ ] Options Analysis
• [ ] Scanning cadence Prod(M, W, F), Staging(On-demand), alpha(on-demand),
• [ ] Logging
• [ ] VMs?
• [ ] Summarize all the moves made.
• [ ] Summarize next steps

[FestiveKyle]

Actions done so far:

• Move from 6 nodes (3 zones with 2 nodes each) to 4 nodes (2 zones with 2 nodes each)
• Moved from cluster DB to single
  • 1 server required instead of 3
  • Faster as joins are slow on shards
  • Cheaper since less logs (servers no longer talk to eachother as there is only 1 server)
• Moved to Premium V2 SSDs on staging. These are cheaper and faster
  • Still need to move on prod
• Goldilocks added to GKE for help with pod right-sizing
• Removed unused SSDs on GKE

Next:

• Add memory/cpu limits for scanners
• Create new node pools for scanners for scaling up/down
• Add KEDA for scanner scaling