Systems Talk: Hot Partitions & Load Variance in Distributed Systems

[eightnoteight]

Hot Partitions & Load Variance in Distributed Systems

Duration: 45 minutes

Covering stories about what leads to load variance in various distributed systems, the behavior it shows when it happens, and finally how to avoid and handle production incidents.

Simple Cases

• CPU Hot Partition Stories
• Memory Hot Partition Stories
• Disk Hot Partition Stories
• Network Hot Partition Stories

Complicated Cases

• Load Variance in Persistent Connection Systems

  • WebSocket Connections
  • gRPC Connections
  • HTTP Persistent Connections
  • MQTT Persistent Connections
  • Control Plane / Data Plane (Istio / Kong, etc.)
  • Multi-Proxy Layers
  • Load Variance in Network Load Balancers

• Load Variance in Application Workloads

  • Kafka Consumer Groups
  • In-App Pub/Sub
  • In-App Cron Job Libraries
    • Celery
    • Sidekiq
    • Gocron
  • Load Variance Due to Noisy Neighbors

• Hot Writer in Writer/Reader Replication Systems

• Hot Leader in Consensus-Based Distributed Systems

Takeaways

• Techniques & Solutions to Avoid Load Variance & Hot Partition Incidents

References

• TODO: I will add the slides 4-5 days in advance.

[srijan-paul]

Thanks for the submission!

This looks great. Since the next meetup is just two hours and we already have the RISC-V talk, I think we can present this in a few ways:

1. Present on October meetup, but with a shorter duration (preferably 25).
2. Present online on Discord/Zoom, for as long as you'd like the session to be.
3. Present in a future meetup (next one is most likely November), for ~40-45 minutes.

Do you think either of these options would work for you?