sahilsk
commented
8 years ago Take aways
~ Techtalk by twitter
Unit and integration Test
- 77% production failure can be reproduced in unit testing
- 35% catastrophic failure are due to dev laziness
- poor or no error handling
- types are not testing
- Integration tests : Testing of integrated modules to verify combined functionality.
- three nodes or less can reproduce 98% failure
- testing on three nodes is incredibly useful
Property based testing
check against various inputs generated through properties, help you finding edge cases on minimum investment.
Fault injection
- Fault injection: introducing faults into the system
- chaos monkey: take down services
- latency monkey
- chaos gorilla: take down entire dc
- Jepsen tests: Fault injection tool that simulates network partitions in the system under test
Game days
How to run a game day?
- notify team that failure is coming
- induce failure
- monitor systems under test
- observing only team monitors recovery processes and systems, and file bugs
- Prioritize bugs and get Buy-in across teams
Monitoring is not testing
Canaries
Verification in production
- use canaries to roll out in production
September 16, 2016 at 01:15AM
via Twitter