search

DiamondLightSource / squid

A repo for UI library developments
Apache License 2.0
0 stars 0 forks source link

Chaos test the cluster #21

Open stan-dot opened 5 months ago

stan-dot commented 5 months ago

explore cluster resilience testing

options considered:

DiamondJoseph commented 5 months ago

Whatever decisions and learning come out of this are going to be applicable to the other beamline repositories

stan-dot commented 5 months ago

note: this tool is more suited to aws and isn't that well-developed https://chaostoolkit.org/reference/tutorials/containerising/

stan-dot commented 5 months ago

potential alternatives

stan-dot commented 2 months ago

raised a chaos-mesh ticket with the cloud team about the namespace creation

stan-dot commented 2 months ago

moved to looking into a more restricted tool - https://github.com/asobti/kube-monkey

stan-dot commented 2 months ago

with a helm chart https://github.com/asobti/kube-monkey/blob/master/helm/kubemonkey/README.md

stan-dot commented 2 months ago

working on it

stan-dot commented 1 month ago

tracking the cloud aspect here https://jira.diamond.ac.uk/servicedesk/customer/portal/2/SCHD-6072

stan-dot commented 3 weeks ago

might use litmus https://github.com/litmuschaos/litmus

stan-dot commented 3 weeks ago

also this relevant https://confluence.diamond.ac.uk/display/SSCC/Graduate+project+idea%3A+Chaos+Engineering+for+the+Workflow+Engine

stan-dot commented 2 weeks ago

Workflow Controller: The Argo Workflow Controller responsible for the creation of Chaos Experiments using the Chaos Experiment CR.

and argo isn't quite ready yet? might need to delay this https://docs.litmuschaos.io/docs/architecture/chaos-execution-plane

stan-dot commented 2 weeks ago

https://docs.litmuschaos.io/docs/concepts/chaoshub

stan-dot commented 3 days ago

this is not on the critical path, deprioritizing