search

hpi-sam / rl-4-self-repair

Reinforcement Learning Models for Online Learning of Self-Repair and Self-Optimization
MIT License
0 stars 1 forks source link

Probability of Unsuccessful Repair #17

Open christianadriano opened 4 years ago

christianadriano commented 4 years ago

Hi @MrBanhBao and @2start, could you please give me some feedback on the ideas below? This is only a reflection, it is not going to become a feature to implement.

  1. What is the probability that fixing this component will be succeful? Success of fixing a component depends on hidden information, the dependencies that this component has on other components which might be failing.

  2. If a component is failing, what is the probability that the failure is in the dependent components instead? If one component fails only because of other components, then we do not need to waste effort repairing the failing component. This probability can be used to make a decision of whether it is worth repairing something that might not be broken. If the probability is low, then it is better to repair the component.

2.1 What do we want to know? Given an observation (component 1 and 2 failing), what is the probability that fixing component 2 will automatically fix component 1?

2.2 Why we want to know this? Because we can make less expensive and faster repair decisions by repairing only the root-causes of the failure.

  1. How to learn these probabilities?

3.1 Model - We can use a hidden Markov Model to express the states of a component and how these states depend on the states of other components. These states are hidden, but we can learn their transitions by observing how the components fail to be fixed or the situations in which they succeed to be fixed. The transitions between hidden states would consist of the probability of a failure propagating from one component to another.

3.3 Usage - We can use these probabilities to compute expected utilities (probability of being an isolated failure in the current component x the utility_increase for that component).

2start commented 4 years ago

I think that modeling dependent failures inside of the environment would certainly enhance the similarity to real world environments and can be learned by the agent. Regarding concrete probabilities: I don't have a clue how to choose them without looking at real world data and I don't know whether such information is available.