Closed ulibath closed 2 years ago
Initial ideas. Zero reward or negative reward (of the actual failing component). This still does not guarantee that we might force a shop to be deprioritized because an error in the action selection. If that is the case, the solution might not be possible at the agent level. The rank-learner/coordinator might be a better realm to decide the magnitude of the punishment because the rank-learner has a more global perspective of all shops involved. The ideal punishment should be one that still keeps the <component failure, shop> within the correct ranking position.
Thoughts?
If we have a wrong fix, we should apply a negative reward to have the agent learn that this was bad faster. A zero reward does imply that this fix had no negative effect. To be consistent with a software system, we should either reduce the utility of the still failing component or reduce the utility of the wrongly fixed component or both. The RidgeRegression is not using the current utility for predicting the utility that is used for the ranking of a <component failure, shop>. Therefore we can change this attribute without any impact on the ranking.
@christianadriano could you explain the deprioritization? Does this mean that we're ranking a shop lower if there were happing more false positives in the past?
Just inverting the component utility might have a bad influence on the training