How to pick the initial state?

[2start]

Each permutation of failures might be a viable state. However, we need a fixed action space for q-learning.

Brainstorming of possibilities:

Choose a fixed initial state:

• Easy to train, we can just use unmodified q-learning.
• Unrealistic because we can usually not determine the exact order of failures.

Mode each initial state as a state in the environment:

• Realistic, and we could assign a reward/cost from transitioning from each permutation to each other
• n*(n+1)! state transitions. Which is too much q-learning can't evaluate all the possible actions.

Idea: Limit the lookahead to a random subset of the states to choose the best action.

Other ideas are appreciated.

[MrBanhBao]

As we discussed we abandoned the idea of swapping components. Component-failure-pairs now can be repaired directly disregarding their order.

Initial state: the given list of component-failure-pairs Action space size is now: number of component-failure-pairs (selected component-failure-pair is getting repaired) State/Observation space size: number of component-failure-pairs to the power of two (each component is ether repaired or still broken)