Closed 2start closed 4 years ago
@2start Implemented a gym.env in the envs dir. You also can find a notebook with some examples how to use this environment.
@brrrachel copied your data_handler into the env directory. I added the method "get_repair_failure_probability". It should return the probability if a repair action will fail or not. At the moment it is static with a probability of 10% that a repair will fail. I do not know if there are information of repair action failures. Maybe @christianadriano has a idea how to choose the failure rate.
@brrrachel @2start @christianadriano We should also think of a strategy to "punish" the agent's action to repair a already repaired component.
e.g.: Components: ["A", "B", "C"] -> repair "A" -> ["B", "C"] -> repair "A" -> ["B", "C"]
At the moment the reward of this redundant actions are zero. It is possible that the agent's still learns that in a certain state specific actions do not make any sense. But believe the agent's policy would converge much faster if we make this punishment negative in respect to the step it already made. Means the negative reward increases by the amount of steps already been taken. This negative reward should be somehow in the range of the difference reward functions ('cubic', 'log10', 'srt').
Thanks! Regarding the negative reward: Whats the counterargument against a fixed negative reward?
@2start thats what @christianadriano mentioned last week. We want to change to a non-stationary environment where e.g. the number of users/computers might influence the overall system behaviour and rewards might be dependent on this / influenced by this (e.g. with more users we get worse reward).
We need to integrate the environment into the interface provided by gym.
Gym is a RL framework that provides different environments with the same interface to test different RL algorithms. We should integrate our environment with the given interface.
Here is an article about creating a gym environment: Gym example
Here is a description by @christianadriano. However, we just decided to use the naming conventions of gym.