Closed evanthebouncy closed 3 years ago
This sounds great. Having clear interfaces between them will make things smooth 👍
w.r.t. filtering out "clearly bad actions", go to this line and add something after it:
actions, action_probabilities = agent.get_actions_prob(cur_graph, target_graph)
# add these lines below
clearly_bad_actions = get_clearly_bad_actions(actions, cur_graph, target_graph) # this should be sufficient to filter out obviously bad actions
# set the probability for those clearly bad actions to 0
for bad_action in clearly_bad_actions:
action_probabilities[actions.index(bad_action)] = 0.0
# re-normalize the probabilities so they sum to 1 again
action_probabilities = action_probabilities / sum(action_probabilities) # this array broadcasting will only work if action_probabilities is a numpy array
as noted above, we should probably be using numpy arrays instead of python arrays, so change agent's code as following as well
return list_actions, np.array(list_probabilities)
2020-08-27
we'll organise the search structure to admit easy experiments and plot generations
base_search.py is perfect the way it is, but, I would rename BaseSearch to ReplEnv, and have it handle interacting with the fusion server, and leave the logging somewhere else. For now leave it as is is fine.
we'll be modifying random_search.py, factoring it to allow easy modifications and additions of other agents and search procedures
the general idea of performing search is that, while an agent is unlikely to produce the correct reconstruction in one go, repeated usage of the agent in clever ways will increase the likelihood of reconstruction.
there are different kinds of agents:
there are different kinds of search procedures:
as we can see, the agents and the search-procedures are factorized and decoupled, so that any agent can be leveraged with any search procedure (for a total of 9 combinations here).
we'll start by building one of such combinations: AgentRandom with random-rollout, keeping the two decoupled so when we get the trained NN agent, AgentSupervised, we can swap out the random-agent to get the combination of AgentSupervised with random-rollout.
Agent
Agent can be an abstract class, it needs to implement the following methods:
Search Procedures
In a nutshell, a search procedure should amplify the success rate of any agent, by running it multiple times in a clever way, it needs to implement the following methods:
Now we have the best_score_over_time for a particular target_file, of a particular search procedure, with a particular agent. Let us generate the plot for it. You'll probably have to adapt it a bit to fit with the rest.