search

INFLUENCEorg / aiagents

A repository for agent components and their algorithms in reinforcement learning scenarios
Other
0 stars 0 forks source link

test examples #19

Open Wouter1 opened 4 years ago

Wouter1 commented 4 years ago

On 03/06/2020 17:12, Aleksander Czechowski wrote:

  1. Revisit the examples directories in aienvs and aiagents. All examples should be executable by running "python3 file.py" (or eclipse equivalent..) Examples may take much longer than unit tests, so you don't need to run them to the end -- it is enough to kill them after few seconds of execution

Some examples may fail because they are outdated in one way or another, and I think we haven't looked into these directories in a while

Wouter1 commented 4 years ago

@czechows Tried that already. but iin this case the parameters are multiple layers deep. And the code would need to know exactly the structure in the file. Which kind of defeats putting it in a file in the first place.

Wouter1 commented 4 years ago

It seems above error indicates that 'robots' is not a name in the env. And that is correct.

RobotAgent has several children. I suppose it should not have an id?

Wouter1 commented 4 years ago

No, MctsAgent is a 'single' agent. But then it should work on a single entity?

Wouter1 commented 4 years ago

@czechows I'm puzzled by the agent_combined_config.yaml that is used for MctsAggrExample

MctsAgent is a single agent. That means FAIK that it is connected to a single entity in the env. So it can't have the id "robots", it should have a existing robot name eg robotN with N in [1,2,3].

Also MCTS agent does not seem to be simple because it contains several like treeAgent, rolloutAgent etc?

czechows commented 4 years ago

This is an example to show how to aggregate entities (using some of the functionality you wrote for us some time ago like PackedSpace etc.). Therefore multiple robots are controlled by a single agent.

treeAgent, rolloutAgent are specific to internals of MCTS; you have to specify what algorithm of picking actions (=agent) one uses at certain parts within MCTS (e.g. random)

Wouter1 commented 4 years ago

@czechows But then the RobotAgent (MctsAgent) should get a VALID agentid. 'robots' is not a valid ID?

Wouter1 commented 4 years ago

@czechows also, if the MctsAgent is only acting for one entity (say robot1), then who is determining the actions for the other robots?

Wouter1 commented 4 years ago

The MctsExample uses agent_config.yaml which seems to contain a more complete and up-to-date content. Still no 'simulator' here though

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 48, in createAgent
    obj = klass(parameters['id'], actionspace, observationspace, class_parameters)
  File "/home/wouter/git/aiagents/aiagents/single/mcts/MctsAgent.py", line 56, in __init__
    envparams = parameters['simulator']
KeyError: 'simulator'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 79, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 57, in main
    complexAgent = createAgent(env.action_space, env.observation_space, agent_parameters)
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 54, in createAgent
    subAgentList.append(createAgent(actionspace, observationspace, subAgentParameters))
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 50, in createAgent
    raise ValueError(classname + " failed on __init__:") from error
ValueError: aiagents.single.mcts.MctsAgent.MctsAgent failed on __init__:
czechows commented 4 years ago

@Wouter1 it is acting for all robots at once See line: packedActionSpace = PackedSpace(baseEnv.action_space, {"robots":["robot1", "robot2", "robot3"]})

According to this line the new (combined) id is "robots".

Wouter1 commented 4 years ago

AgentFactory line 56 is weird

    elif 'subAgentList' in parameters:
        subAgentList = []
        for subAgentParameters in parameters['subAgentList']:
            subAgentList.append(createAgent(actionspace, observationspace, subAgentParameters))
            try:
                obj = klass(subAgentList, actionspace, observationspace, class_parameters)
            except Exception as error:
                raise ValueError(classname + " failed on __init__:") from error

Why are we calling the constructor multiple times and doing nothing with obj? Only the last value seems returned

Wouter1 commented 4 years ago

It seems that the indentation is incorrect, I assume that obj should be called only when the subAgentList is complete.

Wouter1 commented 4 years ago

Getting in MctsExample

Default config 
{'seed': None, 'max_steps': 40, 'environment': {'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}}
{'class': 'aiagents.multi.BasicComplexAgent.BasicComplexAgent', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robot1', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robot1', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot1', 'parameters': {}}, 'otherAgents': {'class': 'aiagents.multi.BasicComplexAgent.BasicComplexAgent', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot2', 'parameters': {}}, {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot3', 'parameters': {}}]}, 'iterationLimit': 500, 'diyBonus': 0.0, 'treeParameters': {'explorationConstant': 1, 'samplingLimit': 30, 'maxSteps': 10}}}, {'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robot2', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robot2', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot2', 'parameters': {}}, 'otherAgents': {'class': 'aiagents.multi.FactoryFloorIterativeGreedy.FactoryFloorIterativeGreedy', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot1', 'parameters': {}}, {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot3', 'parameters': {}}]}, 'iterationLimit': 500, 'diyBonus': 0.0, 'treeParameters': {'explorationConstant': 1, 'samplingLimit': 30, 'maxSteps': 10}}}, {'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robot3', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robot3', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot3', 'parameters': {}}, 'otherAgents': {'class': 'aiagents.multi.FactoryFloorIterativeGreedy.FactoryFloorIterativeGreedy', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot1', 'parameters': {}}, {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot2', 'parameters': {}}]}, 'iterationLimit': 500, 'diyBonus': 0.0, 'treeParameters': {'explorationConstant': 1, 'samplingLimit': 30}}}]}
No SLURM_JOB_ID found
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 56, in createAgent
    obj = klass(subAgentList, actionspace, observationspace, class_parameters)
TypeError: __init__() takes from 3 to 4 positional arguments but 5 were given

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 48, in createAgent
    obj = klass(parameters['id'], actionspace, observationspace, class_parameters)
  File "/home/wouter/git/aiagents/aiagents/single/mcts/MctsAgent.py", line 72, in __init__
    self._rolloutAgent = createAgent(self._simulator.action_space, self._simulator.observation_space, rolloutAgentDict)
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 58, in createAgent
    raise ValueError(classname + " failed on __init__:") from error
ValueError: aiagents.multi.FactoryFloorIterativeGreedy.FactoryFloorIterativeGreedy failed on __init__:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 84, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 62, in main
    complexAgent = createAgent(env.action_space, env.observation_space, agent_parameters)
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 54, in createAgent
    subAgentList.append(createAgent(actionspace, observationspace, subAgentParameters))
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 50, in createAgent
    raise ValueError(classname + " failed on __init__:") from error
ValueError: aiagents.single.mcts.MctsAgent.MctsAgent failed on __init__:

The error message is saying that FactoryFloorIterativeGreedy has incorrect constructor, but the errormessage itself is also confusing. The 'self' parameter is not shown.

The problem is that it should take actionspace:Dict, observationspace but takes environment.

Wouter1 commented 4 years ago

Fixed that. Now getting

warning: Debugger speedups using cython not found. Run '"/home/wouter/git/aienvs/venv/bin/python" "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/setup_cython.py" build_ext --inplace' to build.
pydev debugger: starting (pid: 11543)
Default config 
{'seed': None, 'max_steps': 40, 'environment': {'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}}
{'class': 'aiagents.multi.BasicComplexAgent.BasicComplexAgent', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robot1', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robot1', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot1', 'parameters': {}}, 'otherAgents': {'class': 'aiagents.multi.BasicComplexAgent.BasicComplexAgent', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot2', 'parameters': {}}, {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot3', 'parameters': {}}]}, 'iterationLimit': 500, 'diyBonus': 0.0, 'treeParameters': {'explorationConstant': 1, 'samplingLimit': 30, 'maxSteps': 10}}}, {'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robot2', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robot2', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot2', 'parameters': {}}, 'otherAgents': {'class': 'aiagents.multi.FactoryFloorIterativeGreedy.FactoryFloorIterativeGreedy', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot1', 'parameters': {}}, {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot3', 'parameters': {}}]}, 'iterationLimit': 500, 'diyBonus': 0.0, 'treeParameters': {'explorationConstant': 1, 'samplingLimit': 30, 'maxSteps': 10}}}, {'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robot3', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robot3', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot3', 'parameters': {}}, 'otherAgents': {'class': 'aiagents.multi.FactoryFloorIterativeGreedy.FactoryFloorIterativeGreedy', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot1', 'parameters': {}}, {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot2', 'parameters': {}}]}, 'iterationLimit': 500, 'diyBonus': 0.0, 'treeParameters': {'explorationConstant': 1, 'samplingLimit': 30}}}]}
No SLURM_JOB_ID found
>>> ffAgentList[0]
<aiagents.single.FactoryFloorAgent.FactoryFloorAgent object at 0x7fba1a9ed6d8>
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 56, in createAgent
    obj = klass(subAgentList, actionspace, observationspace, class_parameters)
  File "/home/wouter/git/aiagents/aiagents/multi/FactoryFloorIterativeGreedy.py", line 18, in __init__
    self.pathDict = ffAgentList[0].pathDict
AttributeError: 'FactoryFloorAgent' object has no attribute 'pathDict'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 48, in createAgent
    obj = klass(parameters['id'], actionspace, observationspace, class_parameters)
  File "/home/wouter/git/aiagents/aiagents/single/mcts/MctsAgent.py", line 72, in __init__
    self._rolloutAgent = createAgent(self._simulator.action_space, self._simulator.observation_space, rolloutAgentDict)
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 58, in createAgent
    raise ValueError(classname + " failed on __init__:") from error
ValueError: aiagents.multi.FactoryFloorIterativeGreedy.FactoryFloorIterativeGreedy failed on __init__:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 2225, in <module>
    main()
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 2218, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 1560, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 1567, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/_pydev_imps/_pydev_execfile.py", line 25, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 84, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 62, in main
    complexAgent = createAgent(env.action_space, env.observation_space, agent_parameters)
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 54, in createAgent
    subAgentList.append(createAgent(actionspace, observationspace, subAgentParameters))
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 50, in createAgent
    raise ValueError(classname + " failed on __init__:") from error
ValueError: aiagents.single.mcts.MctsAgent.MctsAgent failed on __init__:
Wouter1 commented 4 years ago

It seems FactoryFloorIterativeGreedy is assuming it can access a 'pathDict' field inside the FactoryFloorAgent. But this is not thhe case. There is a pathDict but it is generated only after step. Also I doubt that it should be visible.

Wouter1 commented 4 years ago

Moved that part of the code to FactoryFloorIterativeGreedy#step

Wouter1 commented 4 years ago

Yes, now we have

Default config 
{'seed': None, 'max_steps': 40, 'environment': {'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}}
{'class': 'aiagents.multi.BasicComplexAgent.BasicComplexAgent', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robot1', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robot1', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot1', 'parameters': {}}, 'otherAgents': {'class': 'aiagents.multi.BasicComplexAgent.BasicComplexAgent', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot2', 'parameters': {}}, {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot3', 'parameters': {}}]}, 'iterationLimit': 500, 'diyBonus': 0.0, 'treeParameters': {'explorationConstant': 1, 'samplingLimit': 30, 'maxSteps': 10}}}, {'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robot2', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robot2', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot2', 'parameters': {}}, 'otherAgents': {'class': 'aiagents.multi.FactoryFloorIterativeGreedy.FactoryFloorIterativeGreedy', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot1', 'parameters': {}}, {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot3', 'parameters': {}}]}, 'iterationLimit': 500, 'diyBonus': 0.0, 'treeParameters': {'explorationConstant': 1, 'samplingLimit': 30, 'maxSteps': 10}}}, {'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robot3', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robot3', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot3', 'parameters': {}}, 'otherAgents': {'class': 'aiagents.multi.FactoryFloorIterativeGreedy.FactoryFloorIterativeGreedy', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot1', 'parameters': {}}, {'class': 'aiagents.single.FactoryFloorAgent.FactoryFloorAgent', 'id': 'robot2', 'parameters': {}}]}, 'iterationLimit': 500, 'diyBonus': 0.0, 'treeParameters': {'explorationConstant': 1, 'samplingLimit': 30}}}]}
No SLURM_JOB_ID found
Action: {'robot1': 3} child value 3.5904761904761906 numVisits 105 totalReward 377.0 variance 0.02552294568621099
Action: {'robot1': 1} child value 3.5 numVisits 100 totalReward 350.0 variance 0.023699999999999992
Action: {'robot1': 2} child value 3.347826086956522 numVisits 92 totalReward 308.0 variance 0.027512944850826
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 84, in <module>
Action: {'robot1': 4} child value 3.6513761467889907 numVisits 109 totalReward 398.0 variance 0.020768646879722383
Action: {'robot1': 0} child value 3.382978723404255 numVisits 94 totalReward 318.0 variance 0.02650665074212843
    main()
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 67, in main
    rewards = experiment.run()
  File "/home/wouter/git/aienvs/aienvs/runners/Experiment.py", line 58, in run
    episodeSteps, episodeReward = episode.run()
  File "/home/wouter/git/aienvs/aienvs/runners/Episode.py", line 51, in run
    obs, globalReward, done = self.step(obs, globalReward, done)
  File "/home/wouter/git/aienvs/aienvs/runners/Episode.py", line 32, in step
    actions = self._agent.step(obs, globalReward, done)
  File "/home/wouter/git/aiagents/aiagents/multi/BasicComplexAgent.py", line 17, in step
    agentActions = agentComponent.step(state, reward, done)
  File "/home/wouter/git/aiagents/aiagents/single/mcts/MctsAgent.py", line 93, in step
    root.executeRound()
  File "/home/wouter/git/aiagents/aiagents/single/mcts/nodes.py", line 77, in executeRound
    selectedNode, rolloutReward = selectedNode.expand()
  File "/home/wouter/git/aiagents/aiagents/single/mcts/nodes.py", line 111, in expand
    expandedNode = self._children[key].expand()
  File "/home/wouter/git/aiagents/aiagents/single/mcts/nodes.py", line 186, in expand
    otherActions = self.otherAgents.step(self._parent.getState(), self._parent.immediateReward, self._parent.isTerminal)
  File "/home/wouter/git/aiagents/aiagents/multi/FactoryFloorIterativeGreedy.py", line 27, in step
    pathDict = self._agentSubcomponents[0].pathDict
AttributeError: 'FactoryFloorAgent' object has no attribute 'pathDict'
czechows commented 4 years ago

It seems FactoryFloorIterativeGreedy is assuming it can access a 'pathDict' field inside the FactoryFloorAgent. But this is not thhe case. There is a pathDict but it is generated only after step. Also I doubt that it should be visible.

Yes so this code was not used for a while but I remember that it was the only way to engineer it in a sane way. Perhaps pathDict should be an empty pointer in FactoryFloorAgent on initialization? Everything is by reference so it will work well.

PathDict being public is OK

Wouter1 commented 4 years ago

The problem is that even in FactoryFloorIterativeGreedy#step we don't always have pathDict in FactoryFloor.pathDict.

It seems better to make a function getPathDict() or so so that FactoryFloorAgent can compute it if necessary.

czechows commented 4 years ago

The problem is that even in FactoryFloorIterativeGreedy#step we don't always have pathDict in FactoryFloor.pathDict.

It seems better to make a function getPathDict() or so so that FactoryFloorAgent can compute it if necessary.

Well so it does compute it each step for itself anyways.

In general I would be careful with any non-trivial refactoring FactoryFloorAgent because test coverage is not great and it is easy to make things very sub-optimal in terms of computation.

Wouter1 commented 4 years ago

Better. Now

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 84, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/MctsExample.py", line 67, in main
    rewards = experiment.run()
  File "/home/wouter/git/aienvs/aienvs/runners/Experiment.py", line 58, in run
    episodeSteps, episodeReward = episode.run()
  File "/home/wouter/git/aienvs/aienvs/runners/Episode.py", line 51, in run
    obs, globalReward, done = self.step(obs, globalReward, done)
  File "/home/wouter/git/aienvs/aienvs/runners/Episode.py", line 32, in step
    actions = self._agent.step(obs, globalReward, done)
  File "/home/wouter/git/aiagents/aiagents/multi/BasicComplexAgent.py", line 17, in step
    agentActions = agentComponent.step(state, reward, done)
  File "/home/wouter/git/aiagents/aiagents/single/mcts/MctsAgent.py", line 93, in step
    root.executeRound()
  File "/home/wouter/git/aiagents/aiagents/single/mcts/nodes.py", line 77, in executeRound
    selectedNode, rolloutReward = selectedNode.expand()
  File "/home/wouter/git/aiagents/aiagents/single/mcts/nodes.py", line 111, in expand
    expandedNode = self._children[key].expand()
  File "/home/wouter/git/aiagents/aiagents/single/mcts/nodes.py", line 186, in expand
    otherActions = self.otherAgents.step(self._parent.getState(), self._parent.immediateReward, self._parent.isTerminal)
  File "/home/wouter/git/aiagents/aiagents/multi/FactoryFloorIterativeGreedy.py", line 42, in step
    action = ffAgent.getAction(stayInPlacePath)
AttributeError: 'FactoryFloorAgent' object has no attribute 'getAction'
Wouter1 commented 4 years ago

The problem is that even in FactoryFloorIterativeGreedy#step we don't always have pathDict in FactoryFloor.pathDict. It seems better to make a function getPathDict() or so so that FactoryFloorAgent can compute it if necessary.

Well so it does compute it each step for itself anyways.

In general I would be careful with any non-trivial refactoring FactoryFloorAgent because test coverage is not great and it is easy to make things very sub-optimal in terms of computation.

@czechows thanks but I take that as a comment on my fixes; I can not do much with your comment. pathDict was used before it was even computed so the caller's code was broken. I had to clean that up.

Can you please answer the open questions I asked earlier? Progress there is blocked awaiting your answers

Wouter1 commented 4 years ago

It seems FactoryFloorIterativeGreedy is assuming it can access a 'pathDict' field inside the FactoryFloorAgent. But this is not thhe case. There is a pathDict but it is generated only after step. Also I doubt that it should be visible.

Yes so this code was not used for a while but I remember that it was the only way to engineer it in a sane way. Perhaps pathDict should be an empty pointer in FactoryFloorAgent on initialization? Everything is by reference so it will work well.

PathDict being public is OK

No it's not OK. It's not even available until an undefined moment. And if it's not avaialble the caller will use a bogus pathDict with your solution.

I fixed this by making an explicit function getPathDict in FactoryFloorAgent that will compute the value if it was not yet computed, and cache it for future use.

Wouter1 commented 4 years ago

@czechows This ticket is MUCH more work than anticipated. Maybe we should make a separate ticket for each example. That also allows us to prioritize the work on this. I worked basically the entire day with just 5 examples and they are not even fully fixed yet.

czechows commented 4 years ago

@Wouter1 which questions do you refer to?

It is fine that things are not fully fixed. I didn't know what needed fixing.

Wouter1 commented 4 years ago

@czechows I'm out of time already, just writing a quick update

Fixed many things but not nearly enough. Almost all examples are broken and in many ways.

@Wouter1 it is acting for all robots at once See line: packedActionSpace = PackedSpace(baseEnv.action_space, {"robots":["robot1", "robot2", "robot3"]})

According to this line the new (combined) id is "robots".

Ah I missed that line. Then there must be something else broken that I still missed

We have to make separate tickets for each example as the amount of work on each is so much. And we then may have to find back the open ends here and collect them.

BTW I put the tickets in aienvs/aiagents but I forgot about them being public. Are all these reports now visible for public? We may must move them then to the private ticket system....

czechows commented 4 years ago

@czechows I'm out of time already, just writing a quick update

Fixed many things but not nearly enough. Almost all examples are broken and in many ways.

That is fine, perhaps even expected. They were not updated during many revisions, because they cannot be ran automatically like the tests (I think?) so this work accumulated over some weeks.

@Wouter1 it is acting for all robots at once See line: packedActionSpace = PackedSpace(baseEnv.action_space, {"robots":["robot1", "robot2", "robot3"]})

According to this line the new (combined) id is "robots".

Ah I missed that line. Then there must be something else broken that I still missed

We have to make separate tickets for each example as the amount of work on each is so much. And we then may have to find back the open ends here and collect them.

Whether you track it in one or several threads is all fine by me.

BTW I put the tickets in aienvs/aiagents but I forgot about them being public. Are all these reports now visible for public? We may must move them then to the private ticket system....

Yes they are all public as indicated in my emails from some time ago. Can you transfer issues yourself -- do you have the button transfer issue? If not, send me the links to the ones you want to hide and I will move them to aienvs-conf-issues (our folder for confidential issues, code there is irrelevant). For me it doesn't matter.

Wouter1 commented 4 years ago

Can you please transfer these? Thanks! https://github.com/INFLUENCEorg/aienvs/issues/39 https://github.com/INFLUENCEorg/aienvs/issues/40 https://github.com/INFLUENCEorg/aienvs/issues/41 https://github.com/INFLUENCEorg/aienvs/issues/42 https://github.com/INFLUENCEorg/aienvs/issues/43 https://github.com/INFLUENCEorg/aienvs/issues/44

https://github.com/INFLUENCEorg/aiagents/issues/16 https://github.com/INFLUENCEorg/aiagents/issues/17 https://github.com/INFLUENCEorg/aiagents/issues/18 https://github.com/INFLUENCEorg/aiagents/issues/19

Why would it be impossible to make unit tests for these?

czechows commented 4 years ago

Can you please transfer these? Thanks! INFLUENCEorg/aienvs#39 INFLUENCEorg/aienvs#40 INFLUENCEorg/aienvs#41 INFLUENCEorg/aienvs#42 INFLUENCEorg/aienvs#43 INFLUENCEorg/aienvs#44

done!

16

17

18

19

Why would it be impossible to make unit tests for these?

I think examples serve a different purpose than unit tests, and they also sometimes take much longer to finish..

Wouter1 commented 4 years ago

@czechows

The unit tests show that the code works. "the code" now includes examples. And we also want to make sure that the examples work.

So there's nothing wrong with testing the examples. Yes they need to be tested quickly so we have to make unit tests that test these without doing the full run but only run say a minute or 10 sec, depending on what we know is sufficient to check that they weork properly. And we may also want to change some examples to improve their testability