search

INFLUENCEorg / aiagents

A repository for agent components and their algorithms in reinforcement learning scenarios
Other
0 stars 0 forks source link

test examples #19

Open Wouter1 opened 4 years ago

Wouter1 commented 4 years ago

On 03/06/2020 17:12, Aleksander Czechowski wrote:

  1. Revisit the examples directories in aienvs and aiagents. All examples should be executable by running "python3 file.py" (or eclipse equivalent..) Examples may take much longer than unit tests, so you don't need to run them to the end -- it is enough to kill them after few seconds of execution

Some examples may fail because they are outdated in one way or another, and I think we haven't looked into these directories in a while

Wouter1 commented 4 years ago

AgentComponent.py gives

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/AgentComponent.py", line 32, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/AgentComponent.py", line 17, in main
    simpleComponentList.append(RandomAgent(i, action_space))
TypeError: __init__() missing 1 required positional argument: 'observationspace'
Wouter1 commented 4 years ago

Fixed AgentComponent demo

Wouter1 commented 4 years ago

AgentFactory.py

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/AgentFactory.py", line 2, in <module>
    from mock import Mock
ModuleNotFoundError: No module named 'mock'
Wouter1 commented 4 years ago

Correct import is from unittest.mock import Mock

Wouter1 commented 4 years ago 
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/AgentFactory.py", line 29, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/AgentFactory.py", line 17, in main
    parameters = getParameters(filename)
  File "/home/wouter/git/aienvs/aienvs/utils.py", line 12, in getParameters
    with open(filename, 'r') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/home/wouter/git/aiagents/examples/configs/agentconfig.yaml'

Strange, did someone rename agentconfig.yaml to agent_config.yaml?

Wouter1 commented 4 years ago 
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/AgentFactory.py", line 29, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/AgentFactory.py", line 19, in main
    env = FactoryFloor(parameters['environment'])
KeyError: 'environment'

So it seems someone took over and modified agentconfig.yaml or removed it completely, breaking this example

Wouter1 commented 4 years ago

Trying to use factory_floor_complex.yaml instead

Wouter1 commented 4 years ago

That gives

warning: Debugger speedups using cython not found. Run '"/home/wouter/git/aienvs/venv/bin/python" "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/setup_cython.py" build_ext --inplace' to build.
pydev debugger: starting (pid: 3601)
Traceback (most recent call last):
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 2225, in <module>
    main()
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 2218, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 1560, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 1567, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/_pydev_imps/_pydev_execfile.py", line 25, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/wouter/git/aiagents/examples/AgentFactory.py", line 29, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/AgentFactory.py", line 19, in main
    env = FactoryFloor(parameters['environment'])
  File "/home/wouter/git/aienvs/aienvs/FactoryFloor/FactoryFloor.py", line 75, in __init__
    pos = item['pos']
TypeError: list indices must be integers or slices, not str
Wouter1 commented 4 years ago

The yaml file is incorrect/not matching the code. The code expects for each robot something like {'id':'robotname', 'pos':[x,y]}. The yaml file contains just [0,0] for the first robot.

Wouter1 commented 4 years ago

Fixed that part of the yaml file. Now we get

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/AgentFactory.py", line 29, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/AgentFactory.py", line 22, in main
    complexAgent = createAgent(env.action_space, env.observation_space, parameters['agents'])
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 32, in createAgent
    raise ValueError("Parameters must have key 'class' but got " + str(parameters))
ValueError: Parameters must have key 'class' but got {'iterationLimit': 500, 'treeParameters': {'explorationConstant': 10000, 'samplingLimit': 20}}
Wouter1 commented 4 years ago

The 'agents' value in the yaml file is this

    agents:
        iterationLimit: 500
        #timeLimit: 30
        treeParameters: 
            explorationConstant: 10000
            samplingLimit: 20

which does not even remotely resemble what is needed for the factory (it needs a class name etc)

Wouter1 commented 4 years ago

@czechows

Basically the yaml files contain a serialized object. What we do is (1) deserialize the yaml to a python dict (2) we have code snippets throughout the code picking parts from this dict (3) build python objects based on these picked parts. Basically we have a distributed hand-built parser that results in a nightmare scenario. You basically have to read through all the code to determine the proper format for your yaml file. Plus all problems you get with simplistic parsing like no /bad error messages. And lots of boilerplate code to fix these bad error messages.

I remember we looked into this before but did not find any suited deserialization mechanism for python.

Now I see that python 3.8.3 has a dataclass annotation. Maybe we can do something with that?
https://docs.python.org/3/library/dataclasses.html

czechows commented 4 years ago

@Wouter1 perhaps let's just fix outdated config files (adapt them to the new, working format).

These config files are not for deserialization but to provide arguments for the constructors/factories (which can do some non-trivial initializations based on these arguments, e.g. creating subclasses).

Wouter1 commented 4 years ago

agent_combined_config.yaml seems to contain something that matches what I reverse engineer in the code.

Wouter1 commented 4 years ago

@czechows yes I'm trying to fix them.But as said this is not nice and they are almost all broken.

Wouter1 commented 4 years ago

Exampel AgentFactory is fixed.

Wouter1 commented 4 years ago

DQNSingleExample.py does not work

sh: 1: netconvert: not found
CRITICAL:root:LDM start failed
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/DQNSingleExample.py", line 18, in <module>
    'gui': False
  File "/home/wouter/git/aienvs/aienvs/Sumo/GridSumoEnv.py", line 226, in __init__
    super().__init__(parameters=_parameters)
  File "/home/wouter/git/aienvs/aienvs/Sumo/SumoGymAdapter.py", line 78, in __init__
    self._observation_space = self._compute_observation_space()
  File "/home/wouter/git/aienvs/aienvs/Sumo/SumoGymAdapter.py", line 81, in _compute_observation_space
    self._startSUMO(gui=False)
  File "/home/wouter/git/aienvs/aienvs/Sumo/SumoGymAdapter.py", line 170, in _startSUMO
    self._sumo_helper = SumoHelper(self._parameters, self._port, self._seed)
  File "/home/wouter/git/aienvs/aienvs/Sumo/SumoHelper.py", line 25, in __init__
    assert(self.scenario_check(self.parameters['scene']))
  File "/home/wouter/git/aienvs/aienvs/Sumo/SumoHelper.py", line 44, in scenario_check
    self._net_file = os.path.basename(glob.glob(self.scenario_path + '/*.net.xml')[0])
IndexError: list index out of range
Exception ignored in: <bound method ldm.__del__ of <aienvs.Sumo.LDM.ldm object at 0x7fbf6df2bf28>>
Traceback (most recent call last):
  File "/home/wouter/git/aienvs/aienvs/Sumo/LDM.py", line 475, in __del__
    self.SUMO_client.close()
  File "/home/wouter/sumo/tools/traci/__init__.py", line 215, in close
    _connections[""].close(wait)
KeyError: ('',)
Wouter1 commented 4 years ago

The error message comes from searching for a non-existing file.

Wouter1 commented 4 years ago

The used 'scenario_path' is pointing to '/home/wouter/git/aienvs/aienvs/Sumo/../../scenarios/Sumo/grid_1x1' which is completely wrong. There is no Sumo directory in aienvs

There is a Sumo dir in aienvs, but there is no grid_1x1 in there

there is a aienvs/scenarios/Sumo/one_grid, maybe that's the one intended here

Wouter1 commented 4 years ago

the 'scene' parameter contains just "grid_1x1". And even worse, it's computed in GridSumoEnv, so it's not just a text value in a file. I can't just hack GridSumoEnv because it's used in other files, esp GridSumo.ipynb which I have no idea about.

I'll try renaming the one_grid into grid1x1 and see what happens

Wouter1 commented 4 years ago

@czechows

I'm trying to fix DQNSingleExample. One problem is that it uses aienvs.SumoHelper (so this is a different package that might be just pip install'ed). SumoHelper searches relative to its own file position. I happen to have SumoHelper inside Eclipse, so it givs a file inside aienvs. This is already weird, because the test I'm running is in aiagents. I don't know what would happen if you would have just pip installed aienvs.

Should we fix SumoHelper for re-use in other packages? Eg so that it looks in another place ? How? Or do you have a different idea about this?

Wouter1 commented 4 years ago

If I copy the directory aienvs/scenarios/Sumo/one_grid to aienvs/scenarios/Sumo/grid_1x1 then SQNSAingleExample seems to start. Unfortunately the map is incorrect, I get

Error: The edge 'e_0_1_l_1_1' within the route '1' is not known.
 The route can not be build.
Quitting (on error).
Wouter1 commented 4 years ago

Maybe I can find the old grid_1x1 files

czechows commented 4 years ago

@czechows

I'm trying to fix DQNSingleExample. One problem is that it uses aienvs.SumoHelper (so this is a different package that might be just pip install'ed). SumoHelper searches relative to its own file position. I happen to have SumoHelper inside Eclipse, so it givs a file inside aienvs. This is already weird, because the test I'm running is in aiagents. I don't know what would happen if you would have just pip installed aienvs.

Should we fix SumoHelper for re-use in other packages? Eg so that it looks in another place ? How? Or do you have a different idea about this?

How does it use SumoHelper?

For me it works (my config is aienvs, aiagents on PYTHONPATH and not installed via pip).

Is the problem in GridSumoEnv?

Wouter1 commented 4 years ago

@czechows do you know if we ever had a grid_1x1 sumo scenario folder? The DQNSingleExample needs it

czechows commented 4 years ago

Ah indeed I have this folder locally but not in the repo. Perhaps it is generated by GridSumoEnv?

Wouter1 commented 4 years ago

@czechows Yes I was just thinking that. But the example is in aiagents and generates files in aiagents. See my previous post on this. SumoHelper looks elsewhere...

Wouter1 commented 4 years ago

I'll skip this for now till we have an idea how to fix this.

czechows commented 4 years ago

Yes it is, in create_scenario. I think @JINKEHE added it to make scenario generation easier. @Wouter1 I think one more problem you had was sh: 1: netconvert: not found

https://github.com/INFLUENCEorg/aienvs/blob/master/aienvs/Sumo/GridSumoEnv.py

Wouter1 commented 4 years ago

GroupingRobotsExample.py

/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/GroupingRobotsExample.py", line 103, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/GroupingRobotsExample.py", line 55, in main
    qagent1 = learnEpisode(ENTITY1)
  File "/home/wouter/git/aiagents/examples/GroupingRobotsExample.py", line 78, in learnEpisode
    agent1 = RandomAgent("random", env)
TypeError: __init__() missing 1 required positional argument: 'observationspace'
Wouter1 commented 4 years ago

GroupingRobotsExample calls QAgent constructor incorrectly. I assume it needs env.action_space and env.observation_space now.

Wouter1 commented 4 years ago

QAgent docu says it has TWO parameters. However the code also shows a third called 'epsilon'.

As in other issues,default values should be used if lacking...

Wouter1 commented 4 years ago

Yes it now finishes learning. But at the end 

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/GroupingRobotsExample.py", line 103, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/GroupingRobotsExample.py", line 60, in main
    runQCoordinator(qagent1, qagent2, qagent3)
  File "/home/wouter/git/aiagents/examples/GroupingRobotsExample.py", line 97, in runQCoordinator
    qcoord = QCoordinator([qagent1, qagent2, qagent3], env)
  File "/home/wouter/git/aiagents/aiagents/multi/QCoordinator.py", line 40, in __init__
    raise ValueError("actionspace must be Dict but found " + str(actionspace))
ValueError: actionspace must be Dict but found <aienvs.GroupingRobots.GroupingRobots.GroupingRobots object at 0x7fd37edbf5c0>
Wouter1 commented 4 years ago

fixed. Now getting

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/GroupingRobotsExample.py", line 103, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/GroupingRobotsExample.py", line 60, in main
    runQCoordinator(qagent1, qagent2, qagent3)
  File "/home/wouter/git/aiagents/examples/GroupingRobotsExample.py", line 99, in runQCoordinator
    episode.run()
  File "/home/wouter/git/aienvs/aienvs/runners/Episode.py", line 51, in run
    obs, globalReward, done = self.step(obs, globalReward, done)
  File "/home/wouter/git/aienvs/aienvs/runners/Episode.py", line 32, in step
    actions = self._agent.step(obs, globalReward, done)
  File "/home/wouter/git/aiagents/aiagents/multi/QCoordinator.py", line 115, in step
    totalQ = totalQ + agent.getQ(observation, action1)
  File "/home/wouter/git/aiagents/aiagents/single/QAgent.py", line 83, in getQ
    return self._getQ(Hashed(state), action)
  File "/home/wouter/git/aiagents/aiagents/single/QAgent.py", line 98, in _getQ
    if action in self._Q[state].keys():
TypeError: unhashable type: 'collections.OrderedDict'
Wouter1 commented 4 years ago

@czechows

Ok but he never considered that this is called from another module I guess. How do we solve this?

I missed that error message. It does not even say error or so so I ignored it. Is it a warning? Is it a problem?

Yes it is, in create_scenario. I think @JINKEHE added it to make scenario generation easier. @Wouter1 I think one more problem you had was sh: 1: netconvert: not found

https://github.com/INFLUENCEorg/aienvs/blob/master/aienvs/Sumo/GridSumoEnv.py

Wouter1 commented 4 years ago

Checking MctsAggrExample.py Default config 

{'seed': None, 'max_steps': 40, 'environment': {'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}}
{'class': 'aiagents.multi.BasicComplexAgent.BasicComplexAgent', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robots', 'parameters': {'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robots', 'parameters': {}}, 'rolloutAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robots', 'parameters': {}}, 'timeLimit': 6, 'treeParameters': {'explorationConstant': 10, 'samplingLimit': 20}}}]}
No SLURM_JOB_ID found
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/MctsAggrExample.py", line 83, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/MctsAggrExample.py", line 61, in main
    complexAgent = createAgent(env, agent_parameters)
TypeError: createAgent() missing 1 required positional argument: 'parameters'
Wouter1 commented 4 years ago

@czechows 

How does it use SumoHelper?

The DQNSimpleExample calls on SumoHelper indirectly by using GridSumoEnv which calls SumoGymAdapter.

For me it works (my config is aienvs, aiagents on PYTHONPATH and not installed via pip).

For me is does not. I did several fixes (maybe you can pull my current version) and I now get

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/DQNSingleExample.py", line 18, in <module>
    'gui': False
  File "/home/wouter/git/aienvs/aienvs/Sumo/GridSumoEnv.py", line 226, in __init__
    super().__init__(parameters=_parameters)
  File "/home/wouter/git/aienvs/aienvs/Sumo/SumoGymAdapter.py", line 67, in __init__
    self._tlphases = TrafficLightPhases(tlPhasesFile)
  File "/home/wouter/git/aienvs/aienvs/Sumo/TrafficLightPhases.py", line 24, in __init__
    tree = ElementTree.parse(filename)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 1196, in parse
    tree.parse(source, parser)
  File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 586, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/home/wouter/git/aienvs/aienvs/Sumo/../../scenarios/Sumo/grid_1x1/grid_1x1.tll.xml'

Is the problem in GridSumoEnv?

I think so, check how GridSumoEnv computes the parameters just above the place in the stacktrace above

czechows commented 4 years ago

@Wouter1 do you have netconvert installed (should come with your sumo build)?

Wouter1 commented 4 years ago

netconvert is not installed here. import netconvert gives ModuleNotFoundError

czechows commented 4 years ago

netconvert is a SUMO binary not a python utility that should be compiled when you build your SUMO installation: https://sumo.dlr.de/docs/NETCONVERT.html It will probably take a while to figure what went wrong, so I would say leave this example as it is and proceed with the other things.

Wouter1 commented 4 years ago

@czechows yes but the error message is not complaining about netconvert and netconvert does not give any error so it's not this problem though we might run into that elsewhere.

But ok I skip this for now

Wouter1 commented 4 years ago

Back to the MctsAggrExample The problem is that the parameters do not have a 'simulator' containing a copy of the env parameters (as is specified in the doc)

czechows commented 4 years ago

@czechows yes but the error message is not complaining about netconvert and netconvert does not give any error so it's not this problem though we might run into that elsewhere.

But ok I skip this for now

I think what happened was: -netconvert did not create the network because was not found -no error was thrown because it was an external binary call (just a notification sh: 1: netconvert: not found, few stack traces back in the posts) -due to lack of generated network your error was thrown

Wouter1 commented 4 years ago

@czechows then that's a really poor message. It should FATAL at that point, or at least say SEVERE sh: netconvert not found

Wouter1 commented 4 years ago

It's pretty hard to get a copy of the env parameters inside the agent parameters. The agent parameters are read from a file agent_combined_config.yaml There MctsAgent (RobotAgent) the treeAgent and rolloutAgent have parameters that are currently {} but should thus contain a 'simulator' apparently, and this simulator should contain the entire environment parameter set.

env parameter is this

dict: {'seed': None, 'max_steps': 40, 'environment': {'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}}

Wouter1 commented 4 years ago

I'll put it to {} to start with because I suspect more problems down the line here. How can the subagents launch another env?

Wouter1 commented 4 years ago

Trying to add the robots list to the params but yaml parser is complaining

ERROR:root:while parsing a block collection
  in "/home/wouter/git/aiagents/examples/./configs/agent_combined_config.yaml", line 21, column 11
expected <block end>, but found '?'
  in "/home/wouter/git/aiagents/examples/./configs/agent_combined_config.yaml", line 41, column 11
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/examples/MctsAggrExample.py", line 83, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/MctsAggrExample.py", line 40, in main
    agent_parameters = getParameters(agent_filename)
  File "/home/wouter/git/aienvs/aienvs/utils.py", line 14, in getParameters
    parameters = yaml.safe_load(stream)['parameters']
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/__init__.py", line 162, in safe_load
    return load(stream, SafeLoader)
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/constructor.py", line 41, in get_single_data
    node = self.get_single_node()
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/composer.py", line 110, in compose_sequence_node
    while not self.check_event(SequenceEndEvent):
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/home/wouter/git/aienvs/venv/lib/python3.6/site-packages/yaml/parser.py", line 393, in parse_block_sequence_entry
    "expected <block end>, but found %r" % token.id, token.start_mark)
yaml.parser.ParserError: while parsing a block collection
  in "/home/wouter/git/aiagents/examples/./configs/agent_combined_config.yaml", line 21, column 11
expected <block end>, but found '?'
  in "/home/wouter/git/aiagents/examples/./configs/agent_combined_config.yaml", line 41, column 11
czechows commented 4 years ago

I'll put it to {} to start with because I suspect more problems down the line here. How can the subagents launch another env?

So they launch another env as their "simulator" of the real env. It is true that everything gets really messy if you want to add a config file for all these simulators.. But they can just reuse the real environment parameters for the purposes of this example, like e.g. in https://github.com/INFLUENCEorg/aiagents/blob/master/experiments/IJCAI2020/MctsExperiment.py lines 53-55

Wouter1 commented 4 years ago

Apparently a few whitespaces got lost before "timeLimit" in the yaml file....

Wouter1 commented 4 years ago

ok now we have

warning: Debugger speedups using cython not found. Run '"/home/wouter/git/aienvs/venv/bin/python" "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/setup_cython.py" build_ext --inplace' to build.
pydev debugger: starting (pid: 9379)
Default config 
{'seed': None, 'max_steps': 40, 'environment': {'steps': 10, 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}], 'tasks': ['random', 'random'], 'P_action_succeed': {'LEFT': 0.9, 'RIGHT': 0.9, 'ACT': 0.8, 'UP': 0.9, 'DOWN': 0.9, 'robot1': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot2': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}, 'robot3': {'LEFT': 0.2, 'RIGHT': 0.2, 'ACT': 0.2, 'UP': 0.2, 'DOWN': 0.2}}, 'P_task_appears': 1.0, 'N_task_appears': 4, 'allow_robot_overlap': True, 'allow_task_overlap': True, 'map': ['11111', '11111', '11111', '11111']}}
{'class': 'aiagents.multi.BasicComplexAgent.BasicComplexAgent', 'parameters': {}, 'subAgentList': [{'class': 'aiagents.single.mcts.MctsAgent.MctsAgent', 'id': 'robots', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}]}, 'treeAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robots', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}]}}}, 'rolloutAgent': {'class': 'aiagents.single.RandomAgent.RandomAgent', 'id': 'robots', 'parameters': {'simulator': {'fullname': 'aienvs.FactoryFloor.FactoryFloor.FactoryFloor', 'robots': [{'id': 'robot2', 'pos': 'random'}, {'id': 'robot1', 'pos': 'random'}, {'id': 'robot3', 'pos': 'random'}]}}}, 'timeLimit': 6, 'treeParameters': {'explorationConstant': 10, 'samplingLimit': 20}}}]}
No SLURM_JOB_ID found
>>> self.getSpace()
Dict(robot1:Discrete(5), robot2:Discrete(5), robot3:Discrete(5))
Traceback (most recent call last):
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 48, in createAgent
    obj = klass(parameters['id'], actionspace, observationspace, class_parameters)
  File "/home/wouter/git/aiagents/aiagents/single/RandomAgent.py", line 18, in __init__
    self.action_space = full_action_space.get(agentId)
  File "/home/wouter/git/aienvs/aienvs/gym/DecoratedSpace.py", line 187, in get
    return self.getSubSpace(id)
  File "/home/wouter/git/aienvs/aienvs/gym/DecoratedSpace.py", line 163, in getSubSpace
    return DecoratedSpace.create(self.getSpace().spaces[id])
KeyError: 'robots'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 48, in createAgent
    obj = klass(parameters['id'], actionspace, observationspace, class_parameters)
  File "/home/wouter/git/aiagents/aiagents/single/mcts/MctsAgent.py", line 65, in __init__
    self._treeAgent = createAgent(self._simulator.action_space, self._simulator.observation_space, parameters['treeAgent'])
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 50, in createAgent
    raise ValueError(classname + " failed on __init__:") from error
ValueError: aiagents.single.RandomAgent.RandomAgent failed on __init__:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 2225, in <module>
    main()
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 2218, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 1560, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/pydevd.py", line 1567, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/wouter/eclipse/plugins/org.python.pydev.core_7.1.0.201902031515/pysrc/_pydev_imps/_pydev_execfile.py", line 25, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/wouter/git/aiagents/examples/MctsAggrExample.py", line 83, in <module>
    main()
  File "/home/wouter/git/aiagents/examples/MctsAggrExample.py", line 61, in main
    complexAgent = createAgent(env.action_space, env.observation_space, agent_parameters)
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 54, in createAgent
    subAgentList.append(createAgent(actionspace, observationspace, subAgentParameters))
  File "/home/wouter/git/aiagents/aiagents/AgentFactory.py", line 50, in createAgent
    raise ValueError(classname + " failed on __init__:") from error
ValueError: aiagents.single.mcts.MctsAgent.MctsAgent failed on __init__: