edbeeching / godot_rl_agents

An Open Source package that allows video game creators, AI researchers and hobbyists the opportunity to learn complex behaviors for their Non Player Characters or agents
MIT License
942 stars 69 forks source link

Observation: The two structures don't have the same nested structure #19

Closed themars2011 closed 2 years ago

themars2011 commented 2 years ago

Hello,

I am trying to implement the RL Agents into my project and I am facing a very strange issue.

First structure: type=dict str={'obs': [1, 1120]}

Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.74411386, -0.83825755], dtype=float32))])

More specifically: Substructure "type=list str=[1, 1120]" is a sequence, while substructure "type=ndarray str=[ 0.74411386 -0.83825755]" is not
Entire first structure:
{'obs': [., .]}
Entire second structure:
OrderedDict([('obs', .)])
Result for PPO_godot_1186f_00000:
  date: 2022-06-20_20-52-10
  experiment_id: e62b8b1e8d5349189952f56cbb07b08f
  hostname: DESKTOP-CQGL49S
  node_ip: 172.23.164.16
  pid: 19821
  timestamp: 1655751130
  trial_id: 1186f_00000

However, my Godot console output indicates that the structure is indeed the same:

getting command line arguments
Waiting for one second to allow server to start
trying to connect to server
02
performing handshake
handshake complete
[1, 1120]
resetting all agents
[1, 1120]
[-1, 1120]
server disconnected, closing

This is my code:

# The observation of the agent, think of what is the key information that is needed to perform the task, try to have things in coordinates that a relative to the play
# return a dictionary with the "obs" as a key, you can have several keys

    var obs := []
    # other player data
    if (get_parent().get_child_count() > 1):
        for child in get_parent().get_children():
            if (child != get_node(".")):
                if (child.position.x > position.x):
                    obs.append(1)
                else:
                    obs.append (-1)

                obs.append(int(position.distance_to(child.position)))

    print(obs)

    return {
        "obs": obs
       }

func get_obs_space():
    # typs of obs space: box, discrete, repeated
    return {
        "obs": {
            "size": [len(get_obs()["obs"])],
            "space": "box"
           }
       }

The only way I can get it to work at all is by appending a 0.0 first and then just appending the direction (the 1 or -1). Everything else I tried just simply does not work and results in this error.

Edit: I figured it out. The observations need to be between -1 to 1. It would be great if this was written prominently somewhere for newbies like myself :)

edbeeching commented 2 years ago

I'm glad that you figured it out. I will aim to improve the docs about the observations needed to be normlized to be between -1,1. Let me know if you have further issues.