semitable / lb-foraging

Level-based Foraging (LBF): A multi-agent environment for RL
MIT License
151 stars 64 forks source link

[Bug] Observation out of the range of low and high of the obs space #9

Closed GoingMyWay closed 2 years ago

GoingMyWay commented 2 years ago

Dear authors, thanks for developing such an awesome library for MARL research. I tried to run the env Foraging-4s-8x8-2p-2f-v2 and found that the obs seems to be out of the low and high range of the obs space.

In RLlib, it will raise errors. Is it a severe issue? Can I clip the obs after getting the observations from env.step?

You can put the following before returning the nobs for checking.

for obs in nobs:
    assert self.observation_space[0].contains(obs)

Please correct me if I am wrong. Thanks in advance.

GoingMyWay commented 2 years ago

@semitable Hi, I guess the bug maybe because the initial low should be [-1, -1, 0] in a partially observable setting. In the code, I found it is in https://github.com/semitable/lb-foraging/blob/master/lbforaging/foraging/environment.py#L132

min_obs = [-1, -1, 0] * max_food + [0, 0, 1] * len(self.players)

In the code, if it is the partially observable setting, the value can be -1 and 0: https://github.com/semitable/lb-foraging/blob/master/lbforaging/foraging/environment.py#L392

for i in range(len(self.players)):
    obs[self.max_food * 3 + 3 * i] = -1
    obs[self.max_food * 3 + 3 * i + 1] = -1
    obs[self.max_food * 3 + 3 * i + 2] = 0

Could you please check it? Can I create a PR?

semitable commented 2 years ago

Hi @GoingMyWay

Thanks for spotting this. Sounds right to me. I will be happy to accept a PR.

GoingMyWay commented 2 years ago

Hi @GoingMyWay

Thanks for spotting this. Sounds right to me. I will be happy to accept a PR.

Hi @semitable, thanks. I made a PR.