Open clemenjuan opened 5 months ago
/I found that line 573 in env_runner_v2.py was commented, so I uncommented it and now it seems to work, but I think that should be checked.
values_dict = {
SampleBatch.T: episode.length, # Episodes start at -1 before we
# add the initial obs. After that, we infer from initial obs at
# t=0 since that will be our new episode.length.
SampleBatch.ENV_ID: env_id,
SampleBatch.AGENT_INDEX: episode.agent_index(agent_id),
# Last action (SampleBatch.ACTIONS) column will be populated by
# StateBufferConnector.
# Reward received after taking action at timestep t.
SampleBatch.REWARDS: rewards[env_id].get(agent_id, 0.0),
# After taking action=a, did we reach terminal?
SampleBatch.TERMINATEDS: agent_terminated,
# Was the episode truncated artificially
# (e.g. b/c of some time limit)?
SampleBatch.TRUNCATEDS: agent_truncated,
SampleBatch.INFOS: infos[env_id].get(agent_id, {}), # this line was previously commented
SampleBatch.NEXT_OBS: obs,
}
@clemenjuan Thanks for filing this issue. During the the release of ray-2.10.0 and the actual release we have changed a lot in the coder of the EnvRunner API
. Could you give the actual version a try and see, if the error prevails?
What happened + What you expected to happen
I am experiencing a persistent issue with my custom multi-agent environment in RLlib, where the
infos
dictionary is not being found, leading to aKeyError: 'infos'
. This error arises during the processing of observations in the_process_observations
function inenv_runner_v2.py
.The error occurs consistently across different configurations and even after ensuring the environment complies with the expected structures for observations, rewards, terminations, truncations, and
infos
.Versions / Dependencies
Reproduction script
Steps to Reproduce
infos
are handled per agent.KeyError: 'infos'
occurs in_process_observations
.Code Snippets
Issue Severity
High: It blocks me from completing my task.