there is only one state_init value despite there being multiple agents and so the line
if use_lstm:
action[agent_id], state_init, logits = agent.compute_action(
state[agent_id], state=state_init, policy_id=policy_map_fn(agent_id))
has every agent overwrite the state_init value of other agents. There should be one state_init per agent.
there is only one state_init value despite there being multiple agents and so the line if use_lstm: action[agent_id], state_init, logits = agent.compute_action( state[agent_id], state=state_init, policy_id=policy_map_fn(agent_id)) has every agent overwrite the state_init value of other agents. There should be one state_init per agent.