Open HJasperson opened 2 years ago
Hey @HJasperson , thanks for filing this issue. Your code looks totally fine and I wouldn't know of any reason, why this shouldn't work. I have a few questions, though. I would expect every single ray.get()
call in your step method to hang for roughly 500s as all parallelized tasks will sleep for exactly 500s and only then return. The ray.get
call is blocking, so the environment will NOT continue until all slowFunctions
have returned their results.
There are a couple more flaw in your repro. Please make sure to test your reproduction scripts to show the reported error, before opening the issue.
ray.get([self.slowFunction.remote(self.someProperty[i]) for i in range(self.numAgents)])
does not work with slowFunction being a method as the self argument is missing. I'm also pretty sure leaving this as a method (instead of an external function) will serialize the entire env as it's being passed as arg to a remote call (this might explain the issue with the hanging).self.someProperty = np.zeros(self.numAgents, dtype=np.int32)
should fix this issue.self._spaces_in_preferred_format = None
to your env's constructor in order to make sure our env pre-checked knows that the obs and action spaces are per individual agent.the environment will NOT continue until all slowFunctions have returned their results
The problem is that it NEVER continues. I know how long this method should take and the environment hangs even after an excessive amount of time has passed.
I'm also pretty sure leaving this as a method (instead of an external function) will serialize the entire env
That was my thought as well, but calling an external method gives the same result (i.e. env hangs).
The script is just to illustrate what I was talking about. I am unwilling to share my actual code with you until it is published.
What happened + What you expected to happen
I have a multi-agent environment where the reward is calculated per agent. The reward function is fairly intensive so I'd like to run it in parallel for the various agents. Whenever I try something like below, the script hangs at the first call to the remote function.
I've also tried something similar inside a custom action distribution with the same result. Moving the slow function out of the class has no impact.
Versions / Dependencies
python 3.9.12 ray 1.12.0 rhel 7.9
Reproduction script
Issue Severity
Medium: It is a significant difficulty but I can work around it.