Closed richardliaw closed 5 years ago
Updated to break the example into separate files for clarity. I think this is probably good to merge; perf should presumably be similar to https://github.com/ray-project/ray/pull/3542 since the code is largely unchanged.
It seems like the total number of SC2 processes created during either run_ppo
or run_qmix
scripts is num-workers + 1
, where one of the environments is not being used but takes extra memory.
Any idea why this happens?
Could you please also state the additional dependencies in the README file, e.g. gym, tf?
Ah, the reason we create n+1 envs is that one is needed for the driver process. It seems like we need to create the StarCraft2 env there to get its action and observation space. However we never call any methods on the env.
Is it possible to defer creation of the StarCraft process until reset() is called? That would allow access to obs and action space without extra overheads.
Added TF dep in readme. The rllib package already includes the rest of the deps.
Done in 91761f1
Awesome, I think that should fix the extra memory use issue then.
This didn't solve it. Apparently, reset() is being called anyway for the first env instance which launches the SC2 process. You can make use of class methods to get action/observation/state spaces w/o calling the reset().
@samvelyan will be fixed in the next RLlib release https://github.com/ray-project/ray/pull/3810
cc @ericl