Create multiple environment Error

[jcwleo]

Hi. I have used the multiprocessing module to create multiple environments. However, when one environment becomes Done, all other environments are also done. How do you handle this issue? thank you. :)

[Kautenja]

Just to clarify, by done you mean the done flag is set to True for all the environments simultaneously? If this is the case, could you provide a simple code snippet that reproduces the issue?

My thought is that this results from the way the communication pipeline between the emulator and the Python client is setup. The Unix pipe name is setup once per Python shell session which certainly would cause an issue when parallelizing internally through either the multiprocessing or the threading package. It's a simple fix, but I'd like to have some code to reproduce the problem so I can be sure the subsequent solution resolves the issue.

[Kautenja]

I've implemented the fix and written some tests in the tools directory. Version 1.1.2 should resolve any problems with the threading or multiprocessing packages and is live now on PyPi. Closing the issue for now. If you find that this doesn't resolve your issue please comment and I'll reopen it.

[jcwleo]

@Kautenja https://gist.github.com/jcwleo/f5daa30a40ea8ae14190a02bc7425236 this is simple code. thank you.

[jcwleo]

@Kautenja i updated gym-super-mario-bros 1.1.2. but still appear same issue :(

[Kautenja]

ah I see the problem now from the sample. Calling make from the parent process results in all the 4 environments having the same PID and thread name and thus the pipe to the emulator is the same for all of them. It appears that standard Atari Envs work with your configuration so I'll take a look at their source to check how they initialize their pipes to Stella. For now, if you can refactor your code to call make from within the subprocess, it will work. i.e.

import gym_super_mario_bros
import multiprocessing as mp

class Env(object):
    def __init__(self, env):
        self.env = env
        self.env.reset()
        self.done = False

    def run(self):
        for step in range(5000):
            if self.done:
                state = self.env.reset()
                self.done = False
            state, reward, self.done, info = self.env.step(self.env.action_space.sample())

def runner():
    env = Env(gym_super_mario_bros.make('SuperMarioBros-v0'))
    env.run()

procs = []

for idx in range(4):
    proc = mp.Process(target=runner)
    proc.start()
    procs.append(proc)

for proc in procs:
    proc.join()

However, I'll likely have the new fix out within the next couple of hours so stay posted. Reopening issue.

[Kautenja]

Alright so the named pipes now use the uuid package to ensure that the pipe name is unique irrespective of where a new environment is initialized (using host ID and current time). Version 1.1.3 on PyPi is live with these changes and works on both your and my example codes. Please report back if the issue somehow still persists on your end though. Closing issue.