search

inoryy / reaver

Reaver: Modular Deep Reinforcement Learning Framework. Focused on StarCraft II. Supports Gym, Atari, and MuJoCo.
MIT License
554 stars 89 forks source link

Windows support (multiprocessing / shared memory) #17

Closed inoryy closed 5 years ago

inoryy commented 5 years ago

There's no os.fork() in Windows, so it seems that when I launch a new worker it re-creates ProcEnv object, which no longer has access to MultiProcEnv shared memory reference. Need to either rewrite how I give the reference or temporarily implement message-based communication instead for Windows.

inoryy commented 5 years ago

@King-Of-Knights Reaver should be working on Windows in 2.0.4 (run pip install --upgrade reaver). I've verified it trains to baseline results on MoveToBeacon (although much slower than on Linux).

I've also added detailed installation instructions on wiki: https://github.com/inoryy/reaver-pysc2/wiki/Windows

King-Of-Knights commented 5 years ago

@inoryy Roger! In this time, there is no traceback, but the marine just won't move. I could even order him to move to the beacon by my mouse. That's a little strange. [•_•]

inoryy commented 5 years ago

@King-Of-Knights that might be normal, I've mentioned the situation on wiki:

During training you should see your agent experiment with various commands - left/right clicking, queuing commands and so on. It might also seem that your agent is doing nothing - that is also fine, unless it lasts more than 200-300 updates. In these cases what is most likely happening is that the agent is trying out less noticeable commands (e.g. control group assignments, using Stop, Halt, and so on).

He might be stuck a lot longer if you're training only on one environment, so try running with at least 2-3. You could also try re-running a bunch of times - should see him immediately move as well sometimes.

King-Of-Knights commented 5 years ago

@inoryy OK! I will try and report it back!

inoryy commented 5 years ago

I've re-tested it some more on a fresh Windows - seems to work ok with 4 envs (aside from relatively slow speed). I'll close this issue.

If you encounter any problems please ensure you haven't missed any steps in instructions first and then open a new ticket with as much details as possible.