Closed GoingMyWay closed 4 years ago
Hi, in our paper, we we run the algorithms for 24 episodes with each agent performing greedy action selection during testing. The percentage of episodes where the agents defeat all enemy units within the time limit is called the win rate. This metric is automatically saved in your tensorboard event file. Refer to https://arxiv.org/abs/1902.04043 for more details on this metric.
Hi, in our paper, we we run the algorithms for 24 episodes with each agent performing greedy action selection during testing. The percentage of episodes where the agents defeat all enemy units within the time limit is called the win rate. This metric is automatically saved in your tensorboard event file. Refer to https://arxiv.org/abs/1902.04043 for more details on this metric.
Thanks, when training, you use dense rewards and when evaluation, you use winrate (win or loose) right?
Yes. Like the standard setting in RL, rewards are used to train agents. Win rate is just a standard metric to report your final performance in the decentralized StarCraft benchmark.
Yes. Like the standard setting in RL, rewards are used to train agents. Win rate is just a standard metric to report your final performance in the decentralized StarCraft benchmark.
Thanks.
@chaovven Hi, for training the SMIX(\lambda) and QMIX, how many CPUs and how much GPU memory do we need? Also, to get the best results, how many hours training do we need? Recently, I am trying to run your code, but I have limited CPUs and GPUs.
@chaovven Hi, for training the SMIX(\lambda) and QMIX, how many CPUs and how much GPU memory do we need? Also, to get the best results, how many hours training do we need? Recently, I am trying to run your code, but I have limited CPUs and GPUs.
Hi, running the simplest map 3m for 2 million timesteps would spend nearly 3.5 hours on my PC, with nearly 1g memory usage in my GPU and 5g of system memory (batch_size_run
=4). more complex scenarios with increasing agents (e.g., 8m, 3s5z) may cost 8~10 hours or more. In your case, you'd better set batch_size_run
to 1 to save your memory. Also, you can disable GPU by setting use_cuda
to False.
ps: my CPU is Intel(R) Xeon(R) W-2133 CPU @ 3.60GHz
and GPU is GeForce GTX TITAN X
.
@chaovven Hi, for training the SMIX(\lambda) and QMIX, how many CPUs and how much GPU memory do we need? Also, to get the best results, how many hours training do we need? Recently, I am trying to run your code, but I have limited CPUs and GPUs.
Hi, running the simplest map 3m for 2 million timesteps would spend nearly 3.5 hours on my PC, with nearly 1g memory usage in my GPU and 5g of system memory (
batch_size_run
=4). more complex scenarios with increasing agents (e.g., 8m, 3s5z) may cost 8~10 hours or more. In your case, you'd better setbatch_size_run
to 1 to save your memory. Also, you can disable GPU by settinguse_cuda
to False.ps: my CPU is
Intel(R) Xeon(R) W-2133 CPU @ 3.60GHz
and GPU isGeForce GTX TITAN X
.
Thanks, 8-10 hours is not that long compared with Atari games. I think 5 CPU cores and one P100 are enough for SC training, do you think so? Another question is did you try the mini-games from DeepMind?
@chaovven Hi, for training the SMIX(\lambda) and QMIX, how many CPUs and how much GPU memory do we need? Also, to get the best results, how many hours training do we need? Recently, I am trying to run your code, but I have limited CPUs and GPUs.
Hi, running the simplest map 3m for 2 million timesteps would spend nearly 3.5 hours on my PC, with nearly 1g memory usage in my GPU and 5g of system memory (
batch_size_run
=4). more complex scenarios with increasing agents (e.g., 8m, 3s5z) may cost 8~10 hours or more. In your case, you'd better setbatch_size_run
to 1 to save your memory. Also, you can disable GPU by settinguse_cuda
to False. ps: my CPU isIntel(R) Xeon(R) W-2133 CPU @ 3.60GHz
and GPU isGeForce GTX TITAN X
.Thanks, 8-10 hours is not that long compared with Atari games. I think 5 CPU cores and one P100 are enough for SC training, do you think so? Another question is did you try the mini-games from DeepMind?
Yes, though more training time may be needed. I didn't try the mini-games.
@chaovven Hi, for training the SMIX(\lambda) and QMIX, how many CPUs and how much GPU memory do we need? Also, to get the best results, how many hours training do we need? Recently, I am trying to run your code, but I have limited CPUs and GPUs.
Hi, running the simplest map 3m for 2 million timesteps would spend nearly 3.5 hours on my PC, with nearly 1g memory usage in my GPU and 5g of system memory (
batch_size_run
=4). more complex scenarios with increasing agents (e.g., 8m, 3s5z) may cost 8~10 hours or more. In your case, you'd better setbatch_size_run
to 1 to save your memory. Also, you can disable GPU by settinguse_cuda
to False. ps: my CPU isIntel(R) Xeon(R) W-2133 CPU @ 3.60GHz
and GPU isGeForce GTX TITAN X
.Thanks, 8-10 hours is not that long compared with Atari games. I think 5 CPU cores and one P100 are enough for SC training, do you think so? Another question is did you try the mini-games from DeepMind?
Yes, though more training time may be needed. I didn't try the mini-games.
Good, thanks.
Hi, in your paper, I fund there is a winrate metric to evaluate the performance of algorithms. How can I manually calculate the value in StarCraft environment?