fix(wzl): fix vdn mixer to avoid Q value dim mismatch with reward

[zerlinwang]

command

./train_mpe_vdn.sh

problem

Traceback (most recent call last): File "train/train_mpe.py", line 192, in main(sys.argv[1:]) File "train/train_mpe.py", line 177, in main total_num_steps = runner.run() File "/home/zerlinwang/Projects/off-policy/offpolicy/runner/rnn/base_runner.py", line 190, in run self.train() File "/home/zerlinwang/Projects/off-policy/offpolicy/runner/rnn/base_runner.py", line 272, in batch_train_q train_info, new_priorities, idxes = self.trainer.train_policy_on_batch(sample) File "/home/zerlinwang/Projects/off-policy/offpolicy/algorithms/qmix/qmix.py", line 164, in train_policy_on_batch Q_tot_target_seq = rewards + (1 - dones_env_batch) self.args.gamma next_step_Q_tot_seq RuntimeError: The size of tensor a (32) must match the size of tensor b (800) at non-singleton dimension 1

reason

next_step_Q_tot_seq dim error

solution

return agent_q_inps.sum(dim=-1).view(-1, 1, 1) -> batch_size = agent_q_inps.size(1) return agent_q_inps.sum(dim=-1).view(-1, batch_size, 1, 1)

result