search

oxwhirl / smac

SMAC: The StarCraft Multi-Agent Challenge
MIT License
1.08k stars 228 forks source link

fix sz enemy unit_type id bug #111

Closed zzq-bot closed 1 year ago

zzq-bot commented 1 year ago

From the ally perspective, stalker_id=1935(-1935), zeolot_id=1936(-1936)-> [1, 0]/[0, 1] in obs/state; while the enemy obs/state takes stalker_id=74-73, zeolot_id=73-73 -> [0, 1]/[1, 0].

samvelyan commented 1 year ago

Thanks for pointing this out. I wouldn't call this a "bug". Even if the [0,1]-[1,0] order is different for vectors including allied and enemy units in observations, it will not change anything on the side of a NN-based agents who choses actions based on these inputs, unless you want to provide some inductive biases to the agents manually.

Given that many models have been trained and saved using the current observation space by the community, it would be wrong for me to update the code at this point and make it hard for them to use the latest version of this library, considering that the proposed change doesn't make any tangible difference.

zzq-bot commented 1 year ago

Thank you for your comment. I agree that this will not affect most MARL algorithms in the community. However, I encountered this issue when I tried to provide some inductive biases to the agents manually for multi-task training as you suggested (aligning obs/state dim across ‘marine’ and ‘sz’ tasks). Moreover, some multi-task MARL algorithms like UPDeT[1], MATTAR[2], etc. use transformer to achieve population-invariant functions and process enemy/ally obs features via self-attention. This might cause problems if the same obs_key/value/query NN extractor is used for both enemy and ally.

I agree that changes do not have to be made, but I suggest adding some comments in the starcraft2.py to clarify this point.

[1] Hu, Siyi, et al. "Updet: Universal multi-agent reinforcement learning via policy decoupling with transformers." arXiv preprint arXiv:2101.08001 (2021). [2] Qin, Rongjun, et al. "Multi-agent policy transfer via task relationship modeling." arXiv preprint arXiv:2203.04482 (2022).

samvelyan commented 1 year ago

Thanks for providing context and references. Happy to merge a PR which adds comments in the starcraft2.py without any changes to the observation space! Feel free to reopen the PR and make changes.