search

oxwhirl / smac

SMAC: The StarCraft Multi-Agent Challenge
MIT License
1.07k stars 226 forks source link

Regarding Relative Representation of Health, Shields, Energy and Cool-down #23

Closed nasimrahaman closed 4 years ago

nasimrahaman commented 4 years ago

Hey folks. First things first, great work on the library! We found it easy to adapt to our needs.

This is regarding one of the things we observed, and deem worth sharing: namely the normalisation of health, shields, energy and cool-down. It appears that these are normalised by their maximum value, and while I see why it's a reasonable design decision, it distorts the game specified notion of hit-points.

For instance, a Colossus has 200 health points, while a Stalker has 80. When normalised by max health, a delta of 0.06 in the (relative) health corresponds to 5 health points for a Stalker, but 12 for a Colossus. This might be significant loss of information for an agent, given the fact that unit attacks are measured in damage per second (e.g. delta in hit points per second, ignoring splash damage).

One solution is to simply scale health-points by a fixed number (e.g. 1/45, such that a healthy Marine has health 1). This is appealing because it allows a (hypothetical) agent to compare the health of two or more different unit types simply by considering the difference between the corresponding inputs (same applies for shields, energy and cool-down). Further, SC2 has rich pairings between unit types, in the sense that some unit types are particularly effective against (or vulnerable to) other unit types. Having absolute measures might help the agent better exploit these.

While this issue is just a FYI, it might be worth considering to give the user the option to specify the normalisation (defaulting to normalisation by max, if need be).

Thanks. :)

tabzraz commented 4 years ago

Thanks for taking a look at this, its definitely something we'll keep in mind if we change the setup for a future version of SMAC.

As part of the observation that each agent sees regarding the enemy units, their unit type should be included which should allow them to distinguish between different unit types.

I'd prefer to not supply extra options to change the normalisation, so that people can easily compare results from one paper to another without checking the nitty-gritty details of the SMAC setup.