Diplo AI has difficulty estimating human military strength

[RecursiveStar]

The diplomacy AI has difficulty estimating a human's effective military strength using functions like GetPlayerMilitaryStrengthComparedToUs. I previously added code to make the AI more aggressive when players were weak and doing certain other things, like hoarding Wonders or expanding recklessly (mainly through the IsEasyTarget function), but this proved to make all the AIs gang up on the human player like a hive mind, and I think some of the problem is still present.

I think the problem is threefold: 1) Humans tend to be better at war than the AI, especially if they're playing on higher difficulties, but this is not accounted for.

2) Humans can defend against large assaults by using a smaller but more skilled military force and/or by exploiting terrain and chokepoints and/or with better tactics and superior memory. The AI does not account for this and considers humans with smaller forces weak, even if they've fended off a dozen wars so far without losing a single city.

3) The AI, especially on higher difficulties, has more units and unit supply, which causes them to consider humans weak.

Any ideas on how this problem could be resolved? @ilteroi @LoneGazebo

[ilteroi]

you could track the results (warscores?) of previous wars and use that to adjust the approach to the next war?

[RecursiveStar]

@ilteroi Any suggestions on how I could do that in a numerical sense? The change in approach from considering the human too weak is a side effect; the main problem is that the AI has trouble determining the actual military strength of a human adversary, and this is relevant in more functions than just the main approach function.

[ilteroi]

we could also add a "killcount" for each unit and increase the unit power for those with high killcount. on the other hand that number would be somewhat correlated with xp which is already tracked. so maybe simply increase the promotion contribution in CvUnit::getPower() for a start?

[RecursiveStar]

The AI also gets bonus XP from difficulty added to every unit, sans kills, more units, and a greater amount of XP added from combat, so I'm not sure that'd be effective as a solution.

A killcount sounds like an interesting idea.

It doesn't necessarily need to be a change to the military power ranking, just a change in how the AI perceives human players (for the three reasons mentioned above), particularly on higher difficulties. I think the AI once had a modifier to the human player's strength estimate on higher difficulties to account for the human likely being more experienced; not sure if that's still the case.

[RecursiveStar]

CrazyG on the forums has also raised the point that humans on the defensive can wear their enemies down with a barrage of ranged attacks, and according to him, ranged units seem to contribute less to military power (I have not tested this, but they do get less XP from combat, so he may be on to something there).

EDIT: There's also the cross-domain issue, where the AI has a large military strength because they have a whole bunch of ships, and consider a human adversary on land weaker even though that adversary might have overwhelming strength. I don't think there's an easy answer to this one, but perhaps the AI could evaluate land and sea strength separately somehow when assessing a target.

[MoiMagnus]

What about a killcount at player level? Or something like a MMR or Elo rating? Which would be a public information to everyone. [If possible, accessible to human players through the interface, similarly to opinion modifiers you could have "their warriors are feared through the whole world" kind of text].

Example of ranking implementation I came up with:

Players start with a martial rating of 1000 (arbitrary value, to be tweaked), and cannot go lower than this value. Barbarian MR is 1000 and cannot increase nor decrease.

Every time player A with a martial rating MRA kill a (non civilian) unit of production cost C from a player B of martial rating MRB, we update both MR with "V = min(C 2, C MRB / MRA) ; MRA += V ; MRB -= V; MRB = max(1000,MRB)". So in other words if you kill significantly more than you lose units, you will be known as particularly good at fighting. But farming units from easy target will become less and less efficient.

[Note: I'm using production costs, because I think it usually represent better the strategic value of a unit than its CS or RCS, and it also allows for early wars (including barbarian hunting), to count less than late game wars.]

Then, whenever the player A need to evaluate the strength of player B, it applies a modifier based on the MR, for example with the formula "StrB MRB/MRA". (taking the min with "StrB 2" and the max with "StrB/2" to avoid degenerate cases).

[RecursiveStar]

@MoiMagnus I think an idea like that has a lot of potential. What do you think, @ilteroi? Maybe something like this could be added and worked into the strength evaluation functions?

[ilteroi]

go for it :)

[RecursiveStar]

@MoiMagnus JamesNinelives on the forums has pointed out that cities gained/lost should also be a factor in such a rating - a very good point. :)

As this is your idea, I was wondering if you had any suggestions? Probably just a larger change based on city size and/or strength and a bump for the capital, I'm thinking.

[ilteroi]

you realize that you are about to recreate the warscore calculation?

[MoiMagnus]

I think there already is a function for computing the "economic power" of a city (which should be used for warscore, warmonger penalty and co). So unless it is unusable, we could just multiply it by a well-chosen factor.

@ilteroi Yes, it is indeed not that far from warscore. The main difference being that it factors the "previous wars" (including against barbarian), so that continuously attacking weak targets does not give you as many points as tackling a major powers.

[RecursiveStar]

@ilteroi Whereas war score applies to individual wars, this rating would apply across the entire game, including when not at war. And you yourself suggested I use war score memory as a solution. :)

[ilteroi]

i guess what i meant to say was, don't start from scratch, instead extend the old 1vs1 warscore with a 1vsN warscore

[LoneGazebo]

Simpler way to do this would be to simply record the war scores and/or peace treaty values of each war as soon as a player makes peace. Average this post-war score against the number of wars they've waged and boom, simple tracker for war efficacy.

[RecursiveStar]

Hmm, that could work. AI could also compare a player's war efficacy compared to the global average, using either method, and adjust strength estimates accordingly. I'll write and test some code for this.

[LoneGazebo]

This also has the benefit of readability - no secret code here, we can track peace score val and warscore in the UI to see if it all matches up.

[RecursiveStar]

@LoneGazebo There are some potential problems I see. For one example:

Suppose the player starts a war with Civ B, and completely devastates them, finishing the war with a score of 100.

While doing this, three other civs declare war on the player, but he ignores them to focus on Civ B or because they're too far away, etc. Those wars are eventually finished with scores of 0.

100 + 0 + 0 + 0 = 100 / 4 = 25

The player's neighbor, Civ C, then sees a rating of 25 when this is actually a significant underestimation of power.

Moi Magnus's suggestion does not have this problem since it's based on units/cities killed and lost and respective strengths, without dividing by number of wars.

[LoneGazebo]

Then don't divide by number of wars. :) Just do a total calculation and find the average amongst all players.

G

[RecursiveStar]

That would still be 25 in my example, though. :P

100 total score / 4 players gone to war with = 25

Edit: Or if there were 8 players total in the game and that was the metric, 12.5. If everyone started at 0 that'd be distinctive enough I guess, although on maps with large amounts of players it might provide an inaccurate strength estimate.

[LoneGazebo]

Don’t average # of wars, just look at mean and see who is at/above mean.

[MoiMagnus]

I think what G means is to just have the military rating be the sum of all warscores (or peace values?) from previous wars. That's an easy solution to try at first.

I'd note that it does lose what I wanted to compute initially, which is unit efficiency: if a player as double number of units, and always win wars by trading 2 against 1, I wanted this to be reflected by having a poor military rating (despite the victories through raw economic supremacy).

Another similar situation would be "AI declare wars and send waves and waves of units, the human defends without losing a lot of unit, then the war stabilise since the human doesn't have any interest in counter-attacking, after a lot of turns, the AI give up and a white peace is signed". The AI should learn from this war that attacking the human is not a good idea, but the warscore doesn't reflect this.

[RecursiveStar]

Ah, I see what you mean now. The average of the totals does sound a lot simpler.

Was about to point out what MoiMagnus just pointed out in the third paragraph, though. Warscore is heavily dependent on offensive conquests, and if there's a skilled defensive player who manages to fend off a dozen wars without losing a single city (or even a single unit), warscore wouldn't reflect that well.

This is especially the case because AI at higher difficulties gets a lot of Production and supply bonuses and has more units, so the death of each one counts for little.

And sending a ton of units to their deaths, giving said defensive player more XP and yields, isn't very strategic.

[MoiMagnus]

I'm thinking of solution which might be easier than my suggestion, but still getting some of my intention:

Every time some warscore is won/loss, unless from decay, it is immediately added/removed from the military rating. This include when some warscore should have been won but cannot since the maximum of 100 is already reached. And since we don't include decay, it shouldn't be that much in favour of offensive wars.

[RecursiveStar]

That doesn't resolve the problem of the AI underestimating the human player because its larger army sizes mean less warscore gained/lost from unit kills, even many of them, though.

Edit: Warscore generally works well for reflecting the progress of an individual war, not so much at reflecting a player's actual military talent based on their actions in combat unless they go on a conquering spree. That's why I thought your first idea was better.

[MoiMagnus]

I have to admit I'm not completely familiar with how the warscore is computed, so I can't really compare. You're saying that for computing a warscore, it is not the number/strength of unit killed that count (which we want), but their number/strength compared to the total strength of the civ (which we don't want)? So killing a knight will be not be worth the same amount of warscore depending on how many knight the civ own?

[RecursiveStar]

When a civ loses a unit, it counts for less if that civ is strong and has many units, and counts for more if that civ is weak and has few units. The AI can produce more units and replace them more quickly than the human on higher difficulties, meaning many units could be killed without warscore changing all that much, if no offensive action is taken, but that's not a reflection of the defender's actual military skill.

[MoiMagnus]

Oh, I see, so yes, this "correction" that make sense for warscore is something we don't want at all. In fact, if anything, we want the contrary:

A civ with many unit that lose one is very bad, since it should have used its superiority to prevent this death, while a civ with few unit that lose one is normal, since it was in numerical inferiority.

[RecursiveStar]

Yeah, that's the problem I'm trying to fix. :)

Re: unit loss, I'm not sure I agree there, because a civ with many units and greater economic strength can afford to lose more units in achieving a goal sometimes, and a defender could just have a really well-fortified position that requires sacrifice to reach.

I think there's actually a tactical AI strategy ("steamroll") that specifically has the AI make some less safe moves to wear down the enemy, because they know they have superior strength.

This is tricky.

[MoiMagnus]

So I have a suggestion, which while I consider it "not as good" as my initial suggestion (because it doesn't use production cost, and doesn't use the ratio MRB/MRA), can be done rather quickly and is probably much more understandable by someone which didn't read this thread entirely and would just look at the code:

The function "ChangeWarValueLost" from CvDiplomacyAI is called everytime the warscore need to be modified. However, an important note is that this value is not yet a proportion of the initial value the civ had (the quotient by the initial value is done in the warscore computation latter). Meaning that for this function "losing a knight" should be the same for every player, disregarding their actual strength.

So you could high-jack this function so that it also update the military rating, making sure to only take in account "positive changes" (the only negative changes are the decay implemented to make sure the warscore go toward zero if nothing happen, so you definitely don't want to take it in account).

Note that there is already a similar code at this position since this it is in ChangeWarValueLost that the code update the "we're happy because you damaged our common foe".

[RecursiveStar]

I was looking at that code myself. It may not be a perfect solution, but it'd be an improvement for the time being. That value could be stored and military strength could be modified up or down based on the % difference from the average (I'd cap it at -50% below and +100% above, and see how that goes).

I'll add that code and see if players report an improvement in the AI targeting humans.

[LoneGazebo]

I think what G means is to just have the military rating be the sum of all warscores (or peace values?) from previous wars. That's an easy solution to try at first.

I'd note that it does lose what I wanted to compute initially, which is unit efficiency: if a player as double number of units, and always win wars by trading 2 against 1, I wanted this to be reflected by having a poor military rating (despite the victories through raw economic supremacy).

Another similar situation would be "AI declare wars and send waves and waves of units, the human defends without losing a lot of unit, then the war stabilise since the human doesn't have any interest in counter-attacking, after a lot of turns, the AI give up and a white peace is signed". The AI should learn from this war that attacking the human is not a good idea, but the warscore doesn't reflect this.

Warscore tracks this data already - there's a set of functions that do this. You can just tap into and record that data.

[RecursiveStar]

@LoneGazebo I'll send a pull request soon, but that's what I'll do, thanks for the suggestions.

I added a memory value to CvPlayer (m_iMilitaryRating), which will go up every time war value is lost by an opponent, and down every time war value is lost - tapping into the existing functions. Barbarians, razing cities, and war damage decay will not update the rating.

Then diplo AI will then calculate the average rating and each player's % difference from the mean when computing effective military strength, adjusting up by 1-100% or down by -1 to -50%.

I think that should solve the problem, or at least improve it substantially.