Freeze on barbarian turn.

[tu-79]

1. Mod version (i.e Date - 4/23):

2018-01-21

2. Mod list (if using Vox Populi only, leave blank): FlagPromotions InfoAddict Improved City View

3. Error description:

Freeze on barbarian turn.

4. Steps to reproduce (optional):

Logs.zip AutoSave_0216 AD-1530.zip This time I have a recent savegame. I just spread religion in one aztec city and killed a hunnic dromon.

Supporting information: Please note that you can attach .zip files by dragging-and-dropping them. If possible, zip up all supporting data and post that way.

1. Log files (always attach your Logs folder, located at My Documents/My Games/Sid Meier's Civilization 5. Make sure you have enabled logging before experiencing an error! Go here to find out how: http://forums.civfanatics.com/showthread.php?t=487482):

2. Save game (always attach a save that was made a turn before the error; located at My Documents/My Games/Sid Meier's Civilization 5/ModdedSaves):

3. CvMiniDump.dmp file (attach if experiencing a game crash. Located at Program Files/Steam/steamapps/common/Sid Meier's Civilization V):

4. Screenshots (optional):

[Infixo]

Do you actually have any freeze/crash problems? Because I have none. Couldn’t even reproduce anything from 2 saves attached here. @LoneGazebo You tracked something yesterday. Was it something else?

[LoneGazebo]

@Infixo I'm not sure, I haven't been able to replicate since yesterday.

[ricanuck]

Got the barbarian freeze yesterday, 26-1, ONLY playing CP core plus EUI 1.28g (which never presented a problem). So you can discard VP extra components, the bug is part of the core CP. I am convinced the bug is inside Infixo's changes, but I have no proof (call it dev intuition if you want, or ignore it... your call).

Edit: LUAs I use from the CP core that do not conflict with EUI:

5\MODS(1) Community Patch\LUA

27/01/2018 05:53 PM

. 27/01/2018 05:53 PM .. 08/10/2015 11:42 AM 10,324 Bombardment.lua 23/01/2017 04:44 PM 92,008 CultureOverview.lua 09/10/2015 08:10 PM 155,102 CultureOverview.xml 02/06/2015 05:46 PM 33,228 DeclareWarPopup.lua 02/06/2015 08:27 PM 14,902 DeclareWarPopup.xml 15/07/2015 06:20 PM 21,863 Demographics.lua 28/10/2016 03:39 PM 66,216 DiscussionDialog.lua 28/10/2016 03:39 PM 10,536 DiscussionDialog.xml 19/10/2016 12:11 AM 64,232 EspionageOverview.lua 02/12/2015 09:58 PM 37,909 EspionageOverview.xml 12/01/2018 03:04 PM 16,681 GPList.lua 09/09/2016 12:37 PM 5,456 NaturalWonderPopup.lua 24/02/2016 07:53 PM 5,992 PuppetCityPopup.lua 28/07/2017 12:11 PM 29,635 ReligionOverview.lua 17/11/2015 06:21 PM 17,483 TradeRouteHelpers.lua

[LoneGazebo]

@ricanuck I don't think Infixo has touched any of those LUA files.

[Infixo]

@ricanuck Please provide a save file.

I am working with (6a) EUI and (6b) Non-EUI components plus (6c) as copy. No Lua changes in core CP. Even Civilopedia is unchanged in CP.

Also, none of my changes (but one) contain any loops. And that exception was made on Jan 8th to a function that is literally never called because VP doesn't use OrPreReqTechs (findPathLength, I made it to fix Wonder Planner).

[LoneGazebo]

@Infixo @ilteroi i'm leaning towards it being a bug in the functions mentioned above

getNumUnitsOfAIType getFirstUnitOfAITypeSameTeam getFirstUnitOfAITypeOtherTeam

or in the rework of the tacticalmap zones. I don't really see anything else in the DLL that could cause it.

[Infixo]

Yeah, I "vote" for getFirstUnitOfAITypeSameTeam. It is the only function called around barbarians that was changed in 1-21.

CvUnit* pRescueUnit = pNewTarget->getFirstUnitOfAITypeSameTeam(BARBARIAN_TEAM, UNITAI_SETTLE);
if (!pRescueUnit)
pRescueUnit = pNewTarget->getFirstUnitOfAITypeSameTeam(BARBARIAN_TEAM, UNITAI_WORKER);
if (pRescueUnit)
m_iUnitToRescue = pRescueUnit->GetID();

I have no idea WHY it could cause problems however. Except that NULL checks. Also, ALL other while loops in CvPlot are constructed as if (cond) do...while(cond) Then again - idk why it would matter, but these are differences.

[Infixo]

And while we're at this, there is a minor bug, 2nd call should be UNITAI_WORKER:

// bonus for captured civilians (settlers and workers, not missionaries)
int iBonus = pPlot->getNumUnitsOfAIType(UNITAI_SETTLE) * 3 + pPlot->getNumUnitsOfAIType(UNITAI_SETTLE) * 2;

[LoneGazebo]

Yup re: typo in second one. The thing is, the 'barbarian turn' thing is misleading - stuff that happens on the global/game level takes place during that phase as well, so it's not technically barbarian. It's either something global or something related to the human player's turn start.

[ricanuck]

I am continuing the same game now, just reloaded from the last save yesterday, 50 turns and nothing happened, could not reproduce the freeze of last night... this is one of "those", nasty mfing hard to catch buggies... so no savegame. OK, so if not LUA, last changes to DLL... again, I'm using Core only.

[tu-79]

To add something. G is rigth, it occurs during Barbarian turn, but at the moment there wasn't any barbarians, unless some city state was suffering an invassion I wasn't aware of.

[ilteroi]

ok guys i see a lot of speculation here. let's recap:

• last known good version was 1-14
• it's a freeze, callstacks as far as available did not point to modded code
• probably a deadlock, not an infinite loop (check cpu load next time)
• it's not reproducable
• it seems to occur during turn change
• it occurs without additional mods

correct?

what about EUI? do we have any feedback from non-EUI players?

anyway, it seems we need more testing. the fastest way would be to create multiple versions and ask people on the forums to test them and report back. to minimize confusion, it's probably best to have only two versions and iterate if necessary:

• DLL 1/14 with latest LUA
• DLL 1/26 with old LUA

[MoiMagnus]

I would add that some players reported a correlation with auto-save. (ex: https://www.reddit.com/r/civvoxpopuli/comments/7tffig/game_crashes_every_10_turns_during_autosave/). But it might be a different issue than this one.

[Infixo]

The post from bigcat88 strongly suggests that 1-14 was ok and 1-21 was not. https://forums.civfanatics.com/threads/new-beta-version-january-26th-1-26.627497/page-4#post-15011166

[Infixo]

@ilteroi Can you explain deadlock using an example? Are they easy to happen in Civ5 code?

[bigcat88]

It is a deadlock. I cant press alt+tab, only can open task manager or windows button. No CPU usage after that.

[ilteroi]

@Infixo : a deadlock is a common multithreading pitfall, when two threads are a waiting for each other endlessly. they are nasty because they are often latent and become manifest only later after subtle timing changes in completely unrelated code.

so it's no use trying to find a bug in our modded code. we can only try to find the change that triggered the deadlock and build a workaround.

[LoneGazebo]

@ilteroi there were some unit promotions added by another user in 1-14 - mind taking a peek and seeing if the code is okay? I’m away all day.

[Infixo]

:) I am quite familiar with theory and was asking about Civ5 situation. I am not familiar with the code as you, so basically e.g. which parts are multithreaded? Is it only engine or the DLL as well? what internal ‚resource’ or operation could possibly trigger deadlock? Etc. Basically things specific to Civ5.

[LoneGazebo]

The UI is threaded on its own thread relative to the game DLL. So if there are any routines being called by the UI that break this link it can cause problems.

[LoneGazebo]

Even if the lua isn’t changed, you did modify a bunch of vanilla lua hooks in the dll that would affect this.

[MoiMagnus]

Have you example were the freeze appeared out of an autosave ? The user in (ex: https://www.reddit.com/r/civvoxpopuli/comments/7tffig/game_crashes_every_10_turns_during_autosave/) seemed to only have freeze during autosave.

(And has the same settup : VP with EUI and no other mods)

[Infixo]

People are actually saying that autosave is done correctly, they can reload and continue. It has to be something done directly after autosave, like end-turn things or preparation for the next turn, when „barbarian turn” icon is still on the screen.

[MoiMagnus]

People are actually saying that autosave is done correctly, they can reload and continue. It has to be something done directly after autosave, like end-turn things or preparation for the next turn, when „barbarian turn” icon is still on the screen.

Yes, the user was saying that it was succeeding, but that it only froze turns of autosave. (but it might be a biais of observation)

The fact that the autosave succeed does not mean that it is not guilty. It can be a failure of the "autosave" to give back the hand to the main thread after finishing the save. (In case auto-save also have its own thread)

[HungryForFood]

Are you still looking for a save file? I have one where all you need to do is to click next turn, and the only save game incompatible mod was Historical Religions Complete.

An observation I had was that it did not freeze every time. However, if I moved the camera around and spam clicked the ground after clicking next turn, the freeze was much more likely.

[Infixo]

Can we post a debug version with AI-perf.csv log enabled and info from OutputDebugString (not sure if the latter is possible)? It would give more info about how far the game progresses and where it stops.

@HungryForFood I would like one. I have not yet had the freeze on my comp.

[ilteroi]

ah, ok. basically there are two main threads in civ, on is the UI thread and the other is the GameCore (DLL) thread. the problem is that all lua is executed by the UI thread, but it does make calls into the DLL, which is not threadsafe at all (except for a few places, check calls to GetGameCoreLock / ReleaseGameCoreLock)

however, there are most definitely other locks used in the 3D engine and so on, so this is not the only place where you can run into problems.

[bigcat88]

can u get and use EUI from 1-14 version? if yes - which folders replace? if no - I will try no eui version for test but it will be pain for me :)

[Infixo]

@bigcat88 The problem happens even when using only (1) CP. Here are the files: (6a) from VP 1-14.zip

[bigcat88]

Maybe there are 2 different problems. Even when I turn autosaves off, it is not ok sometimes. I just want to test first with old eui, then if all is still bad - without eui. Just be sure that problem is in dll. thanks - will continue my hanging game and write if it will hang/crash

[LoneGazebo]

3919

[LoneGazebo]

Is JFD's comment at end a confirmation that it is the LUA? I think it might be. Anyone else with freezes getting errors in their lua.logs?

[bigcat88]

No, just look after freeze at lua.log - no errors. My last 4 lines is usual: [708856.921] TurnProcessing: Hiding TurnProcessing [708933.718] WonderRace: Oda Nobunaga is building Taj Mahal in Kyoto (4 turns) [708943.906] WonderRace: Nebuchadnezzar II is building Taj Mahal in Babylon (5 turns) [708943.906] WonderRace: Notification fired: [COLOR_RED]Nebuchadnezzar II[ENDCOLOR] has begun constructing Taj Mahal in Babylon

I am in progress of looking what causes this...

[Infixo]

@bigcat88 Can you post the save file that causes the freeze?

[bigcat88]

I trying to find the turn in game where it will cause a freeze every time or maybe place where will be 1 freeze every 5-6 turns, to test changed eui/old eui/no eui. And it is a problem. I even set autosaving on each turn to catch them. Maybe I need start a new game just for test without random seed. In current game I can play for 90 minutes without any freezes and then 2 freezes on 2 turns. Still cant find what causes this(after last freeze I turned off WonderRace). Patient :)

[HungryForFood]

@Infixo Here's the save. It doesn't happen every time, but you can move the camera around after clicking next turn to make it more likely. And the logs too if you're interested.

My mod list in the order the game displays, all of which except VP and Historical Religions Complete claims to be save-game compatible. I managed to recreate the freeze when using just VP and HR Complete to load.

(1) Community Patch (v. 88) Historical Religion Complete (BNW or GK) (v. 45) Ingame Editor (v. 39) Quick Turns (v. 10) Tectonic Map Script (v. 4) (2) Community Balance Overhaul (v. 13) (4) C4DF - CBP (v. 11) (3) City-State Diplomacy Mod for CBP (v. 27) (5) More Luxuries - CBO Edition (5-14b) (v. 155) (6a) Community Balance Overhaul - Compatibility Files (EUI) (v. 1) UI - Show XP In Military Overview (v. 2)

I'm using the 1/26 version of VP. Autosave was on and set to every turn. I was playing in windowed mode.

[LoneGazebo]

Loaded it and trying it now.

[LoneGazebo]

CvGameCore_Expansion2.zip

Test for me please. The slight hang at the end of the barb turn is your game autosaving. Otherwise it should be working. IF this works...the bug was my fault. If it doesn't help...someone else is to blame! :)

[HungryForFood]

Nope, freeze is still there. What do you mean by slight hang though, did it proceed to next turn without having to relaunch Civ 5?

Curiously the AI turn went a bit differently, Pocatello declared friendship even though he didn't before. Does swapping the DLL like this affect the RNG somehow?

I was playing the game while waiting for your reply, and from my observation, it never froze as long as I do nothing while the AI turns are processing. No camera movement, no clicking, etc.

Same goes for the save. No camera move = no freeze. Violently moving the camera = freeze.

[LoneGazebo]

No freeze for me period if autosave is set to 1 per 10 turns.

[Albertolona]

@Infixo

Have game settings been eliminated?

There was a problem before where animations where causing problems and it wasn't being reproduced since modders were running with it switched off, so perhaps try with different settings.

[HungryForFood]

So I turned off autosave and loaded my save again, and I couldn't get the freeze.

Yep, definitely something to do with autosave. I turned it back on, and can trigger the freeze again.

[LoneGazebo]

@ilteroi any possibility that the method by which your new tacticalmapanalysis stuff is saved is causing an issue with saving/loading the game? I noticed it's a new kstream format. Twice I loaded hungry's game and it froze on me while loading. That's definitely new.

[ilteroi]

what do you mean? i didn't change the savegame format (afaik)

also, when it freezes, what does the callstack say?

[Infixo]

@Albertolona Which settings do you mean? Quick movement / quick combat? Something else?

[Infixo]

@HungryForFood Why do you have like a half a million of these in GameCore.log: [616915.218] LOAD ERROR: Type not found for hash: 4294967295

Edit. The timestamp of the file is much earlier, so probably from a different session. But weird, anyway.

[HungryForFood]

I have no idea what it means. Nothing seems off with my game.

[ryanmusante]

For me, to trigger a freeze is to "quick-save" and than immediately click "exit to desktop" from the menu. I don't know if this is because of my wine+steam configuration or because of the aforementioned bug.

[bigcat88]

Hm, played for 7 hours. Got two crashes(not repeatable, looks like crash happens after autosave. first crash 100% was after autosave, now will start civ and look - was game saved or not after second)(not freezes). And first crash was as I remember invalid eip register(I have no debug symbols for civ5 so cant say more, and it is just a minidump). Second the same, but in stack there present "right" address of return(so corrupted stack by 1 parameter?) that points to civ5 code(u will see if load in windbg). Attach 2 dumps, second dump was last. Logs folder only for second dump, I forgot to save log folder for first crash.

Logs2.zip

Dumps.zip

[bigcat88]

But if look at message_debug log that lines tells me that autosave was present. And I looked in autosave folder, there is present autosaves with time creation equal to dump file. [766123.843] DBG: UpdateMoves() : player 63 Crom running AutoMission (AI_TACTMOVE) on Lancer id=15309 [766123.843] DBG: CheckPlayerTurnDeactivate() : auto-moves complete for Crom [766123.843] DBG: changeNumGameTurnActive(-1) m_iNumActive=0 : setTurnActive() for player Crom [766128.953] DBG: Game Turn 516 [766128.968] DBG: changeNumGameTurnActive(1) m_iNumActive=1 : setTurnActive() for player 0 Catherine

@Gazebo Can you post a debug symbols(pdb file) for dll(1-26)(if u have it) and say where I can download debug symbols for CivilizationV_DX11 module(if Firaxis provide them) Just interesting what is that function. I never compiled civ5 dll, and I am too lazy to run VM with studio to do that(and I am not sure that I have the same studio version, and if not I need to install new vm for specified version)