[Feature Request] ID remapper in DLL before caching the database entries

[azum4roll]

Currently, we have a Remapper.sql in the (2) mod that reorders the ID of each table so there is no skipped ID due to row deletion.

However, this is only guaranteed to work for VP. Modmods frequently delete from tables, and CP-only doesn't have this for performance reasons. Similarly, we can't have a trigger for this either.

The best way to handle this is to do it in the DLL, after all database entries (except Defines and CustomModOptions, due to PostDefines and CustomModPostDefines being a thing) are finalized. We just need someone familiar with database code to do it.

Rough draft of the algorithm:

1. CREATE TABLE IDRemapper (ID integer PRIMARY KEY, Type text); 2. for each table with ID and Type that's not IDRemapper: (CvDllDatabaseUtility::ValidateGameDatabase() has existing code that does this)

• INSERT INTO IDRemapper (Type) SELECT Type FROM <table name> ORDER BY ID;
• UPDATE <table name> SET ID = (SELECT IDRemapper.ID - 1 FROM IDRemapper where <table name>.Type = IDRemapper.Type);
• DELETE FROM IDRemapper;

3. DROP TABLE IDRemapper;

This function should be placed before the ValidateGameDatabase() call in CvDllDatabaseUtility::CacheGameDatabaseData(). The ID validation in CvDllDatabaseUtility::ValidateGameDatabase() and Remapper.sql can be removed.

[azum4roll]

There are a couple of tables that don't have IDs starting at 0:

• MultiplayerOptions: cached but doesn't seem to be used in the DLL. The IDs start at 1, but the table is missing MPOPTION_SIMULTANEOUS_TURNS which should be ID 0 if it were to be used. The DLL has the closed enum of MultiplayerOptionTypes instead. Seeing as some of the Description and Help text keys in the table aren't defined, the table probably isn't used in UI either. Either way, we can try deleting the table contents and test if an MP game can still be started.
• ReplayDataSets: only used in Lua (the replay graphs on victory screen), but the Types correspond to hardcoded setReplayDataValue() calls in CvPlayer::GatherPerTurnReplayStats(). It's very likely safe to have IDs start at 0 instead of 1 (IDs aren't used in Lua), but we need to test.
• HistoricRankings: only used in Lua (the ranking tab on victory screen). I don't have the game here to check, but the IDs shouldn't be used.
• FakeFeatures: the IDs deliberately start at 200 to avoid clashing with normal Features. I'm not sure if this is still needed. Are FakeFeatured used anywhere other than Civilopedia?
• CivilianAttackPriorities: supposed to be used as a reference for the Units.CivilianAttackPriority column, but it isn't. It's just a copy of the CivilianAttackPriorityTypes closed enum otherwise. Need to check if it's used by some Lua in the base game.
• GreatWorkSlots: used in a lot of UI. Despite having an open enum in the DLL, the contents are still hardcoded in CvTypes::AcquireTypes() instead of using CvDllDatabaseUtility::PrefetchCollection() to fetch from the database. I can't be sure if making its IDs start at 0 would break anything.
• GreatWorkClasses: I don't even understand how the enum is populated for this one. The code is a mess and keeps querying the database instead of using the cache.
• GreatWorkArtifactClasses: see GreatWorkSlots above.
• GreatWorks: see GreatWorkClasses above. Blame this all on whoever implemented great works in Civ5. It's definitely written by a different developer from everything else.
• Leaders: I think the DLL part is safe, but will need to test if pregame UI breaks.