search

X2CommunityCore / X2ModBuildCommon

An improved XCOM 2 mod build system
MIT License
5 stars 5 forks source link

Switched to DLC cooking functionality #46

Closed Xymanek closed 3 years ago

Xymanek commented 3 years ago

Demo: https://github.com/WOTCStrategyOverhaul/CovertInfiltration/tree/a443cb9646eb7cb2e9d83c9ed292c71f02817220

Xymanek commented 3 years ago

Overview of how this works:

This is using the "dlc cooking" functionality built into the cooker by passing the -DLCName= argument. This causes the cooker to use a copy of the shipped GPCD (instead of starting from scratch), which allows properly reusing base game TFCs. It also fixes gfxCommon, startup and native packages issues since the cooker loads them but doesn't cook - this allows us to get rid of "first cook" logic as well as build the mod in debug with no issues.

Deciding what to cook

Standalone SF

The cooker treats all XCOM 2 War of the Chosen SDK\XComGame\Content\(Mods|DLC)\[DLCName]\**.upk packages as Standalone SF and then uses the file timestamp to decide if cooking is needed or can be skipped for a specific one. There is one exception to this rule - the very first one alphabetically is always skipped (I didn't manage to find any reasoning either via logs or procmon). To counteract this, I put the actual packages into Content\Mods\[DLCName]\Standalone and create a guard package at Content\Mods\[DLCName]\aaaaaa\TEMP_DlcCookIteratorGuard_[DLCName].upk, which is guaranteed to be iterated first. This package is duplicated from EmptyPackage in X2MBC and contains a single instance of ObjectReferencer with the array being empty.

Maps

By default, the cooker will attempt to cook all maps it's aware of, dlc cooking or not, which is about 2.5k maps with around 15GB of output. Since this is obviously undesirable, I pass -skipmaps but this makes it skip the mod maps as well. To force it to cook a map, its name needs to be passed on the command line. However, unlike with the previous approach of cooking the "game", in dlc cooking mode, passing a map on the command line trumps all other checks, causing it to be recooked even if nothing has changed. To prevent this waste, I implemented a rudimentary timestamp-based check

Scripts

While possible, the benefit is minimal and there are associated gotchas so there is no automated way of doing so right now

Mod project layout

The ContentForCook folder has been split into 3 subfolders:

The iteration/inclusion of packages in the Standalone folder is done by the cooker itself, while a similar approach is replicated by X2MBC for the Maps\**.umaps. Packages living outside of these folders will not be visible to the cooker.

This layout is designed to be intuitive (given the requirement for standalone packages) and naturally guiding the mod developers towards preventing unnecessary duplication of assets in their cooked packages.

ContentOptions.json remains, but sfStandalone and sfMaps options are removed (replaced by file iteration).

Drawbacks

There is no dlc-specific GPCD (every cooker invocation starts from the shipped one) so it doesn't know what textures are already included in your TFCs - editing a package with textures, even if the textures were not changed, will cause the textures to be dumped again into your TFCs. Fortunately the existing X2MBC cleaning functionality takes care of that (e.g. before building a version that will be published)

The map up-to-dateness check currently doesn't properly consider dependencies. I have some ideas how to approach this, but I don't see it as a huge blocker

Xymanek commented 3 years ago

maybe the addition of a ModBuddy task

I implemented the "rebuild" task/option

this function is getting quite large, would it make sense to split it up some?

Agreed, with a caveat - IMO it needs to be split into its own class (e.g. like #43 does), since otherwise the variables will need to be prefixed with assetsCook and a lot of them have quite lengthy names already. In any case, I don't see this is as an immediate priority