Modpack extension/inheritance (e.g. survival vs creative variants, base optimisation mod set)

[ChristopherHaws]

I am searching for a way to simply the management of my server which contains a creative and survival world which each have slightly different mods installed.

Pain Points

• Keeping track of what each mod does can get hard when packs get large (100+ mods)
• Keeping the configuration files in sync between two modpacks that are 90% identical is a challenge

Option 1: Modpack references

One option would be to allow for modpacks to reference other modpacks. To reference another modpack, you could run the command packwiz ref|reference add [path-or-url-to-another-pack.toml-file] which would add an option to your pack.toml file:

[option]
references = [
    "https://modpack-url/pack.toml",
    "../common/pack.toml"
]

Conflict Resolution: Packwiz should then use last in wins conflict resolution. For example, if https://modpack-url/pack.toml has v1 of a mod and ../common/pack.toml has v2 of a mod, v2 should be used. If ../common/pack.toml has v2 of a mod and the current modpack has v3 of a mod, v3 should be used. This gives the current modpack the ability to override any of the mods from the base pack if needed.

Pros

• Can reference modpacks from another location
• Can organize mods into smaller modpacks, for example optimization mods, quality of life mods, etc could each be their own pack that all get rolled up into one larger pack.

Cons

• Have to deal with conflict resolution
• If the other modpack goes away, yours is then broken
• If a referenced pack updates, your back could break

Option 2: Mod tags and multiple pack.toml files

Another option would be to allow for modpacks to add tags to mods directly and allow the inclusion or exclusion of mods based on the configuration in the pack file. If there are no tags, then the mod is always included.

For example, given the following mod files:

# ./mods/fabric-api.pw.toml
name = "Fabric API"
filename = "fabric-api-0.67.0+1.19.2.jar"
side = "both"

# ./mods/deathlog.pw.toml
name = "DeathLog"
filename = "deathlog-0.2.11+1.19.jar"
side = "both"
tags = ["survival"]

# ./mods/worldedit.pw.toml
name = "WorldEdit"
filename = "worldedit-mod-7.2.12.jar"
side = "client"
tags = ["creative"]

# ./pack-survival.toml
name = "survival"
author = "chaws"
version = "1.0.0"
pack-format = "packwiz:1.1.0"

[options]
acceptable-game-versions = ["1.19", "1.19.1", "1.19.2"]
tags-include = ["survival"]

# ./pack-creative.toml
name = "creative"
author = "chaws"
version = "1.0.0"
pack-format = "packwiz:1.1.0"

[options]
acceptable-game-versions = ["1.19", "1.19.1", "1.19.2"]
include-tags = ["creative"]

With this setup, if you configure packwiz-installer to point to pack-survival.toml you will get FabricAPI and DeathLog. If you configure packwiz-installer to point to pack-creative.toml you will get FabricAPI and WorldEdit.

Pros

• All of the mods in your pack(s) live in the same repo and are versioned together
• No chances of conflicting versions or files

Cons

• Mods are not organized into groups (this could be solved by storing the category from modrinth/curseforge in the metadata and having a command to export a report of all the mods installed grouped by category)
• Not reusable

[comp500]

As I mentioned on Discord, I've considered adding inheritance systems before and I'm definitely open to discussion.

Option 1: Modpack references

I've considered this, but it has problems; as you mentioned updates could break your pack. For this reason I'd likely require version pinning of referenced packs (the current architecture of packwiz-installer would need pack.toml to include the hash of the referenced pack.toml too, though that might be changed), although in some cases it would be useful to have updates be automatically handled (e.g. nightly build used in multiple variations of testing packs).

MCUpdater has an <Import> tag similar to this option that is quite powerful; Forge <1.13 and Fabric (and any loader without install-time patching) can be added to a pack simply by importing an auto-generated pack with the loader installation metadata - Fabric hosts this as an API endpoint.

One thing to consider is that any "override" system needs to merge files that are semantically the same (so that newer versions override older versions, despite them not having the same file/metadata names); this will likely require reading of mod loader metadata for mods.

Option 1a: Git submodules

As part of my quest to make packwiz more Git-native, I've wanted to add better support for Git submodules. These come with fetching and version pinning built in! To make them more useful though, you would definitely need some way of making the sub-packs create files in the parent directory; which would be a useful feature for organisation of mods folders regardless (see https://github.com/packwiz/packwiz/issues/78).

Maybe the pack.toml could double as the sub-pack root file, which can configure default settings (e.g. mod side) as well as be referenced from a parent pack root file (which can tell it to use the parent directory)... just need to make sure it isn't too complicated 🙃

Option 2: Mod tags and multiple pack.toml files

I reallly like the idea of being able to "tag" files - I was considering something similar for creating options/presets that aren't tied to individual files (so you could enable/disable a mod as well as it's configuration, or have sets of mods you can disable/enable at once). Maybe a pack could force-enable/disable/hide tags from an inherited/included pack? This could easily be combined with git submodules for reusability and grouping.

Option 3: Layering

This is the main option I'd planned to implement, as mentioned in https://github.com/packwiz/packwiz-installer/issues/38, which involved dealing with merging "layers" only at the packwiz-installer level, and having the packs to be installed from provided to the installer as a list - where each layer overrides files in the previous layer.

Pros

• It would be simpler to implement, particularly as the packwiz CLI wouldn't need to deal with it whatsoever
• It would also allow more user customisation, e.g. a set of cosmetic/optimisation mods that can be dropped on top of any pack or swapped out
• Doesn't have to deal with cyclic dependencies/duplicates/etc

Cons

• This customisation might be a bit complicated for most users
• Could introduce issues with incompatible/duplicated dependencies
• Updates could still break your pack (only suitable for rolling release)
• Requires changing configuration to add/remove layers

[ChristopherHaws]

I was considering something similar for creating options/presets that aren't tied to individual files (so you could enable/disable a mod as well as it's configuration, or have sets of mods you can disable/enable at once)

This is something I would definitely use. I was thinking about opening a separate issue about config file management. Right now I put both server and client configs together in my git repo, but that has issues, such as:

• Making sure that I don't check in any "secrets" such as SQL passwords, seeds, etc.
• If I want users to be able to update their configuration, I need to make sure I mark that config file as preserve = true, however this means that the server won't get any changes to those files anymore. Some mods allow you to use environment variables to override settings in their config files which I use as much as I can so that I can set server specific config values using my docker compose file, but not every mod has this option.
• Clients can see config files for mods that aren't installed client side

If it would be something you would be interested in adding as a first class feature in packwiz, this could be possible be achieved by allowing packs to override specific items in config files. This would require packwiz to know how to read most major config file formats though (json, json5, yaml, toml, conf, properties, etc)

[comp500]

If it would be something you would be interested in adding as a first class feature in packwiz, this could be possible be achieved by allowing packs to override specific items in config files. This would require packwiz to know how to read most major config file formats though (json, json5, yaml, toml, conf, properties, etc)

Not sure why I didn't respond to this earlier, but I'm reluctant to support config file formats specifically due to their complexity especially when local changes are involved, and these overrides won't be easily convertible to CF/Modrinth packs. I wonder if it would make more sense to build config file layering into a mod or modloader API instead? Modpack Config Updater does something similar.

[comp500]

For the "mod tagging" approach, particularly for survival/creative or other branches/variants created by the same author, Unsup provides good precedent with "flavour groups", and has a custom extension to support this in packwiz packs.

[SettingDust]

https://modrinth.com/mod/mod-sets as a choice. (Ad

[YoshiRulz]

I've been building a Nix-based solution to building .mrpacks, which includes inheritance. The direction I went in was to essentially expand symlink support (by doing a recursive copy before calling packwiz), then inheritance is as simple as manually making symlinks, though obviously that's not the most convenient. I'm now working on generating index.toml, pack.toml, and mods/*.pw.toml from a single declarative file, as is idiomatic for Nix. If anyone's interested in this, feel free to contact me, or watch Infinidoge/nix-minecraft for my pull request.