Resolving resource duplication in modules

[MatanShahar]

Apparently I like opening issues. Well this issue is somewhat, if not complex, large.

Abstract

Up until now all the 1.11 modules where built separately so name duplication was fine. The problem appears when merging all the modules into one jar. Since a jar is an archive it can't have duplicate paths at all, this means that resource duplication will, and is failing builds.

Risk management

Why this issue needs to be resolved

As it stands the issue does not affect current builds and does not cause any unexpected behavior that we are aware of. The only thing this issue affects currently is the future of the project's release strategy, if we choose to release as with a combined 'jar' file with all the modules this issue must be addressed.

Cost of resolving this issue

This is definitely a large issue and solving it will take up development time and may delay the 1.11 release. Additionally it is very hard to validate, since it involved resource object and not code all references to them are strings so the compiler is of no use. These objects are accessed throughout the codebase in many classes and many new bugs may be introduced.

Resolution strategies

The following are the different possible ways of resolving the issue.

Migrating all resources to base

Theoretically it should be possible to move all the resources back into one module (base) and call it a day.

Effort: medium

• Moving (git mv) all the resources to base.
• Merging all the files with duplicate path.

Maintainability: low

• Doing this will basically nullify all the work done splitting the project.
• Any resource will be added to base, even if it is not used there.
• base will have to be release on every resource change.

Reliability: high

• Everything will stay mostly the same, no resource paths will be changed.

Customizing the build to merge resources

It may be possible to use the build engine (gradle) to merge the resources on build.

Effort: high

• Currently I'm not aware of anyone who know who to do this
• Probably each resource type we'll require a different merging method: lang files can be concatenated but xml files can't.

Maintainability: modium

• This will not affect the code itself and will be mostly transparent.
• Adding another codebase to manage and maintain.

Reliability: medium

• Most of the time everything should work but new resource types will break the build.

Additionally I think this strategy is very risky, over time it will be forgotten and no one will know how to fix the same issues we are having now.

Moving the resources to separate domains

This is the real solution to the problem - giving each module its own resource domain.

Effort: high

• It requires vising any point in the code where a resource is referenced.
• Making sure we caught everything will be hard - no compiler support

Maintainability: infinite

• This will solve the problem completely

Reliability: high

• This will solve the problem completely, forever
• It may introduce a few bugs

Risks:

• Leaving old code pointing to old resource paths
• Braking some sound mechanics (@tterrag1098)

Alternatives

Are there any alternatives to resolving this issue?

• Giving up single jar builds - the single jar is not a requirement but is very convenient to current mod users.

[JoduWei]

No matter how you decide, I can help if you tell me how. Copy files, fixing references etc.

[HenryLoenwind]

Let me give you a tiny bit of history. All projects are using the the DOMAIN, and that is not an accident but was done on purpose. I would have preferred for them to use their own modid as domain, but there are too many places that want to add a domain to something, or where Forge decides for you which domain is the right one. You see all those hundreds of warnings about a wrong domain at startup? That is there because I changed the modid of the main project to "enderiobase". I did that to provoke those errors, so I can see if something is wrong. Quite soon I will change the modid of enderiobase back to match the domain.

So, resource merging is the only way to go. Give me a moment to make a list of all conflicting files. There shouldn't be that many.

[HenryLoenwind]

• *.lang
• mcmod.info
• pack.mcmeta
• recipes.xsd
• sounds.json

Not too many, (Yes, I de-duped resources recently)

Strategies:

• Take One

pack.mcmeta and recipes.xsd can be taken from ' base'. All submods must(!) have identical(!) copies.

• Concatenate

*.lang files can simply be concatenated.

• Merge

sounds.json and mcmod.info need to be merged. Merging strategy is relatively simple (drop first/last line, add a comma).

[MatanShahar]

Now that each module is really a different project why would Forge decide on a wrong domain? It's pretty much the same as TE having different repositories. I really would like to avoid the merging option because I have no idea how to do this or even if that's possible.

[HenryLoenwind]

Because Forge keeps track of which Mod's code is currently running. While you can supply resources for any domain in jour jar, once you register objects with Forge you must supply the correct ID for whatever code Forge started. And from that ID is derived which resources will be loaded for it.

So if the ModObjectRegistry registers a block, that block must have an ID of "enderiobase" or we get that nice warning we see at the moment. Same for registering the sounds. If we want our central sound registry to handle that (and don't want to make a copy of it in each submod), they all need to be defined in asset/enderio/sounds.json. The .oggs themselves could be anywhere, as could the textures (Forge doesn't care what resources are used by resources, only about the initial one). But those don't conflict...