Properly defining a mod metadata format

[asiekierka]

As you might've noticed, in 0.1.x-0.3.x the fabric.mod.json format is effectively "defined" by the Java class - this doesn't really make for good documentation or specification.

This is an issue opened to discuss things which should be covered by a fabric.mod.json standard ("schemaVersion": 1) which I hope to add in 0.4.0 - with the prior JSON files being implied as "schemaVersion": 0.

These are the existing fields in the mod.json format as of 0.3.6:

    // Required
    private String id;
    private String version;

    // Optional (environment)
    private DependencyMap requires = new DependencyMap();
    private DependencyMap conflicts = new DependencyMap();
    private String languageAdapter = "net.fabricmc.loader.language.JavaLanguageAdapter";
    private Mixins mixins = Mixins.EMPTY;
    private Side side = Side.UNIVERSAL;
    private boolean lazilyLoaded = false;
    private String initializer;
    private String[] initializers;

    // Optional (metadata)
    private String name;
    private String description = "";
    private Links links = Links.EMPTY;
    private DependencyMap recommends = new DependencyMap();
    private Person[] authors = new Person[0];
    private Person[] contributors = new Person[0];
    private String license = "";

Some prior art:

• Forge/Sponge's mcmod.info (duh)
• craftson spec - this is what we based the initial mod.json format on back in 2016, but have since deviated
• Spigot's plugin.yml
• MCubed's craft.json - designed for project formats, but could be useful here

I'd also like to call in @peterix as he might have lots of prior art from his work on MultiMC.

[asiekierka]

PROPOSAL: https://fabricmc.net/wiki/format:modjson

TODO:

• [ ] dependency resolution information

[asiekierka]

There is a concern I have about dependencies: The dependency system needs to be built to be able to express virtual dependencies, and strange setups thereof. "There is an API, there are multiple implementations of it, but only one can co-exist", for example - in this case I would foresee fabric-loader having a configuration file allowing the end user to select their preferred implementation. Also "mod depends on an implementation of an API, any implementation, but of this specific API" in general.

Also, NOW is the time to propose breaking changes to the format.

[kitlith]

re: dependencies, maybe it'd be a good idea to look at package managers that handle stuff like virtual dependencies. After all, it seems that what you effectively want is to build is a local package manager that chooses the latest version of everything that is available on the local filesystem.

[asiekierka]

Yes, pretty much that, yes. Debian's alternatives, Arch's virtual packages, etc.

[Prospector]

Get rid of the single initializer in favor of the array.

[LemmaEOF]

What's Links look like? Is there a reason for it to be a class instead of just a String[]?

[asiekierka]

@Boundarybreaker It's going to be a Map<String, String> with some default getters for default values, if we stick to craftson proposals

[LemmaEOF]

Sounds good to me, then.

[Gegy]

Would multiple modids within a single mod.json file be something to consider?

[LemmaEOF]

@gegy1000 Craftson had provisions for that, but iirc they've been removed in current Fabric.

[asiekierka]

I have not, but I'm torn.

Generally, I feel that we should have one mod ID per JAR. This would greatly simplify the resolution system, as adding a JAR to the classpath adds all the mod IDs to the class path.

[asiekierka]

About nested JARs: Should the nested JARs just be assumed to sit in a directory (such as jars/), or should they be explicitly listed in the fabric.mod.json (or maybe the JAR manifest)?

[LemmaEOF]

I think explicit listing the manifest, so you don't need to change the fabric.mod.json for a slim build.

[Gegy]

I suppose the issue of more than one mod is not so much when jar-in-jar is introduced. Never mind that.

[asiekierka]

I'm not sure if there's a point in explicit listing at all, myself; primarily because people are unlikely to include JARs in a special directory they don't want to be loaded

[asiekierka]

I have posted a proposal. It can be considered complete, barring dependency resolution information.

[LemmaEOF]

It looks like links has been swapped out for a more general contact field, yeah?

[asiekierka]

That is correct. It also defines some non-URL types, and an official option for adding your own fields to the list.

[asiekierka]

Pushed notes on dependency resolution information.

I think this is it - if you want to request additional features or changes, NOW is the time.

[asiekierka]

NOTE: One feature still planned is missing - a "provides" string->version dictionary for providing virtual packages - though I'm not sure if, say, virtual packages should have a version of their own or just expect the ID to be changed if a breaking change occurs.

[SquidDev]

Just curious as to why you've gone for envType rather than environment?

[asiekierka]

Good question. Changed.

[asiekierka]

What about adding a "group" field (mandatory or optional?) allowing group/id/version triplets to function as Maven artifact identifiers?

On a separate note, do we want to change initializers in any way, such as allowing them to define sides and dependencies not unlike the existing dependency resolution fields?

(I'm torn between doing it in the JSON and doing it as annotations on the class, but the former feels more Fabric-y and we already have a parser.)

[asiekierka]

Another question (as I see this issue is getting the attention it deserves): after a discussion in #fabric, we are considering a move from "jars/ folder defines nested JARs" to "the mod JSON provides a list of nested JAR files". Thoughts?

[asiekierka]

Due to lack of responses I'll just make the necessary decisions myself. o/

[asiekierka]

.... Except I can't. Other than the nested JARs thing - which will go into the JSON as opposed to searching across the JAR filesystem - I really have no clue what the best approach is, on my own.

[asiekierka]

More questions nobody will read:

• Should languageAdapter remain a string which points to a class, or should it perhaps name a mod and then the mod would define the languageAdapter class path?

[LemmaEOF]

I think it should point to a mod.

[LemmaEOF]

also, re: initializers change, I think it might be good to have a mix of both. Use strings for a simple initializer, but if someone wants to set stuff like an environment or dependencies let them make an object with it.

[modmuss50]

Having a maven artifact identifier could be useful, just trying to think of a use case though.

I think there should be a json that lists all the nested jar info.

Having the languageAdapter point to a mod id is a good idea

[asiekierka]

I think there should be a json that lists all the nested jar info.

Since, by definition, a nested JAR must be a Fabric mod, I think an array of JAR files is sufficient for now.

[asiekierka]

Having a maven artifact identifier could be useful, just trying to think of a use case though

This way, you could easily convert between Fabric mod dependency chains and Maven artifacts. This could be useful for development tools, launchers, etc.

[modmuss50]

Ah yeah, I can see that being quite useful, would having a url saying what maven its on also be a good idea? or is that not needed?

If nested jars have to be a fabric mod then an array is just fine, as those jars would contain all the info inside them.

[asiekierka]

would having a url saying what maven its on also be a good idea?

Yes and no. It puts the reasoning for the feature into question; on one hand, adding URLs could promote certain bad practices in launchers and mod loaders; on another, it means such a launcher or mod loader needs to be aware of all the mavens mods could be on.

If nested jars have to be a fabric mod then an array is just fine, as those jars would contain all the info inside them.

Technically, storing the inner version data in the outer mod JSON provides a way to early-discard nested JARs, but it also means the data is now in two places and, as such, can be desynchronized; such an extension could be added later if needed.

[kitlith]

RE provides: I'd say let it have a version and see what happens. If needed, it could be marked as just metadata/ignored in the future.

Other dependency things: do we want 'replaces' or similar? I'm guessing "no, just remove the old mod".

maven URL: would the launchers be performing these bad practices if the URL is not present? Or are they going to try anyway, and giving the URL would just help them?

[Pannoniae]

@asiekierka what practices are you talking about? ;)

[asiekierka]

@Pannoniae autodownloading dependencies without any sort of verification allowing for MITM attacks (if HTTPS not present), replacing JARs with exploitable versions a bit too easily, among others... unless signing was done

[peterix]

Personally, I don't think including any sort of download URL has lasting value. It is a short term workaround for missing mod repositories, lack of support in launchers and user laziness.

Your magical download servers will go down and will turn any such workaround into a failed DRM scheme. In other words: it's never a good idea.

Ideally keep the metadata to what is strictly necessary for loading mods. You are making a mod loader, not a content hosting platform, a repository of mods, or anything like that.

Also sorry for not participating more here... I wanted to, and then other things stole my attention.

[asiekierka]

It's fine. We're in no hurry.

[3TUSK]

Two cents: for license part, can we assume that the absence of license field implies "All Rights Reserved"?

[holly-hacker]

That is the legal default, so yeah.

[asiekierka]

For any non-SPDX identifier, you probably may as well. It's Safe

[asiekierka]

So, the changes to make, relative to the spec, as I propose:

• Initializers are now namespaced; if they use a non-standard namespace (f.e. "scala:[...]"), they use a non-standard language adapter;
• Add a "languageAdapters" field; it's a dictionary of namespace keys to language adapter classes;
• Reserve the "group" field, but not do anything with it (yet?);
• Add a "jars" field - an array of either strings or objects with a "file" key. Designed this way for potential future expansion,
• Add a provides field - an id->version dictionary for virtual dependencies. Will not be in 0.4.0-beta.1, as I want it out the door ASAP.

This should cover everything, more or less.

[LemmaEOF]

Sounds good to me!

[asiekierka]

Done. More everything:

• "custom" (or "extra", or "extensions"): a key->JsonObject map, free to mess with by modders (namespaced),
• "icon", providing a square PNG file icon (rationale: Minecraft resource pack icons, CurseForge project icons, Mod Menu icons and icons in general are square). I'm evaluating two options:
  • always have it point to a PNG file,
  • allow an extended { "width": "file1", "width2": "file2", ... } form for optimized PNGs, but I fear people bloating up the JAR too much then.

[asiekierka]

And the final question before 0.4.0's release: Virtual dependencies!

The usecase is generally "allow depending on an implementation of an API without specifying a specific implementation", as well as "only load one of a given hook type".

"provides": {"fabric-rendering-plugin": "1.0.0"} OR "provides": ["fabric-rendering-plugin"] I'm not sure if provides should be versioned

[LemmaEOF]

include versions with virtual dependencies so they can match dependencies with other mods

[asiekierka]

The main issue is that it additionally complicates the already complicated dependency resolution.

Currently, it can at least assume that one mod candidate corresponds to one mod ID/version pair; with the addition of this, it can't, and it worries me that it might lead to some kind of fluke where the dependency resolution times themselves start being noticeable.

[LemmaEOF]

oh, that's a good point. hmm.

[Daomephsta]

Versioned API dependencies are necessary IMO. Without them a mod must assume any methods or classes added after initial release of an API may not be present, and breaking changes cannot be clearly communicated to users.

Could we perhaps have candidate types? API type candidates wouldn't be allowed to have initialisers and their version would be required to match the API version they provide; mod type candidates would be able to have initialisers, but wouldn't be allowed to provide virtual dependencies. That way 1 candidate still only has 1 version.

Alternatively, could we compute dependencies once at first startup of an instance, and only recompute them when the mod list or a mod's version changes?