Opt In / Opt Out | Required changes to CKAN.

[VasVadum]

CKAN has officially become a threat to the community because mod author are beginning to change their mods over to All Rights Reserved in order to stop CKAN from listing. This will hurt modding in the long run, and cause potentially permanent damage if this ARR trend kicks on just to get back at CKAN for not allowing choice.

You need to change your de-listing policy immediately because I will not continue to support something that is going to ignore this immediate threat. It has been stated that when CKAN becomes opt in, most of this issue will go away and the people who are angry and wanted to de-list, are likely to no longer want to de-list.

So here is MY personal suggestion which is slightly getting some negative opinions against but I still think it is good for de-listed mods to help us manage everything in one place; If an author de-lists, you can grey their mod out in the program and still show the home page for users to click and go get the mod manually, also showing a version number on it and all that jazz. Instead of checkboxes or dashs where you would click to auto install/update a mod, place an X for a delisted mod. When a mod has an update, turn the "update" column X into a clickable button which brings you to the home page of the mod so that you can find the download URL. Make sure to make a note however for de-listed mods when a user clicks a home page button in one for the first time or an update button on one for the first time that it pops up an un-closable message that lasts for a minimum of 5 seconds; The author has requested that his or her mod not be downloadable via CKAN, so you must do this via his or her page. Please do not pester the mod author to change their listing status, it was their choice to make and they made it for whatever their reason was and they are even less likely to change their mind if people spam them with requests to list on CKAN.

This way I don't need to manage mods via browser bookmark folder, and can still find a mod via CKAN even if I can't install it via CKAN. I think an even better option for this though, is three versions of opt. Opt-In: Your mod is listed and downloadable via CKAN. Opt-List: Your mod is listed in CKAN but not downloadable. It can still notify of updates too. Opt-Out: Your mod is not listed in CKAN unless it is detected in your game install, and then it is marked red in CKAN with only the information found in the meta-data that the author has provided with his mod. (Or don't list at all, I suggest listing if installed because this would allow me to export a list of mods I use easily if I want to share that list with someone, particularly for diagnosing an issue.)

I also suggest that you include a method for authors to write their own meta-data. No more allowing non-authors to submit data to CKAN too. Authors must be the ones to submit their information to CKAN, and a file that comes with mods can correct any meta-data CKAN has on the mod. That's for another topic though I suppose as this one is specifically about opting in and out and the results of you not enabling it.

[BobPalmer]

The representations made by the first part of that post are inappropriate, and false.

(edit) Ahh good. I see he edited it.

[VasVadum]

You mean the part I changed after having noticed I made a mistake?

[Sigma88]

I like the greyed out idea as an additional feature provided that a modder that do not want to be listed in any way shape or form has the option of being completely removed from any CKAN related list

[dewiniaid]

I also suggest that you include a method for authors to write their own meta-data. No more allowing non-authors to submit data to CKAN too. Authors must be the ones to submit their information to CKAN, and a file that comes with mods can correct any meta-data CKAN has on the mod.

The only problem with this approach is the number of cases where authors don't care if their mods are on CKAN, but don't feel like managing it themselves.

As far as opt-in/out/list goes, I think it might be better implemented as:

Supported: The author(s) of this mod officially support installing through CKAN-based installs.

Available: This mod is available for CKAN-based installs. There is no statement of support (in either direction) by the mod author. This is the default for all mods that don't explicitly state a policy.

Unsupported: This mod is available for CKAN-based installs, but the authors will not provide support for them.

Prohibited: This mod is not allowed to be installed via CKAN.

External: This mod is only available for CKAN-based installs when using a separate repository. (A pointer to the new repository should be in the metadata).

---

With the above in mind, here's how I think this can be fixed:

1. Add a new "policy" field to the metadata for updates, reflecting the above:
2. Add a highly visible "maintainer" field to metadata that reflects the person who is responsible for the metadata. The special value "author" or "authors" indicates that it is strictly under control of the mod author(s); PRs from anyone else to change the metadata should be rejected automatically (exception being abandoned mods under a new maintainer). This gives both players and mod authors a point of contact if there's issues (technical or otherwise) with CKAN metadata.
3. To reduce burden on mod authors, see if KSP-AVC's spec can handle embedding partial metadata within its own JSON format -- namely the policy field. (My theory is it will ignore fields it doesn't recognize, but this should be formalized). This allows mod authors to clearly specify their own policy while not increasing the number of files they have to update on a release.
4. Refuse to install a mod version whose policy field states that it should not be installed by default. (Prohibited) Find a way to make it where someone who's savvy enough to do their own troubleshooting (including uninstalling from CKAN and doing an author-supported install prior to posting) can workaround this -- either by compiling CKAN themselves with the relevant options disabled, or by manually editing a configuration file. (CKAN being open source means there's no way we can truly prevent this anyways). This should still very loudly warn that the author does not support this method of installation and does not provide support for CKAN installs; ideally the warning requires more effort to dismiss than merely clicking "OK".

[DuoDex]

höhöhö.

It's about time that this policy got itself revised, IMHO. I, too, would support the additions that @dewinaid, @sigma88, and @VasVadum propose.

[TeddyDD]

CKAN has officially become a threat to the community because mod author are beginning to change their mods over to All Rights Reserved in order to stop CKAN from listing.

@VasVadum @pjf explaiend all this stuff very well in #1078

IMHO This whole discussion about opt-out is pointless (and not motivated from legal standpoint). Mods authors should learn how to use licenses properly.

This will hurt modding in the long run

No, it will not. Few moders will sentece their mods to disappear in future, oh well. ARR mods will disappear sooner or later, that's why I don't use it. There is plenty FOSS mods.

[anxcon]

any mod i make has been changed to ARR until ckan respects modders

legally speaking ckan crosses into the same territory has torrents, i have no issue with how ckan software itself works, i have no issue with ckan auto fetching mods / having an opt-out if a modder decides ckan is an issue to him. what i do have issue with is those who run ckan having policy to abuse "what they can legally do" without respect to mod authors. i disagree with ARR whole heartedly, thinking it's rude to the community, as ARR blocks other mod authors from learning / using / expanding upon what i (or other authors) learned / expanded upon from the community in the first place, and yet its the only option i see when disrespect is purposely given by a few in the community.

add at least as far as "de-list mod no questions asked", and - just given the past disrespect in policies - a policy to not re list if author had it removed. i like what ckan does, i accept bugs in software, i dont have to accept disrespect towards myself or others in the community.

[Sigma88]

I like how zealously you keep doing all your best to keep all the FOSS mods listed on here tho ;)

[dewiniaid]

legally speaking ckan crosses into the same territory [as] torrents [...] [I] disagree with ARR whole heartedly, [...] and yet its the only option i see when disrespect is purposely given by a few in the community.

Torrents involve distributing (possibly-)ARR content that you do not have a license to distribute. CKAN does not handle the actual distribution of mods, it only points at a source that is (presumably) authorized to do so. Even in the case of an ARR license, CKAN is still only deep-linking to your content -- a practice that -- while controversial, has generally been ruled legal in various jurisdictions[1][2][3]. So ARR does not solve this issue at all (though it was a valid option in cases of e.g. the 64-bit debacle)

i have no issue with how ckan software itself works, i have no issue with ckan auto fetching mods / having an opt-out if a modder decides ckan is an issue to him. what i do have issue with is those who run ckan having policy to abuse "what they can legally do" without respect to mod authors.

To clarify, you have no technical issue with CKAN; the problem is more one of policy and the human factor? (This is not an attempt to derail the discussion or diminish its importance, just an important distinction that I think needs to be made.)

Technology is rarely the solution to what is -- at its core -- a human problem. We can come up with a technical solution to aid in implementing and abiding by some set policy, but technology cannot replace policy as a whole. My earlier post, for instance, only holds weight if CKAN contributors actually abide by the intent of the policy definitions specified.

I'm personally of the mind that a complete and total opt-out of CKAN is harmful to everyone:

• It's harmful to authors, because it means people are either less likely to install their mods or less likely to update them before reporting problems. CKAN updates do have a lag time (though perhaps ways to improve that can be developed), but they do at least generally show up for the mods it can detect updates to.
• It's harmful to players, who don't have many other mechanisms for comprehensively browsing a list of available mods, nor quickly checking for updates to all of their mods at once. (KSP-AVC is often touted as a solution to the latter, but requires explicit author effort and support -- and mods have shipped with incorrect .version files before)
• It's harmful for CKAN as a whole, since less mods listed on it translates to less usage overall.

If authors have the ability to restrict CKAN solely to update checking (as opposed to delisting entirely), it mitigates most of these issues.

---

[1] https://apps.americanbar.org/litigation/committees/intellectual/news_0209.html [2] http://fairuse.stanford.edu/overview/website-permissions/linking/ [3] https://en.wikipedia.org/wiki/Copyright_aspects_of_hyperlinking_and_framing#History_of_copyright_litigation_in_field

[TeddyDD]

After reading forum I think I underestimated scale of problem. It's sad that good tool becoming a threat because of users that can't read and spam modders with CKAN issues :>

[anxcon]

@dewiniaid first off, torrents are do nothing more than ckan, a torrent is the equiv of ckan meta, instructions on where to find the file, information on the file, etc. what ckan does is go a step further, accually handling distro of the file itself. it does not host it, but it does aquire it and deliver it, which falls under distribution. if you goto the corner drug dealer, distributes drugs, taking them from a higher dealer and delivering them to you. yes ckan itself uses code to deliver to you. that is why earlier (before bit torrent protocol) programs got hauled into court, the user was hosting the file, but the program (and sole reader of "meta data" was downloading and delivering the files. torrenting nowadays has the niche they dont accually download / distribute the file, as they don't touch the file.

2nd yes, my issue is the human factor and disrespect that standing policies have towards the community. disrespect drives authors away, which hurts the community far more then having to manually click the download link on forum post, and unzipping

that said, ckan has made no effort to lessen that burden aside from full auto install, which granted is a noble goal, but software will always have bugs, and lessening the burden of manual install (buttons to open appropriate folders without navigating, etc is the simple minimum step) can be nearly as good

[Cphoenix]

Refuse to install a mod version whose policy field states that it should not be installed by default. (Prohibited) Find a way to make it where someone who's savvy enough to do their own troubleshooting (including uninstalling from CKAN and doing an author-supported install prior to posting) can workaround this -- either by compiling CKAN themselves with the relevant options disabled, or by manually editing a configuration file. (CKAN being open source means there's no way we can truly prevent this anyways). This should still very loudly warn that the author does not support this method of installation and does not provide support for CKAN installs; ideally the warning requires more effort to dismiss than merely clicking "OK".

This is the only suggestion I take issue with. Mod authors shouldn't have the authority to dictate what tool a user can use to install their mods. Don't support it, fine. Don't be willing to troubleshoot installs made with it, fine. Don't want it automatically listed? Absolutely fine. But this is over-reach in the extreme. If CKAN supports it and a user wants to install a mod directly so they can use CKAN to manage it, why shouldn't they be able to?

Seriously how would anyone expect people to react if in other communities like the ones around Bethesda's games had mod authors suddenly saying "Oh no, you're not allowed to use Mod Organizer to install this one! You have to use NMM!" (Or any other particular mod tool.) I don't see that particular change in anyone's best interest. At best it's going to lead to annoyed users who will just pester the mod authors more asking why they would do such a thing.

I fully support every single other suggestion in that post. I myself brought up similar suggestions on the KSP forums not long ago, but I'm pretty sure they got lost in the animosity mod authors (or at least a vocal few of them) have expressed at CKAN as a whole. Ferram et al. made it abundantly clear on the forums that suggestions like this to mitigate the issues they're experiencing aren't acceptable to them.

But again, I'll reiterate my support of this, sans prohibiting installs. It would provide more information to users and hopefully direct them to the right support areas instead of pestering mod authors with issues that are actually with CKAN itself. (Basically I would hope to see users directed to CKAN's support pages first for Available, Unsupported, or External installs. Sending them to a link for common troubleshooting steps here on the GitHub wiki would probably help redirect users to the right spot.)

After reading forum I think I underestimated scale of problem. It's sad that good tool becoming a threat because of users that can't read and spam modders with CKAN issues :>

That's why I personally think having a way to use CKAN to direct users to the right place for support would be very beneficial to all concerned.

[dewiniaid]

@dewiniaid first off, torrents are do nothing more than ckan, a torrent is the equiv of ckan meta, instructions on where to find the file, information on the file, etc. what ckan does is go a step further, accually handling distro of the file itself.

A torrent has a source (or many fragments of a source) somewhere, and the people responsible for said sources (e.g. someone else running a client) are not authorized to distribute the content.

In the case of CKAN metadata, the source of the data is the author's own website/Spacedock/Github Releases/Curseforge/etc; linking to that source is perfectly legal. I'm not infringing on copyright by linking to, say, one of the early releases of KSP on Squad's own website, but if I link to the exact same file on freepiratedsoftware.example.com I'm potentially participating in "contributory infringement" (the same way various torrent sites have been shut down), and the owner of said site (where the actual data lives) is definitely infringing. Never mind that it's a free download, the ARR license means "Hey, only I have the rights to distribute it."

[dewiniaid]

@Cphoenix "Prohibited" is intended to serve as a way for an author to explicitly say "No, I don't support CKAN at all, please install my mods using my preferred distribution method(s)." There is no technical way to actually enforce it (aside from DRM of some kind, and nobody wants that), but the intent is that the author's preference is made very clear and cannot be disregarded by means of "user is conditioned to just click OK without reading the prompt". I don't like it either, but it seems it a necessary option.

Remember that one of the issues here is that various things -- from CKAN metadata breaking an install to "Your mod doesn't work" "Did you read the install instructions and edit the .cfg files?" have led to an increased support load for several mod authors. There has to be some way for them to go "Nope, can't/won't handle that" -- the lack of that option is the crux of the current issue. That said, I still feel the best possible result is to expand the options that CKAN metadata has in such a way where delisting/"Prohibited" becomes the option of last resort in these circumstances, rather than the only option.

[DuoDex]

@cphoenix It's not a matter of legal standing, it's a matter of common sense. If a modder asks you to remove your listing as a matter of courtesy, you should do it, otherwise they're likely to come back with an ARR licensed mod, and ARR mods are a pain in the ass for everyone involved.

[janbrohl]

From my perspective it looks as simple as adding a mandatory "maintainer"-field (obsoleting "x_maintained_by") analog to "author" and adding a field "manual_install" set to true for mods that must not be auto-installed.

[lamont-granquist]

"torrenting nowadays has the niche they dont accually download / distribute the file, as they don't touch the file."

torrent sites only host metadata. torrent clients actually do file distribution of the bits that they download. that is where it gets legally a bit grey.

ckan's github site only hosts metadata. the ckan clients themselves do absolutely no distribution of the bits that they download (other than delivery to the end user running the client -- pulling from the publicly available URL that the author of the mod has made publicly available).

legally i don't see how it could be more clear that this is a "deep linking" issue vs. a "bittorrent" issue. ckan clients do not deliver bits to other ckan clients. ckan is not bittorrent.

[lamont-granquist]

if mod authors want to control their own distribution, they could introduce a similar scheme to the way that oracle requires a time-sensitive cookie to download java, requiring users to click through the EULA.

[janbrohl]

Nothing could stop authors from moving their files (invalidating the CKAN-download-URLs) and relicensing them with the same conditions plus "CKAN-Exception" stating installation via CKAN client is forbidden. (Not making them ARR this way.)

[anxcon]

fun fact, majority of torrenting is 100% legal, most MMOs use it to lower the server load, lets get that bit out of the way before someone claims its all bad - the bad bit is the places that decide to share stuff they arent allowed to, and ofc the media will fuel any fire that gets em ratings, nobody cares "this guy did nothing bad today, neither did this guy"

@janbrohl many licenses can't be changed from what they are now. and ckan policy as stated in another issue, is if permission is given it stays given - edit: clearly noting if alternate downloads of the mod exist, switch to them

[passinglurker]

@Cphoenix improving the backend to cover more usage cases and minimize the number of errors is all well and good but these issues have been known for over a year and instead of fixing them this project seems to have instead prioritized keeping up with the metadata.

As a result what the authors want is a way out NOW with the option to come back when the service improves LATER. simply put there is no confidence that the needed code changes will be made in a timely manner.

[ferram4]

I think we've gotten off-topic here. No mod author has ever stated that indexing mods is against the law (regardless of license), and discussions about that are pointless anyway. Although the fact that arguments seem to be heading towards an implicit "we should index ARR mods regardless of modder opt-in/opt-out too, it's legal just like for torrents!" has me concerned for the resulting workload.

My main concern is that CKAN's policies are set up so that create more support requests for me, and then provide little option to protect myself or to convince CKAN to change those policies. Because of these policies, I have no confidence that CKAN and its contributors can be persuaded to implement proactive error checking methods or whatever else may be needed to reduce modder workload in a timely manner, considering they have the option of simply doing nothing and the mod remaining listed.

For context, about a year ago CKAN created a gigantic support nightmare for a FAR release due to a metadata error (actually, several of them back-to-back), burying crucial bug reports that I needed to fix actual critical bugs in the mod. The "no courtesy de-list" policy was officially adopted a month later. An issue to address the cause of the FAR debacle only got created in April. No progress has been done on it since.

As of right now, there are a lot of modders that are very ticked off because they have no simple means to deal with the consequences of CKAN's errors. The extent of those we have are to license more restrictively (at least for now), to put in the effort to manage the metadata ourselves (though given the recent happenings with @BobPalmer even that isn't a guaranteed fix) or to simply quit modding. This is not a healthy situation that could easily be addressed through changes to CKAN policies.

[politas]

The problem that I see is that a "Opt-in only" policy means a lot of mods would never be listed on CKAN through author apathy.

I would like to find out how satisfactory it would be to change the policy to allow open licensed mod authors to opt-out from CKAN installations, so that CKAN is not pushing authors to switch to restricted licenses?

This seems to me to be a simple policy change that would allow us to quickly lower the level of antagonism in the debate. Any technical solution for listing mods for user tracking without CKAN installation is going to take time. Time during which the current situation is maintained, and mod authors are getting angrier and angrier.

So I propose a change in CKAN de-indexing policy to say that open licensed mod authors can opt-out of CKAN installation. Initially, this would effectively mean opting out of CKAN listing entirely, until CKAN has a system in place for listing mods with no installation.

@ferram4 , @BobPalmer @anxcon, @Sigma88 , could you please respond to this specific suggestion?

[BobPalmer]

I'd like to cover that suggestion, also toss out a total of three (IMO reasonable) things that can sort at least the concerns on my end.

Yes, the option to opt out regardless of licensing and be absent from any listing (even just the mod name, though I doubt many would do that, I do see some cases where a mod goes on hiatus, or has been deprecated (how long was Regolith in CKAN?), etc.).

A staging area (as suggested before) so that pull requests are vetted. So anyone messing with FAR, or adding one of my mods. or even the case where I fat-finger my own metadata. Make sure it at least passes some cursory checks. This ties into the next point (which again - helps everyone).

I agree with @politas on modder apathy. I also expect that 95% of the modders are fine with their stuff magically appearing. But we also want to make sure we handle the few edge cases.

A new mod comes up. Ask said modder 'We are planning on listing this... let us know if you are not ok with this'. Wait 48 hours (or whatever). If no response, put it up. Done. If they say 'no thanks', respect that and move on. If modder comes back later and says 'please take it down', do so. In all cases, there's a conversation that takes place. The wishes of content creators are respected, but CKAN is not stuck having to chase down every modder getting explicit permission,

An answer may also be 'please don't ever list any of my things - I will sort them myself'. Plunk that author on a list. This is an easy gate to check in pre-staging. Ferram may prefer to vet his changes himself. I may prefer to do my own metadata, or not list all of my mods. 95% of everyone else may be fine with the default system. I think the changes below sort it for most everyone (though I cannot speak for @ferram4, @anxcon , and @Sigma88

To recap.

Ask first. If no reply, go ahead and list it. If a request comes to de-list, do so, Add a staging step to catch some of these common issues.

I respect that the last may take time, so steps 1 and 2 are a good start. Tho I think step 3 is pretty important to help alleviate the frustration.

[politas]

@BobPalmer , can you expand on what you believe is the minimum acceptable level of vetting? What checks should be carried out?

[BobPalmer]

@politas - At a minimum - make sure the file is valid, and that the dependencies are there / respected. Check against authors known to publish their own metadata, or who want a chance to review the netkan before it changes. Pie in the sky would be to have maintainers who would actually check the CKAN download first and make sure KSP loads, etc. before giving the thumbs up.

That last one is of course an edge case - relevant to things with complex dependency paths or very compex installs (like RSS, etc.). Me, I'd be happy knowing that my Netkan gets a spot check, and that if a netkan slips in where someone accidentally submits one of my mods, it can get caught and someone can ask me before it pops into the index.

[BobPalmer]

Another good example will be when we have the bruhaha post release where everything gets messed up due to incompatible versions. A quick 'is this compatible with the new version? Mind if we set that to push other mods through?' goes a long way.

[politas]

Another issue is the lack of clear APIs for dependencies. CKAN by default assumes that dependencies are not version-specific, and that assumption has messed up CKAN installations of FAR releases. We probably need to disable automatic metadata updating in such cases and switch to vetting each new version. It's a pain in the arse, but that would make things better for CKAN users and affected modders both.

[BobPalmer]

Agreed, but good news is that will be an edge case. If a modder sees something blow up, they can always de-list and have the conversation on what steps need to happen to sort that - including a manual or joint vetting process.

i.e. if mod X has some weird issues, agree that it stays in staging until either the modder or a designated maintainer can vet it out.

[ferram4]

I'll back up @BobPalmer 's suggestions, they seem good, although I would personally prefer a stricter Opt-In only policy instead. There is a risk under the proposed method that an apathetic modder's first interaction with CKAN will be discovering that someone indexed one of their mods because the PM got ignored, or the post got buried, or that the author was away for a bit and that they're getting issues they didn't ask for. Not gonna get a lot of CKAN popularity there, tbh. At least they will have options though.

I think there are three other things needed besides metadata vetting:

1. Mandatory public postmortems for large-scale CKAN failures (read: don't sort themselves within a day or two, say, during KSP updates). That would allow finding out what parts of metadata vetting are weak, if someone simply dropped the ball, if new policies are needed to address things. As time goes on, it'll allow for a large number of case studies to point towards what can go wrong and how to avoid it and by making it a procedure it'll build confidence in CKAN again because failures will be publicly acknowledged and addressed.
2. An abridged history of this entire mess. If @TeddyDD 's reaction is any indication, there seems to be a disconnect between what CKAN devs/contributors and what mod authors think is going on. Things got and have been very bad but no one with any influence to change things seems to have realized it. If new CKAN contributors join and only ever talk to the CKAN contributors and not modders, things stay this way; seeing exactly how bad things can get should be part of the "things you need to read and know before contributing" to try and prevent the disconnect in the future.
3. An always-installed plugin that dumps the installed CKAN data into the output log when the game starts up. It will help with troubleshooting CKAN installs and also identify issues that might be caused by people trying to get around any of these policy changes through unofficial repos.

Sound good?

[pjf]

I'm appearing late to the conversation, so thanks in advance for everyone's patience while I'm still getting up to speed. There are lots of concurrent discussions going on; I'm going to try and contribute here and link in from others. I very much want us to have a consistent policy regarding how mods are indexed. There's been a lot of discussions, and while I can't address all of them, here are my arguments for many of them:

All Rights Reserved Mods (and other restricted licenses)

We do whatever the mod author wants, or we don't list the mod. ARR is a clear indication that the authors have absolute control over the code, and we should respect that.

Temporarily de-listing mods

We can and should continue to do this when a mod is causing issues that can't be solved by fixing the metadata. In the cases where something can be fixed via metadata, a temporary de-list may still make sense while that's sorted out. Installing mods in a way that is broken benefits nobody.

Permanently de-listing open source mods:

Some mods have dozens of contributors, and open source licenses make it clear that anyone has permission to use, modify, and redistribute mods distributed under an open source license¹. RealismOverhaul is a great example here, it's got sixty-three independent authors. Even FAR has a dozen listed, albeit most other than Ferram are reasonably minor.

I don't think I should be able to permanently de-list RealismOverhaul, for example, and what's more there should never be a reason for me to ask this. If a mod is working correctly, and has a free and open source license¹, then we're not respecting the wishes of that license or its contributors by permanently de-listing.

The Attic

I do think it's entirely appropriate to modify existing metadata to make sure extant mods don't get installed on systems they're not designed for. I'd also say it would make sense to have an "attic" repository where old mods go to gather dust. Users can opt-in to get mods from the attic, but there's no risk of typical users getting mods that are no longer maintained or otherwise outdated.

Re-licensing of FOSS mods to ARR

I view this as a matter between the mod authors themselves. While one can't retrospectively remove a license from already released code¹, but it's certainly possible to release new versions under a different license, although I won't debate the mechanics of that here. If mod authors mark new releases as ARR, we should respect that. I feel the correct objection here is to fork and maintain a copy of the last FOSS version of the mod.

Addressing a few comments in particular:

@ferram4 : I would personally prefer a stricter Opt-In only policy instead. There is a risk under the proposed method that an apathetic modder's first interaction with CKAN will be discovering that someone indexed one of their mods because the PM got ignored, or the post got buried, or that the author was away for a bit and that they're getting issues they didn't ask for.

• For restricted license mods, I'm in 100% agreement. I'd be totally fine with an opt-in only policy for restricted license mods.

• For FOSS mods, I still feel we should be trying to CC in authors/maintainers as much as possible. It's polite, and we'll likely end up with better experiences for all.

• Mandatory public postmortems for large-scale CKAN failures

• An abridged history of this entire mess

We're a team that consists entirely of volunteers, but to the best of our knowledge our processes and communications are public. I'm very supportive of being able to analyse failings and friction, and find ways in which we can improve. However, that also requires finding someone with the time and motivation to do these things.

• An always-installed plugin that dumps the installed CKAN data into the output log when the game starts up.

This sounds like an excellent idea, and the CKAN core has been built specifically so it can be loaded by KSP should we ever need that, Again, we need someone to code it, but I can see very few downsides of having this at all.

@politas : CKAN by default assumes that dependencies are not version-specific, and that assumption has messed up CKAN installations of FAR releases. We probably need to disable automatic metadata updating in such cases and switch to vetting each new version. It's a pain in the arse, but that would make things better for CKAN users and affected modders both.

I agree here. The metadata does allow for version ranges of dependencies to be specified, but often this hasn't been specified. A simple "vet-new-releases" flag for NetKAN would be a fairly straightforward change that means human attention can be better directed where it's needed. In the case of staging (see below), "vetting required" releases need a human to authorise them, and won't be assumed good by the bots.

Staging

This gets its own heading, because I think it's so important.

@RoverDude: A staging area (as suggested before) so that pull requests are vetted. So anyone messing with FAR, or adding one of my mods. or even the case where I fat-finger my own metadata.

This sounds like a great idea to me. We already support multiple upstream repositories, and it makes sense to have a staging repo which users would need to opt-in to receiving. I'd even weakly argue that all new mods appear here first, and are then bot-moved to the main area after a couple of days. Users who "need the latest mod now" can always opt-in to the unstable area, and are valuable for helping catch bugs early, but your typical user will only see mods from stable.

Under the hood, we can have an anointed branch in our existing CKAN-meta repo, which would mean we could more easily track how metadata moves should that ever be needed in the future.

One could also argue for there being "supported" and "unsupported" repos. There'd need to be some discussions as to how mods end up in each, but they're extremely possible with the existing clients and infrastructure.

I am still catching up with conversations, so thank you again for all your patience.

TL;DR: I really like @RoverDude's suggestion of a staging area, and think there's potential to solve a lot of problems by having separate stable, staging, and unstable repositories, with users needing to opt-in to anything but stable.

¹ We use the Open Software Initiative's definition of open source. Any mod that doesn't have a perpetual, transferable license to use, modify, and redistribute is already listed as "restricted' in our licensing metadata.

[BobPalmer]

In the interest of community goodwill, allowing a modder, regardless of their license choice, to have the option of de-indexing their mod is core here. Let's not lose sight of that. Because without that, we'll just move to ARR licenses (those that are not on them already).

If you are willing to respect a modder's implicit wish in the case of ARR, why can CKAN not respect a modder's explicit wish to not participate, for whatever reason, in CKAN indexing?

Currently, there is a tremendous backlash building up against CKAN because of this - something that can be solved with one sentence in the de-listing policy by extending the courtesy afforded to those of us who now use ARR licenses to all users. It's also the right thing to do.

Without this, you will continue to see more modders switch to restricted licensing and de-index. And while it is simple to say 'we would just fork the last good version', experience has proven that that's rarely feasible, except in the simplest of part mods. And we're not talking about part mods here.

So. Let us start by looking at what it is going to take to get that particular policy changed. Specifically, That if someone, for whatever reason, does not wish their mod indexed, it can be removed regardless of their license.

@pjf , @politas - thoughts?

[blowfishpro]

@BobPalmer opt-out is straightforward in the case of mods that have only one author, but what about when that isn't the case? Does only one have to want it de-listed? Do all?

[BobPalmer]

It should be the main maintainer. In almost all cases, this is pretty obvious (whether it's the original creator, or the current curator). Looking at the account tied to the Github repo or KSP thread will point you right there. ie. there's no doubt who speaks for my mods, even though I have had dozens of contributors accross them. Nor is there any doubt who speaks for FAR.

If there's a doubt, a conversation can be had (and I expect those will be very few and far between).

[anxcon]

ckan has already made the claim for legality that it can list anything, even ARR, but wont list ARR for courtesy reasons. ie listing cannot be stopped. now the other direction, users cannot demand a mod be listed on ckan, atleast legally.

so now that the legal bit of "do we list it" is out of the way leaving the policy "do we list it" up to policy, lets look at that. single author is accepted as straight forward, so we are left with multi author mods and policy (not legality) of what to do, since policy can state anything that is wanted (p.s. policy against my little pony mod plz) you have 4 1) all agree to list else dont (i dont see ckan happy) 2) any 1 (or majority) agree list else dont 3) ignore all, list anyways (seems ckan fav idea) 4) and rover just posted above me primary maintainer and having policy state they need repo control etc, not just submitted a PR to contribute, limits those who accually control it

[BobPalmer]

I would agree with 4, unless someone gets control transferred per the maintainer from 4. i.e. for Firespitter, that is Snjo even tho I do most of the updates these days as a courtesy. If I ever wanted Firespitter pulled for whatever reason, I would have to get Snjo to do so, or to officially transfer the mod over to me. Repo ownership and thread ownership are two very easy ways to sort this.

[Blu3wolf]

This sounds like a great idea to me. We already support multiple upstream repositories, and it makes sense to have a staging repo which users would need to opt-in to receiving. I'd even weakly argue that all new mods appear here first, and are then bot-moved to the main area after a couple of days. Users who "need the latest mod now" can always opt-in to the unstable area, and are valuable for helping catch bugs early, but your typical user will only see mods from stable. Under the hood, we can have an anointed branch in our existing CKAN-meta repo, which would mean we could more easily track how metadata moves should that ever be needed in the future. One could also argue for there being "supported" and "unsupported" repos. There'd need to be some discussions as to how mods end up in each, but they're extremely possible with the existing clients and infrastructure.

Id like to argue for supported vice unsupported repos. One simple solution would be to move all metadata in the present repository to a new repo, 'Unsupported'. This unsupported repo would function much as the present one does, but would not be the default repository.

Mod Authors or a nominated Maintainer could add metadata to a new repository, Experimental. This repo would hold new versions of the metadata (and new metadata for newer versions of the mod) for a period of time, afterwhich it would be moved to another new repo, Stable.

The only repository enabled by CKAN by default, would be the Stable repository. Users could enable Experimental, or Unsupported, with a corresponding decrease in the level of support offered by the modder.

Potentially, CKAN Contributers could add mods to Experimental (nominating themselves as the Maintainer), with Experimental and Stable repos always permitting mod authors to Opt Out.

Ideally, mods displayed in CKANs listing would then be color coded to indicate which repository they come from, perhaps a subtle difference in the background color of the list elements.

This set of Official repositories would likely also assuage concerns over users creating their own sets of repositories, by obviating the need for it.

[BobPalmer]

Except getting someone to admit to something they know precludes them from support pretty much never works. We've been over this with Win64, etc.

[Blu3wolf]

Thats a fair and reasonable complaint. I seem to recall it was suggested somewhere that it would be possible to have CKAN log data in KSP, as a plugin? So that users submitting valid bug reports would be easily caught out by that?

An always-installed plugin that dumps the installed CKAN data into the output log when the game starts up. It will help with troubleshooting CKAN installs and also identify issues that might be caused by people trying to get around any of these policy changes through unofficial repos.

That.

Id hate to think that the only solution that worked for Win64 would be applied here, although it has been threatened.

[pjf]

If you are willing to respect a modder's implicit wish in the case of ARR, why can CKAN not respect a modder's explicit wish to not participate, for whatever reason, in CKAN indexing?

Because a Free and Open Source Software (FOSS) license is a hard promise that others can use, modify, and redistribute code, even in ways that the original author objects to. It's one of the reasons why FOSS projects are often so vibrant, because users and contributors know that the code will always be there for them to use and change.

Nobody's ever forced to release their mods under a FOSS license, it's a choice that mod-authors have to opt-in to, voluntarily and willingly. Contributions to those mods are done under the same license, in good faith that those contributions will remain open and available in the future.

A FOSS license in which the author says "You can't use this in a ways I don't like" isn't a free license, and the software should not be licensed as such.

It's not my goal to make the lives of authors difficult. I really like the idea of having staging and testing areas, of making it so that users have to opt-in to things which may not be working right, or may not be properly checked for correctness, or which need human vetting. I absolutely support mod re-licensing where that can be done, and I absolutely support us temporarily de-listing mods where we know they're causing problems. It's only the permanent de-listing of mods that are under a FOSS license that I struggle with.

My head's become fuzzy from staring at a screen for so long, so I'm going to break for some exercise and to see the sky.

[passinglurker]

@pjf this isn't one of your big professional FOSS projects this is a modding community a labor of love just because we give you a license with such freedom that you can kick us in the face doesn't mean you should. You need to play nice if you want to spread FOSS here.

[pjf]

I feel we're starting to get off-topic here. @Blu3wolf, @passinglurker, I think you've both made excellent points. In the interests of keeping the discussion easy to follow would you both be okay with letting this issue be used for discussing the other (but related) issues which have been raised?

[TeddyDD]

As @anxcon noticed here opt-in unsupported repo could not works so well as we think. Many users would activate it mindlessly and spam author thread anyway. It's too easy to add repo url in ckan gui. That's why you don't find AUR helpers like yaourt in offical Arch repos - to force noobs to actually read a wiki and build package manually.

I'd suggest making CKAN completely ignore repo marked as unsupported. So noob user would not be able to install unsupported mods using CKAN. When it comes to advanced users... I'm not sure. AUR like approach (grab *.netkan, build package for yourself, install it) could not work here because some mods don't have netkans. Maybe CKAN could allow for adding repositories from local directory so advance user could just git pull unsupported repo manually?

And I think that even FOSS mod authors should be able to opt-out (move his mod to unsupported repo effectively delinting mod for normal / noob users) Its against logic, its against FOSS license they chosed, its against users interest but moders overreaction with changing license to ARR is nothing good in long term. This way we keep modders and advance users happy :disappointed:

An always-installed plugin that dumps the installed CKAN data into the output log when the game starts up.

This is really god idea but this could be just first step. CKAN could help with preparing bug reports (for example automatically find all KSP logs and upload them to pastebin on user request?). CKAN should also keep log similar to pacman.log (it looks like this ) and also allow to revert last transaction (so player would be able to get back to working game without hassle)

Mandatory maintainer for supported ckans/netkans is also pretty good idea :+1:

I'm also thinking about some bots improvement that would allow modder to nuke mod release in seconds. For example when your mod is causing trouble with CKAN you just open issue nuke XYZv1.2.ckan and boot would delist mod, close this issue and open new one devoted to repairing mod. It would require to check if nuke issue was opened by mod author but that could be done (github username check?)

[sarbian]

Sorry @pjf but that pov remind me a bit too much of the sourceforge mess.

[politas]

While I understand and agree with @pjf on the fundamental concepts of FOSS licenses, I also agree that CKAN should be responsive to community standards in the specific realm of KSP game mods, which as mentioned, are not the same as large FOSS projects. Courtesy delisting for FOSS mods by primary maintainer request should be our policy.

EDIT: When I say de-listing, I mean no installation, which currently equates to de-listing

[politas]

For mods we mark for no installation, we could still have the "download contents" button available and an "open zip file" button to make manual installation easier. Then we could have a "mark as installed" button so that CKAN is aware of which version is installed of a listed, but not automatically installed mod.

[solarsootysmudge]

@TeddyDD is right that end users will still spam mod authors by using the opt-in unsupported by default. Ref issue #1502

Also can we look at @Dazpoet ideas again under #1679. As it might help here.

[TeddyDD]

I wonder if forcing installing mods from unsupported repo via cli would do the job

[Blu3wolf]

what it might do is convince folks to learn how to write a GUI.

[janbrohl]

A good way to deactivate installation could be to set the "install" field to a string (with instructions for manual installation). This would stop the ckan client from installing it (not sure if it would still be listed) and future versions could show these install instructions while the entries would still be useful for finding the mod like with my Web-View. If you do not want your mod listed in the Web-View just contact me on the Forum as i can blacklist mods idependently of CKAN.