search

minetest / contentdb

A content database for Minetest mods, games, and more
https://content.minetest.net
GNU Affero General Public License v3.0
93 stars 45 forks source link

Add policy on Featured Packages #323

Closed rubenwardy closed 2 years ago

rubenwardy commented 3 years ago

View file

Fixes #321. See that discussion for more generic discussion

This is Wuzzy's list with some modifications - I've merged some things, and added other criteria

Wuzzy2 commented 3 years ago

Please add CC BY 3.0 and CC BY-SA 3.0 since they are not FSF/OSI approved (yet). Those licenses are perfectly fine IMHO. It seems those are still much more popular than version 4.0.

I still think 90% player approval is way too high. Do you know how hard it is to get a 90 or higher on Metacritic? Our first priority to have at least a few featured games to begin with. If we are too strict from the start, we might kick out too many games. Once we have a large number of games, and there is in general more "competion" between games, the criteria for being featured can be more strict.

I think it should also be possible to un-feature games when they no longer meet quality standards.

MUST: Be well maintained (author is present and active).

Still problematic. As long the package works and does not break, there is no absolutely reason for the author to be "present and active".

MUST: Comply with Rule 3 (Technical Names).

The link seems broken. What's Rule 3?

Wuzzy2 commented 3 years ago

@rubenwardy: Please invite other people to this this discussion. This is important. Thanks.

rubenwardy commented 3 years ago

Please add CC BY 3.0 and CC BY-SA 3.0 since they are not FSF/OSI approved (yet). Those licenses are perfectly fine IMHO. It seems those are still much more popular than version 4.0.

I still think 90% player approval is way too high. Do you know how hard it is to get a 90 or higher on Metacritic? Our first priority to have at least a few featured games to begin with. If we are too strict from the start, we might kick out too many games. Once we have a large number of games, and there is in general more "competion" between games, the criteria for being featured can be more strict.

I think it should also be possible to un-feature games when they no longer meet quality standards.

Updated to fix all of these

Still problematic. As long the package works and does not break, there is no absolutely reason for the author to be "present and active".

I think it's important for the author to still be around to fix issues, we want to avoid promoting packages that are likely to fall into abandonment. By active, I don't necessarily mean pushing features but active to respond to questions

Warr1024 commented 3 years ago

I would add having all the CDB metadata fields appropriately filled out. For example, we require "uses source control", so we should require that they actually use the proper source control field in CDB, as opposed to just having a link in the longdesc, or only knowing about the source control from the forum topic. Also to clarify, we're talking about public source control, not just "I use git at home and upload my zips from there".

For the 3:2 embedded screenshot, I think there's a minimum resolution that we can reasonably expect, like 320x240 or something, based on how it's used in the engine.

For the large screencap, I would drop the minimum resolution down to 1280x720. Requiring 1080p would effectively bar any package I maintain or would create in the foreseeable future from being eligible. I would be okay with re-raising this requirement if MT supported any mechanism for screenshots at higher resolution that the physical viewport, or externally scripting screenshots, such that these could be captured in a way that is not byzantine and error-prone. Alternatively, drop this down from MUST to SHOULD.

Game readmes: I would remove the mention of pointers to LICENSE (at least specifically) here because I think the LICENSE file is well-known enough that people who know at all to check the license will probably not need to be told to look there. A simpler thing would be to only consider files that match the common naming convention (all-caps in the root of the repo) and any files the explicitly point to as counting toward documentation requirements.

Crafting guide, instructions, and the "5 minute test": These feel redundant and like they should be combined into one thing, like that there should be a certain minimum amount of content that's playable without game-specific knowledge that's not already included in the game.

Everything that is marked "CAN", I would just remove from the final draft (we can keep them here for now for discussion purposes). This is a pass/fail system, there are no "bonus points" to be had, and a package these suggestions already stand as desirable on their own merits without having to be codified into a standard anyway.

ghost commented 3 years ago

MUST: Include a README file that includes at least:

  • Game name
  • Short game description
  • List of relevant links (download, bugtracker, etc.)
  • Pointers to other relevant files (like license)
  • Which Minetest versions are supported

This seems a bit over-specific as far as hard requirements go, particularly since much of this information is already available on CDB. The other thing that gives me that impression is.....

Of the 5 featured packages I see on the front page today:

If only one of five existing featured packages passes a MUST requirement, I can only assume that the requirement is either not as important as claimed, or too specific to be reasonable.

Warr1024 commented 3 years ago

Maybe we should make mention of who are most core target audience is for this. In particular this is valuable for:

This focus in particular would shift us toward recommending packages that would give new players the best experience starting from nothing, rather than just recommending what's popular among existing players. In particular, installing MTG and then going shopping for mods and trying to figure out how to put them together into something that isn't a complete mess is an experience we are trying to correct rather than encourage.

Note that while we are focused on featuring SP experiences, games at least would do well to offer both SP and MP experiences consistent with one another, and this could be included at maybe the SHOULD level (I would probably exempt shorter games like Box World 3D)

rubenwardy commented 3 years ago

I would add having all the CDB metadata fields appropriately filled out. For example, we require "uses source control", so we should require that they actually use the proper source control field in CDB, as opposed to just having a link in the longdesc, or only knowing about the source control from the forum topic. Also to clarify, we're talking about public source control, not just "I use git at home and upload my zips from there".

This is generally required on CDB - the review process is supposed to check this, and Editors should fix/ask authors to fix any breakage to this

If only one of five existing featured packages passes a MUST requirement, I can only assume that the requirement is either not as important as claimed, or too specific to be reasonable.

Packages can be changed to match the requirements. However, I agree that being that specific about the README isn't neccessary

Wuzzy2 commented 3 years ago

Those README criteria are super easy to add for the game dev, I don't really see a reason to not have them. Having a good README should really be standard practice. You don't know how often I have old software packages on my computer, finding them randomly again and tell myself "WTF is this thing?". The README can be very useful then.

ghost commented 3 years ago

Most of them are already included in metadata, and checking a package id on CDB is quite easy--why should anyone add them if these rules only apply to packages on CDB? Moreover, my understanding is that featuring is primarily for recommending packages to novices, the user group least likely to check the README files of various packages.

  1. The 'supported versions' clause is particularly suspect, since there's no way to update a README on someone's local machine and there's also no way to know which future version(s) a package will function on.
  2. Pointing to the license file is redundant and not terribly useful, since a LICENSE file must already be included in the package.
  3. 'List of relevant links' is not terribly specific, and the two examples given (download and bugtracker) are already handled by CDB, which these packages are being downloaded from in the first place. Again, I see little value for those, only yet another potential spot for redundant information to become out of date.

Name and description are fine, that's a pretty universal expectation of README files so I don't mind making it a requirement. I'm willing to change my mind with regards to requiring links, but I'd need to hear a more compelling argument than "why not".

rubenwardy commented 3 years ago

Those README criteria are super easy to add for the game dev, I don't really see a reason to not have them. Having a good README should really be standard practice. You don't know how often I have old software packages on my computer, finding them randomly again and tell myself "WTF is this thing?". The README can be very useful then.

I've moved the requirement to a sub bullet point of having a readme

Warr1024 commented 3 years ago

Having all that info in the README is a DRY principle violation. As it stands, I already have way too much in my own README, causing it to become out of date, which would not have been possible if DRY were followed. To fix it, I would either have to gut the readme and make it just a list of other files to read, or cram everything into the readme and then have some process to parse the readme to extract that data at deploy-time.

CDB itself could automatically obviate this need by simply constructing a cdb.txt file automatically from a snapshot of the metadata, long-desc, etc. either at release upload time, or at download time the same way the release ID is patched into the conf file. If there's support for the idea I can file a separate issue.

It's also worth noting that READMEs are often featured on git hosting services like github/gitlab as part of the root/landing page for a project, so they tend to have a different audience, i.e. they're often more developer-focused instead of player-focused.

Warr1024 commented 3 years ago

As an editor I would also like some clarity on the standard of evidence for each of these points, e.g. which ones are we expected to verify ourselves, which ones can we accept a positive assertion from someone else (e.g. seeing them present in a review), and which ones can we assume are met based on the lack of a negative assertion (e.g. there is no review complaining about them)? I will not be able to promote very many packages if I might have to audit 20kLOC of mod code to verify that there are no craft recipe collisions or something.

By default, I would assume that all the "fields are filled out" kinds of things I can check within a few seconds I should do myself, but for anything that would require reviewing code or even play-testing beyond checking for a crash on startup, I would just have to take the community's word for it that if there were a problem then one of the reviewers would have brought it up before the package received enough reviews to qualify.

Wuzzy2 commented 3 years ago

will not be able to promote very many packages if I might have to audit 20kLOC of mod code to verify that there are no craft recipe collisions

QA-Block can verify that automatically.

rubenwardy commented 3 years ago

As an editor I would also like some clarity on the standard of evidence for each of these points, e.g. which ones are we expected to verify ourselves, which ones can we accept a positive assertion from someone else (e.g. seeing them present in a review), and which ones can we assume are met based on the lack of a negative assertion (e.g. there is no review complaining about them)? I will not be able to promote very many packages if I might have to audit 20kLOC of mod code to verify that there are no craft recipe collisions or something.

By default, I would assume that all the "fields are filled out" kinds of things I can check within a few seconds I should do myself, but for anything that would require reviewing code or even play-testing beyond checking for a crash on startup, I would just have to take the community's word for it that if there were a problem then one of the reviewers would have brought it up before the package received enough reviews to qualify.

There should be a form with a checkbox/textarea field for each of the points, where the package author can add a sentence or so for proof. At least one reviewer should actually play the game for a while, to check it out

The process is fairly flexible, it probably worth trying it with one or two packages first