Contribution Requirements Update

[ryangurn]

Can there please be clarification made in the contribution doc to explain the following items:

1. Are there specific requirements that a repository contains tags/releases? Just one, or some arbitrary number of tags/releases?
2. Do we want to require software being added has been “actively maintained”? What does it mean for a project to be “actively” maintained? Some commits here and there? Responding to issues? Closed PRs?
3. Do we require code to exist in version control or just be accessible publicly?
4. Do we want 4 months to add but 6-12 months before removal? Can we make that more explicit in the docs?
5. What common sense items do we need to make more explicit? How do we make things explicit enough without preventing flexibility?

I have just been helping out a bit more with contributing to this project and gotten confused a couple times. There are good points brought up in issue and PR discussions that is not listed in the contribution doc, sometimes leading to confusion…

I am more than willing to update the docs, just wanted to start the conversation about what goes into the contribution.md file.

[ryangurn]

574 is the software that begs the question posed in 3.

505 & #593 are good examples for the question posed in 1.

[nodiscc]

Are there specific requirements that a repository contains tags/releases? Just one, or some arbitrary number of tags/releases?

At least one, at least 4 months old, as mentioned in the PR template Any software project you are adding was first released more than 4 months ago.

Edit: a "canned" response for projects that do not conform to this guideline was discussed in #33 , I just used it in https://github.com/awesome-selfhosted/awesome-selfhosted-data/pull/593

What does it mean for a project to be “actively” maintained?

• Commits or any kind of updates to the code in the last 6-12 months (exceptions can be made on a case-by case basis when no maintenance is required, see https://github.com/awesome-selfhosted/awesome-selfhosted-data/blob/master/.hecat/awesome-lint.yml#L10), as mentioned in CONTRIBUTING.md Software with no development activity for 6-12 months may be removed from the list
• No security issues left untriaged/unaddressed for an extended time period (there's no particular threshold but 1-2 weeks would start to worry me)
• No mention of the project being unmaintained in the docs

Do we require code to exist in version control or just be accessible publicly?

Just be accessible publicly (although you can ask the submitter why it is not under version control, it's not mandatory, there are no guidelines about this, it's just strange/out of the ordinary)

Do we want 4 months to add but 6-12 months before removal? Can we make that more explicit in the docs?

4 months before qualifying for inclusion in the list, 6-12 months without updates for removal, as mentioned in the guidelines above.

How do you suggest we make this more explicit? (without cluttering the PR template which is already quite a long read)

[Rabenherz112]

I'd like to give my two cents on this as well where I feel like nodiscc overlooked a few points.

What does it mean for a project to be “actively” maintained?

I think this aspect is somewhat lacking in documentation and heavily focused on the reviewer's standpoint (not good). As nodiscc mentions, there's a rule stating that software with no development activity for 6-12 months can be removed, essentially indicating a lack of "activity". As well as some common sense parts (which aren't documented (again not good)) such as no open security issues, maintained issues tab, and so on. However, the challenge lies in defining "active" since different types of projects naturally have different development cycles. For instance, a single-developer project built on PHP might only have a few commits over time (which could be acceptable), while a company using Node.js with numerous packages might require much more maintenance and, consequently, more commits. Thus, establishing a static metric is rather difficult.

Do we want 4 months to add but 6-12 months before removal? Can we make that more explicit in the docs?

I believe is this already well documented. The 4-month period is mentioned in every PR and needs to be checked by the person proposing an addition. Similarly, the 6-12 month removal timeframe is outlined in the Contributing Guidelines.

[nodiscc]

maintained issues tab

Not a requirement, free support is not mandatory. However it's another useful indicator when a project also does not receive any maintenance for >6 months.

For instance, a single-developer project built on PHP might only have a few commits over time

Which should fine by our criteria as long as the last activity is in the last 6 > 12 months

some common sense parts (which aren't documented (again not good)) such as no open security issues

I agree, not good, should be explicitly mentioned in https://github.com/awesome-selfhosted/awesome-selfhosted-data/blob/master/CONTRIBUTING.md#other-guidelines

[ryangurn]

thanks for the discussion, that is the entire point of this particular "issue".

some common sense parts (which aren't documented (again not good)) such as no open security issues

To be fair to all of us, let's just try and get some of the common sense items written down (I am also adding this as an additional edit to the main "issue" at the top of this thread).

In my eyes this boils down to the following questions.

5. What common sense items do we need to make more explicit? How do we make things explicit enough without preventing flexibility?

@Rabenherz112 you mentioned security issues and a maintained issues tab. What do you think we should do to address the differences in support provided between larger open source contributors (generally in the form of companies) vs single devs vs smaller teams? Also are there other common sense items you wish for us to address?

@nodiscc I think one common sense item I personally want to address is the open security issues. This is a challenging one however I think we need to get something more specific written down then "Unmaintained software without an active community and/or persistent security issues may be removed from the list" in CONTRIBUTING.md. I think putting something in place to incentivize better security scanning in open source software is crucially important (but that doesn't necessarily mean we should flat out require security scanning, and I would bet you agree). Starting to look at this holistically, it will be quite hard to automate security review initially (many repos do not have security scanning setup within GH, many larger orgs use external systems.) Using CVSS we might be able to establish a scale based on the CVSS score assigned to known vulnerabilities for open source software in the list. If we are to automate something like this, it would be interesting to see how you think we should deal with all of the external security applications that https://cve.mitre.org points to for the various CVSS scores.

I think that is a big enough block of security jargon (sorry about that). TLDR on the last paragraph: I think we should use more realistic words to describe that automated security scanning in any form is HIGHLY recommended, and that depending on the severity it needs to be fixed in a timely manner (whatever we define timely manner to mean).

Let's keep the discussion going even if it takes time. Updating procedures is rarely a quick process especially with open source software.

Fascinating Security Things (at least to me):

• NIST Vulnerability Metrics
• NIST CVSS Score Calculator
• OWASP Risk Rating Methodology
• SANS An Overview of Threat and Risk Assessment

[nodiscc]

If we are to automate something like this

I don't think we will, this is too much work and out of scope. Triaging CVEs is extremely time consuming, the CVSS system is broken (I mean look at this and this, should we all uninstall vim and drop JPEG support?), some vulnerabilities are only applicable in some configurations, the possibility of DoS/crashing the application counts as a security vulnerability in the CVE book, etc. What I personally consider important are vulnerabilities that affect confidentiality/integrity, allow Remote Code Execution, etc. But this is subjective. In the end, analyzing the security posture of each project is left as an exercise to the user/admin.

The "Unmaintained software without an active community and/or persistent security issues may be removed from the list" guideline is one of the "common sense" items which is not worth detailing in the contribution documentation IMHO, it boils down to:

• Software without any development activity for the last 6 months may be removed, at the discretion of contributors/maintainers, and with a basic explanation (it doesn't work anymore, major bugs were never fixed, etc)
• Software without any development activity for >12 months should definitely be removed
• Unless said software can be considered feature-complete and not needing much maintenance, there is still a sizable community of users, activity in the bug tracker, support requests, etc... In which case an exception can be made. Check the history of this file for examples.
• Regardless of the previous, if the project or its run-time dependencies have major security vulnerabilities, and they are not at least acknowledged by maintainers in a short time frame (in the order of weeks), they should be removed right away.

[ryangurn]

I don't think we will, this is too much work and out of scope. Triaging CVEs is extremely time consuming, the CVSS system is broken (I mean look at this and this, should we all uninstall vim and drop JPEG support?), some vulnerabilities are only applicable in some configurations, the possibility of DoS/crashing the application counts as a security vulnerability in the CVE book, etc.

Agreed

What I personally consider important are vulnerabilities that affect confidentiality/integrity, allow Remote Code Execution, etc. But this is subjective. In the end, analyzing the security posture of each project is left as an exercise to the user/admin.

Yep, I personally agree on CIA, and RCE vulnerabilities.

The "Unmaintained software without an active community and/or persistent security issues may be removed from the list" guideline is one of the "common sense" items which is not worth detailing in the contribution documentation IMHO, it boils down to: ....

Is there a good reason we shouldn't just put the exact verbiage (with the bullet points) into CONTRIBUTING.md?

[nodiscc]

It was mentioned in https://github.com/awesome-selfhosted/awesome-selfhosted-data/pull/837 that we should list somewhere what kind of software does not qualify. regular contributors have some notion of what does/does not qualify (off the top of my head, no desktop software even it produces files that can be synchronized by file sync solutions, no software which depends on a specific cloud provider, ...), but it should still be stated explicitely somewhere.

[nodiscc]

https://github.com/awesome-selfhosted/awesome-selfhosted-data/pull/608#issuecomment-1986591687

Basically, software that fits the one of the following criteria does not qualify:

• requires you to write application code before producing a working end-user application (libraries, SDKs, ...)
• acts as a platform to build and deploy arbitrary applications (PaaS, "serverless"...)

[nodiscc]

https://github.com/awesome-selfhosted/awesome-selfhosted-data/issues/891, https://github.com/awesome-selfhosted/awesome-selfhosted-data/pull/829

anything that is a generic container/deployment automation/virtualization/... tool is better suited for awesome-sysadmin