coreinfrastructure / best-practices-badge

🏆Open Source Security Foundation (OpenSSF) Best Practices Badge (formerly Core Infrastructure Initiative (CII) Best Practices Badge)
https://www.bestpractices.dev
MIT License
1.21k stars 203 forks source link

Support publicly-visible comments/feedback/peer review by others on a project badge entry #1233

Open david-a-wheeler opened 6 years ago

david-a-wheeler commented 6 years ago

Some projects may incorrectly claim to meet requirements they don't, or worse, falsify an entire badge entry. This is one of the problems with self-reporting. We already require badge entry information to be public and automatically check some values, which counters the problem in some cases, but we're starting to see cases where people seem to be falsifying information and/or entire badge entries.

The obvious next step is to support publicly-visible comments/feedback by others on a project badge entry. This would allow others to comment on a project entry. I intend to follow up this comment with a more detailed proposal, but hopefully this explains the motivation and basic idea.

david-a-wheeler commented 6 years ago

Here is a more detailed proposal:

Create a new SQL table called "feedback" with the following fields:

Proposed access control: Only logged-in users can add feedback about a project. Only the owner of the feedback or an admin can hide or delete a comment (note that project owners cannot control comments about the project entry). Only an admin can mark a comment as reviewed or not.

User interface:

Debatable options:

This is a first cut. Suggestions welcome.

dankohn commented 6 years ago

I'm opposed to this concept. Many top sites have stopped allowing comments because they are often so bad: https://www.quora.com/Why-doesnt-Vox-allow-comments-provide-a-comments-section-on-its-articles?share=1

I would instead just suggest that if people think badge info is erroneous or out of date, they should send an email to the cii-badges discussion group, where we can evaluate the suggestion.

-- Dan Kohn dan@linuxfoundation.org Executive Director, Cloud Native Computing Foundation https://www.cncf.io +1-415-233-1000 https://www.dankohn.com

On Wed, Sep 5, 2018 at 9:55 PM, David A. Wheeler notifications@github.com wrote:

Here is a more detailed proposal:

Create a new SQL table called "feedback" with the following fields:

  • id: Primary key for each feedback value (as normal)
  • created_at, updated_at: Time the comment was created/updated (as normal)
  • project_id: NOT NULL, Id of project it comments on.
  • user_id: NOT NULL, Id of user making the comment.
  • comment: String, to be interpreted as markdown text that is a comment by user_id about project_id. As with all markdown processing (as implemented in ProjectsHelper), the renderer will have filter_html: true and rel: 'nofollow', so any hypertext links will not improve any site's Google ranking (eliminating any reason for unscrupulous SEO consultants to add comments).
  • hide: Boolean, default F, NOT NULL. If T, this comment is not shown on the public site
  • reviewed: Boolean, default F, NOT NULL. If T, an admin has reviewed the comment.
  • reviewed_at: DateTime, default null. Set to the last time an admin reviewed the comment.

Proposed access control: Only logged-in users can add feedback about a project. Only the owner of the feedback or an admin can hide or delete a comment (note that project owners cannot control comments about the project entry). Only an admin can mark a comment as reviewed or not.

User interface:

  • A logged-in user, on seeing a project that they CANNOT edit, will see a new "Feedback" option (instead of "Edit"). Should it be called "Comment" instead? On selecting that, they get a form that lets them fill in a comment; the form provides hints on how to insert useful hypertext links to specific project issues. The form has a "submit" button that, once pressed, adds the comment. (Should there be a preview mechanism?)
  • Anyone who views a project can see, at the bottom, a list of all unhidden comments about the project entry. There will be a "delete" and "hide" button on each feedback entry that is owned by the current user; every feedback entry will have a "delete", "hide", and "reviewed" button if the viewer is an admin. Selecting those buttons, where access control permits it, changes the feedback record.
  • Admins will have a separate admin-only page that can show all comments or all unreviewed comments in the datetime order, along with links to the project commented on.

Debatable options:

  • If a user or project is deleted, should we delete the related feedback? The simple answer is "yes", so I think we should start with that; then we don't have to deal with missing links.
  • Should users be allowed to edit a comment, once made? The easy answer is "no", so I suggest we start with that. We could change it later to edit for only a period of time, like Hacker News, or just allow arbitrary edits like GitHub.
  • A non-admin user should not be able to add feedback to a project that user can edit. That's debatable since maybe a group would want a way to discuss the project entry, but members of a project probably already have their own way of doing that and we aren't trying to supplant that.
  • "Hiding" and "deleting" aren't much different; maybe we should drop hiding, at least at first, and add it back later if it turns out to be needed.

This is a first cut. Suggestions welcome.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/coreinfrastructure/best-practices-badge/issues/1233#issuecomment-418938221, or mute the thread https://github.com/notifications/unsubscribe-auth/AC8MBr2AWs8h_5RF2CxtrMgs7q1PRljnks5uYICugaJpZM4Wb9En .

david-a-wheeler commented 6 years ago

@kestewart - thoughts?

kestewart commented 6 years ago

Typos happen. Suggest allow editing, but mark it as "edited". Interface on issues is fairly clear about this sort of thing - and seems a reasonable model to start from.

kestewart commented 6 years ago

@dankohn, challenge is scaling this out.

If we get people able to do the equivalent of +1 on others badge info, it will build confidence. Similarly, if they want to -1 and provide a reason - that can the be reviewed by the cii-badges discussion group, and the posters will get the feedback directly as well.

Not all of the people submitting for badges are auto-subscribed to cii-badges. I didn't learn about it until much later after we got we initial Zephyr badge.

david-a-wheeler commented 6 years ago

At the bottom of every website page is text that says:

Need help? Have a question? See a problem? Please send an email or file an issue.

"File an issue" links to GitHub; anyone who's familiar with GitHub (and most OSS developers are) will be able to navigate that. The "send an email" is a mailto: link to a special email address, cii-badges-questions-owner, which sends emails to the badge admins. So even if you don't want to use GitHub, we already have an easy way to send an email to the CII admins. Of course, emails can get lost, so wanting something better is reasonable.

We clearly don't want useless comments. An alternative might be to let people create comments on project entries, but only allow admins to see them. That would eliminate most of the incentive for creating bad comments, though at the cost of making it so that no one else can see them. Is that better? Worse? I'm not sure. Comments eagerly sought!

david-a-wheeler commented 6 years ago

A related note: Our "send an email" mailto link does not work, presumably because of the recent changes in Linux Foundation mailing list infrastructure. We need to get that sorted, but that should be done as part of issue #1207.