compdemocracy / polis

:milky_way: Open Source AI for large scale open ended feedback
https://pol.is
GNU Affero General Public License v3.0
782 stars 186 forks source link

Allows comments to be edited #297

Closed patcon closed 3 years ago

patcon commented 4 years ago

Re-ticketed from https://github.com/pol-is/polisServer/pull/247#issuecomment-640094321

patcon commented 4 years ago

@crkrenn's comment from other issue:

Forgive me if I have mentioned this before, but I would like the capability for an administrator to transparently edit a comment for clarity or truthfulness.

The way I’ve done this in the past was to create a new edited comment and to disapprove the original comment. Typically this has been 1) to correct grammar or clarity without changing meaning. Or 2) to add context to a claim.

This capability could be extended to split one comment with two ideas into two comments.

If the conversation has a small number of participants and the admins are attentive, there won’t be many votes on the original comment. With lots of participants and a small number of admins, the analyst may want to make some simple assumptions on the correlation between the original and edited comments.

@patcon, is this already captured as a feature request?

C.

patcon commented 4 years ago

I suspect there might be some strong feelings on why this should or shouldn't be included, so I could imagine a starting point might be to start collecting scripts for niche usage in a contrib/ folder. I know that PDIS has some things like that that it's created helper scripts for.

(That's not to say I'm personally opposed to it being first-class in the UI, and I'd def be curious to hear team polis' thoughts on it)

patcon commented 3 years ago

@JohnFallot suggested in Gitter chat:

On the Configure page, when submitting seed comments: it'd be great if, upon submitting them, they appeared to the right of the creation field as a list of cards. It'd also be great if they could be edited from those cards directly.

colinmegill commented 3 years ago

1.

Polis will never, under any circumstances or at any point in the future, allow comments to be edited by any party after they have been voted on.

Polis has demonstrated over and over again that minute changes in language have dramatically different outcomes in producing consensus between groups and stakeholders. We will always err on that side.

2.

Polis will never under any circumstance or at any point in the future allow comments submitted by participants to be edited by moderators.

Participant comments edited by moderators fundamentally breaks the model of the tool at its most core level: the tool means to allow people to speak in their own voices, and assumes they have more knowledge about how to speak for themselves and to each other than centralized authorities.

This explicitly includes and extends to slang, typos, dialects, and other signifiers of own voice.

It also explicitly means to rein in overconfident moderators ready to get busy bonsai-ing and introducing tons of bias to their conversation.

colinmegill commented 3 years ago

To be completely clear: it is our position, methodologically, that there is no circumstance in which we can be confident that changing text does not change meaning.

Thus we always require resubmission, which will identify new author.

patcon commented 3 years ago

Totally agree on all this.

It also explicitly means to rein in overconfident moderators ready to get busy bonsai-ing and introducing tons of bias to their conversation.

omg yes.

Recognizing that it's important for moderators to respect and find balance with language chosen by users, I wonder if pol.is could offer an affordance in the UI to "accommodate" (though not budging on the above logic) and/or explaining your positioning more clearly.

As in, what about an "edit" button simply as an affordance in the UI, that allows moderators to quickly perform the sanctioned version of the "edit" they're trying to do, while explaining the rationale. Offering a path toward the expected behaviour and curtailing it inline would perhaps be better UX than expecting people to find it in out-of-band docs, or even a link. So clicking "edit" icon would maybe present a pop-up like this:

Editing statements with prior votes is not allowed, since <reason>. Read more about why

This statement by @username has been seen by X participants, received Y votes, and represents a [majority opinion / differentiator of at least one opinion group].

Submitting the form below will instead:

  1. moderate the original statement (including authorship) out of the conversation,
  2. discontinue further voting on that prior statement, and
  3. create a new statement (with moderator authorship), with no prior votes, for all users to re-engage with.

If this is what you'd like to do, please submit a replacement statement.

Replacement Statement:

_________________textarea________________________

cancel Submit & Replace

Thoughts? (I feel this will forever be a request we get, and so maybe we can do "user experience aikido" and redirect that misplaced desire into toward proper knowledge and the more appropriate alternative, esp without a human setting them straight :) )

colinmegill commented 3 years ago

Great!

ThenWho commented 3 years ago

I agree wholeheartedly with the 'no change if voted' policy. This is the only way to ensure people's voices are heard, as intended.

I feel the need to point out though the two challenges in moderation documented by Demos' recent report (p.21, paraphrasing here):

  1. Fears that "misspelled or ‘shouted’ comments" might have been less likely to be voted on.
  2. Statements that try to hijack the conversation (they give two examples of "SNP need ousted from ruining Scotland” and “END IMMIGRATION”).

The 'no change if voted' policy trumps Point 1 in my opinion, but Point 2 is grey area. It is the voice of the people, to be sure, but people that willfully ignore the topic of the conversation and try to subvert it. Of course it's a delicate issue too: what consists subverting, who decides when a stamement is relevant, etc. No easy answers here. But even if it occurs rarely in practice (in clear cut cases), it is a serious risk that will be exploited eventually. Needs some thinking.

Seen another way: moderating is doing some heavy lifting here. It might not be so easy to tie the hands of moderators or phase them out entirely (yet).

Sadly, I have no solutions or suggestions, just pointing things out. (PS: Maybe provide some tools for moderators, e.g. semantic connections between statements -such as 'related to' and 'refinement of' relations- and/or tone/topic detection?)


Re: the pop-up suggested by @patcon , I find it very good and I agree it's a pretty good steps towards better UX. Two points on the text though:

  1. I'm not sure the 'seen by X participants, received Y votes' part is needed. Moderators should not change statements even with one vote, and showing specific numbers might lead to all kinds of 'influence attacks', if moderators should not have access to these numbers while the conversation is in progress or afterwards.
  2. Point 1 of the quoted text seems to imply that moderators can delete voted participants statements just by editing them - replacing them with their own (unvoted) versions in the process (!). Which is clearly Not Good in my book. Maybe clarify that moderators can only edit their own submitted statements - by replacing them etc.
patcon commented 3 years ago

Thanks @ThenWho :)

Point 1 of the quoted text seems to imply that moderators can delete voted participants statements just by editing them - replacing them with their own (unvoted) versions in the process (!). Which is clearly Not Good in my book

My experience was that "moderating out" a user statement very early (while watching the convo closely after launch) and replacing it with variant statements, was pretty common. I'm open to hearing pushback that this was misguided. We mostly used it to tease apart multiple sentiments into standalones, not to change arbitrary words. If there was a face attached (through social login), we'd lean more toward leaving it untouched.

I feel there's tons of grey area in moderating, and very few hard and fast rules. My impression is that moderators will just navigate it, and we should just pressure them toward airing on the side of caution.

ThenWho commented 3 years ago

It seems there are several use cases covered here @patcon . One is small scale conversations, maybe 100-200 participants, where the moderator(s) act more like facilitators. In these cases, some more control is desirable and makes sense. I had similar experiences to what you describe, and in some cases it even makes sense to split the convo into two parts - one for gathering statements (some some editing/clarifing/spliting etc), and one for the actual voting.

Another use case would be large scale voting, perhaps on some official setting (e.g. local/regional community). I think this is what @colinmegill has in mind, and kind of matches the Demos setting I commented on above. In these cases, editing by moderators is undesirable, and their role should be limited wherever possible. For once, when we are talking about hundreds of thousands of participants, there needs to be more than one moderator, to catch up with the work. Then coordination between moderators becomes a problem, as well as policies they might adopt, ad-hoc solutions, bias, and more. (The Demos report describes some of these challenges.) These are the cases when 'no change if voted' really matters - I'm particularly weary about minority voices, incl. opinions, worldviews, even dialects, getting surpressed by 'overconfident moderators', as @colinmegill put it.

Not all is lost though 😃 . I think UI and a better UX is the answer here, as you pointed out. As long as the participant knows what they're getting themselves into (i.e. "the rules of engagement"), it should be fine. The same goes for moderators: maybe decide from the beginning what kind of convo they'd like to have? But what would be really bad is for a participant to go into the convo expecting e.g. a plebiscite or referendum, and having their voted statement changed by a moderator.

patcon commented 3 years ago

Should this be re-ticketed as the feature that eventually ended up surfacing?

colinmegill commented 3 years ago

Thanks @patcon, yes, was thinking the same, how would you phrase it? I might say, to be specific:

"Conversation owners (extend to all moderators perhaps) should be able to edit seed comments while vote count for a given seed comment is 0"