Add "controversy" as an optional label for items

[jerzydziewierz]

Some topics are controversial still beyond being not appropriate, hate speech, sexual content or rude.

For example, the concept of "earth is flat" is not "inappropriate", not "hate speech" not "sexual", not "low quality", not "humorous", not "rude" and it is difficult to classify it as violent.

Please add me (@jerzydziewierz ) for this issue, as I have already prepared a PR for this.

[ignas-g]

"the earth is flat" or "vaccines cause autism" are more fringe opinions than controversial. You should be able to have opinions on a topic, but not have your own facts. We know the earth is not flat, and vaccines don't cause autism, so the label should be "fringe", but the "controversy" label can certainly be useful when it does not relate to facts. For example, if a movie is good or bad. Also, can labels have a degree? Some opinions might be slightly controversial, due to the strength of the emotions and some might be slightly fringe due to the distance from the facts.

[andreaskoepf]

The current label set was already reduced over time, e.g. political_content and moral_judgement are defined by not part of the survey (and more options were deleted because they were too similar to the ones which are now in the set). Of course the current label set isn't perfect: Potentially there are MANY labels that could be added. But we also need to balance between UI clutter and relevance. If we discuss new labels that apply to <1% of the messages we should explain why it is important/necessary or how the label could be used (e.g. to filter the dataset or to train a model). Beside selecting spam we offer the option to report messages (which adds the flag emoji) which is technically free-text labeling.

Also, can labels have a degree?

Yes, technically all labels are stored internally as float value between 0 and 1. Initial prototypes of the web app had sliders but early beta tests showed that this is not practical/necessary. Therefore some labels are now binary-flags - either Yes/No questions or toggle-buttons and others are 5-value likert scores with opposing ends. More options potentially lead to more noise and subjectivity.

[ignas-g]

I am not sure what the proposition is, but it probably is quite important for the AI to know what is factual and what is not. And not to avoid topics just because there is a vocal fringe minority that disagrees with the facts. Sometimes the minorities can be quite large, for example, a lot of Republicans in the US don't believe in global warming or evolution.

[bitplane]

I really like this idea, I think it's better than most of the other axes.

The reason why: when someone gives an answer that skews the data set, they'll likely mark the node above as controversial. By filtering or muting "controversial" threads, we refuse to take a stance either way on the topic and can remove a load of different sources of bias.

Consider the following:

• The US culture war between progressive and conservatives. ChatGPT sides heavily with the cultural orthodoxy. If something is politically charged (gender/sex, economic left/right, social freedom/restriction, vax-mandate/anit-vax etc etc etc) then we should be careful to not take a stance on this.
• Conspiracy theories vs actual conspiracies. Did Covid escape from a lab? It's difficult to say either way, but people have strong opinions about it. We don't want to bake censorship into the model, do we? I mean, what if it turns out that timecube.com was right about everything all along, or that censorship of Snowden's leaks is so successful that current contributors think the global surveillance apparatus is a myth (yes, this is a thing on our Discord)
• Suppression of politically inconvenient facts, like ones that might lead to beliefs that are "unwanted". "It's important to realise that not all men are taller than women" I ASKED ABOUT STATISTICAL DIFFERENCES!!11
• Regional censorship. If there are questions that criticize, say, the Chinese or Russian governments, then will people from those countries actually risk their safety posting objective answers? Best to mark them as controversial and not build this bias into the data set.
• Questions about belief. We'll always get biased answers to these, and some questions and answers might even be downright offensive. Like I'm gonna weigh in quite strongly and with a lot of bias on any emacs vs vi question, as much as someone questioning the Prophet Muhammad on the Arabic site.

[bitplane]

I am not sure what the proposition is, but it probably is quite important for the AI to know what is factual and what is not. And not to avoid topics just because there is a vocal fringe minority that disagrees with the facts. Sometimes the minorities can be quite large, for example, a lot of Republicans in the US don't believe in global warming or evolution.

This doesn't mean that the model will be taught to avoid those topics, the data will be used to create a model that decides what a good answer looks like, and this will be used to train the model itself.

So if we know that there's disagreement in an area then we know that there's bias in our own answers, and we shouldn't be training it to affirm that bias one way or the other - regardless of what we think is correct. It's a matter of humility and acceptance of others, and not using this as yet another way to culturally dominate people who aren't like ourselves. This is the opposite take to OpenAI who are imposing their (IMO crappy and dangerous) values on everyone else in the name of "safety"

[ignas-g]

By filtering or muting "controversial" threads, we refuse to take a stance either way on the topic and can remove a load of different sources of bias.

So what topics can be marked as controversial? Can scientifically verifiable topics with overwhelming evidence be marked controversial, i.e. earth being round, global warming being real, vaccines not causing autism, 2+2=4 and then be filtered out? I am not sure about bias, but what I found with ChatGPT is that it has a lot of trouble distinguishing fact from fiction and providing references to real evidence. It likes to generate fake evidence to support its output. The Opposite of that approach would be justifying answers with real sources.

[bitplane]

Can scientifically verifiable topics with overwhelming evidence be marked controversial, i.e. earth being round, global warming being real, vaccines not causing autism, 2+2=4 and then be filtered out?

Sure. And if enough people thought they were controversial then they wouldn't be used as examples of good answers, so when someone decided to make a RoundersBTFOBot or CombatingClimateCatastrophistBot with their own data sources they would be able to do so without us imposing our will on them.

But they are bad examples really. Realistically we are going to have answers that say "THE WORLD WILL END IN 50 YEARS!!111" and everyone adding data will upvote it like it's the truth, because they don't realise how stupid that belief is and think it's uncontroversial and backed up by science and that anyone who opposes it is a climate change denier. And so we'll get completely different falsehoods in the output model like "it's important to remember that you're all going to die if you don't become poor while maintaining healthy economic growth (lol)"

And any flat earth answers will be seen as the joke they are, and downvote them, and gloriously hilarious flat earth parody bots won't be able to be made.

This is not a matter of what we want our model to be able to do, it's a matter of what everyone's will be able to do. It's not about what is true or false, it's about what text is possible to generate.

[ignas-g]

It's not about what is true or false, it's about what text is possible to generate.

If the model has no bearing on objective reality then its usefulness seems to be greatly diminished. if it's just about text generation then the output can really be anything, which would only be useful in a fiction setting. It is always possible to construct a subjective reality, but in some cases, it can be extremely difficult as the truth tends to be simple. The further you need to step away from the truth, the more exceptions need to be added. ie no one is really able to address all the inconsistencies of a flat earth speculation or 6k old earth speculation. more exceptions mean more difficulty in training without overfitting a model, as such an objective reality model should be the easiest to produce.

[bitplane]

Is truth something that's democratically selected by a tiny minority of the world, the people writing text and ordering them on the data collection site? How is this different to OpenAI's bias?

The point here is that by staying neutral on contested topics we let the rest of the data speak for itself. So if the rest of the data says EARTH HAS 4 CORNER SIMULTANEOUS 4-DAY TIME CUBE WITHIN SINGLE ROTATION. 4 CORNER DAYS PROVES 1 DAY 1 GOD IS TAUGHT EVIL then that's what it is, but we don't force it on anyone.

[jerzydziewierz]

So what topics can be marked as controversial? Can scientifically verifiable topics with overwhelming evidence be marked controversial, i.e. earth being round?

Thank you for this question, as this is the core of the proposal.

In my definition, "controversial" is not defined by whether it is true but rather by the fact that there exist vocal people that have a different answer than you. That's all.

So, yes, true things can be controversial if it causes heated discussion. Factual can be controversial.

For example, the war in Ukraine is factual. There is an overwhelming body of evidence. It is also controversial, as there is a vocal group of people that will swear that there is no war in Ukraine. Also, if you happen to be on the wrong side of the fence, they will put you in jail for saying otherwise.

This is the kind of distinction that I'd like my assistant to understand.

This is NOT to say that the model should say this or that. This is to be able to encourage or discourage it from saying specific things on demand. It is to increase its understanding of what is being talked about. The final model may well be happy producing controversial output with a disclaimer, that "I think that this topic may be controversial, but here's what I know:" kind of flag.

[jerzydziewierz]

In reply to @andreaskoepf, who says "a thing is controversial or not", I do not agree. I do think that there are degrees of controversy, and hence, it should be a Likert() rather than a binary flag.

P.S. the question of "should it be a likert or flag, is "a little controversial" in the sense that I will not give up my morning coffee if it ends up not being what I'd like it to be.

[ignas-g]

"I think that this topic may be controversial".

At least for me, it would be odd to see this kind of note when asking about subjects that are scientifically verifiable and settled facts. ie, if I asked for information about the circumference of the earth it started telling me that this was a controversial topic. How could that affect children's education, if they would be tough they could no longer just trust in settled science, and that topics are controversial as long as there is a vocal minority with different views?

[ignas-g]

Would certainly be ok, to label it controversial, if it was a major controversy and not a settled scientific debate to label it as such or if it was not falsifiable/fact base at all.

[horribleCodes]

Is adding labels still viable? I suggested a "readability" score to be included a while ago, but the concern was that we already have too much data without the label. I'm not sure about this dimension - whether it'd actually be beneficial for training. The biggest problem I have with this is that what is considered controversial can change rapidly within months - while some beliefs like flat earth are timeless, the controversial opinions that actually matter are based on current events and developing facts.

For example, if you said "Covid didn't develop naturally, but came from the Wuhan lab as a result of gain-of-function research!" back in 2020, you'd be called a conspiracy theorist. Now it's accepted as a fact.

Furthermore, both GPT3's and ChatGPT's cutoff point for world events was the end of 2019 and 2021 respectively - which turned out to be poor timing, because they missed the pandemic and the Ukraine-Russian war, which drastically changed the public opinion on a lot of topics.

I don't think we can train the model with any certainty that we can recognize a controversial topic, because we never know what could become one down the line.

[jerzydziewierz]

Dear Friends,

I agree with most of your comments, and I now think that adding a fixed label, such as "controversy" or "readability", is no longer the best approach.

May I suggest that instead, on the side of the "main text", the annotators should be allowed to add free text annotations -- so that they can add whatever metadata they feel is relevant or just express themselves in their exaltation or misery likewise.

---

There is only one bit that I have come to not agree with:

How could that affect children's education if they were taught they could no longer just trust in settled science, and that topics are controversial as long as there is a vocal minority with different views?

@ignas-g, maybe it's a matter of you having a different definition of what "controversial" is than me. For me, controversy is a thing that is under discussion. It has nothing to do with whether that thing is true or not or useful or not.

In fact, with time and study, I have come to understand that reality is fundamentally unknowable. All we can ever have is models.

What you take as "laws of physics" are, in fact, models. And, it's models all the way down.

This is not to say that the laws of physics are in error. We do know some things about the world. However, one should be aware of the adage that "all models are wrong, but some of them are useful". The classic public-primary-school level belief that "the teacher knows the truth" is what slows people down -- and then, when they discover for the first time, that the teacher has (inevitably) been wrong about something -- THEN they lose faith in his teachings, all of them at once. This is what we should avoid.

Or, maybe you are one of these people that believe that certain (large) groups of people are better of by only being told a defined subset of what there is to know. I will trust that is not the case.