markwhiting
commented
1 year ago Originally posted by @amirrr in https://github.com/Watts-Lab/Commonsense-Platform/issues/86#issuecomment-1757692680
Dimensions of statement and their definition:
behavior
- Social: it refers to beliefs, perceptions, preferences, and socially con- structed rules that govern human experience; it can be “real” or opinion, but is intrinsically of human origins. e.g., I exist and am the same person I was yesterday. He yelled at me because he was angry. There are seven days in the week.
- Physical: it refers to objective features of the world as described by, say, physics, biology, engineering, mathematics or other natural rules; it can be measured empirically, or derived logically. e.g., Men on average are taller than women. The Earth is the third planet from the Sun. Ants are smaller than Elephants.
everyday
- Everyday: people encounter, or could encounter, situations like this in the course of their ordinary, everyday experiences, e.g., Touching a hot stove will burn you. Commuting at rush hour takes longer. It is rude to jump the line.
- Abstract: this claim refers to regularities or conclusions that cannot be observed or arrived at solely through individual experience, e.g., Capitalism is a better economic system than Communism. Strict gun laws save lives. God exists
figure_of_speech
- Figure of speech: it contains an aphorism, metaphor, hyperbole, e.g., Birds of a feather flock together. A friend to all is a friend to none.
- Literal language: it is plain and ordinary language that means exactly what it says. e.g. The sky is blue. Elephants are larger than dogs. Abraham Lincoln was a great president.
judgment
- Normative: it refers to a judgment, belief, value, social norm or convention. e.g., If you are going to the office, you should wear business attire,not a bathing suit. Treat others how you want them to treat you. Freedom is a fundamental human right.
- Positive: it refers to something in the world such as an empirical regularity or scientific law, e.g., hot things will burn you; the sun rises in the east and sets in the west.
opinion
- Opinion: it is something that someone might think is true, or wants others to think is true, but can’t be demonstrated to be objectively correct or incorrect; it is inherently subjective. e.g., FDR was the greatest US president of the 20th Century.. The Brooklyn Bridge is prettier than the Golden Gate. Vaccine mandates are a tolerable imposition on individual freedom.
- Factual: it is something that can be demonstrated to be correct or incorrect, independently of anyone’s opinion, e.g., the earth is the third planet from the sun (this is correct and we know it is correct), Obama was the 24th president of the United States (this is incorrect, but we know it’s incorrect). It will be sunny next Tuesday (we don’t yet know if this is correct, but we will be able to check in the future).
reasoning
- Knowledge: the claim refers to some observation about the world; it may be true or false, opinion or fact, subjective or objective e.g., The sun rises in the east and sets in the west. Dogs are nicer than cats. Glasses break when they are dropped.
- Reasoning: the claim presents a conclusion that is arrived at by combining knowledge and logic, e.g., The sun is in the east, therefore it is morning. My dog is wagging its tail, therefore it is happy. The glass fell off the table, therefore it will break and the floor will become wet.
Also:
category
Which knowledge category or categories describe this claim? (choose all that apply)
- General reference
- Culture and the arts
- Geography and places
- Health and fitness
- History and events
- Human activities
- Mathematics and logic
- Natural and physical sciences
- People and self
- Philosophy and thinking
- Religion and belief systems
- Society and social sciences
- Technology and applied sciences
amirrr
joshnguyen99
Check how GPT labels statements on our labeling task. Use $Global R^2 = 1-\frac{mse(prediction,actual)}{mse(baseline,actual)}$ to score, and we can visualize in observable.
Would be nice to see how we do on each question.