Notes

[seanzhou1207]

Shared Task Description (Argmining 2022 on Validity and novelty) Task Description Paper

Team(Paper)	Short Description	ValNov(code)	Validity	Novelty
CLTeamL-3	GPT-3Val&Nov+NLIRoBERTaVal&Nov)	45.16	74.64	61.75
AXiS@EdUni-1	FFNNVal&Nov W/ NLIBART & WikiData	43.27	69.80	62.43
ACCEPT-1	SVMVal&Nov W/ ConceptNet & SBERT	43.13	59.20	70.00
CLTeamL-5	GPT-3Val&Nov+ARCRoBERTaVal&Nov	43.10	74.64	58.90
CSS*	NLIRoBERTaVal&Nov	42.40	70.76	59.86
AXiS@EdUni-2	FFNNVal|Nov W/ NLIBART & WikiData	39.74	66.69	61.63
CLTeamL-2	NLIRoBERTaVal&Nov	38.70	65.03	61.75
CLTeamL-1	GPT-3Val&Nov	35.32	74.64	46.07
CLTeamL-4	ARCRoBERTaVal&Nov	33.11	56.74	58.95
ACCEPT-3	SVMVal&Nov W/ ConceptNet	30.13	58.63	56.81
ACCEPT-2	SVMVal|Nov W/ ConceptNet & SBERT	29.92	56.80	48.10
NLP@UIT	SBERT	25.89	61.72	43.36
Baseline	RoBERTaVal|Nov	23.90	59.96	36.12
Harshad	BERTVal + novelty := validity	17.35	56.31	39.00
-	overall system-average excluding the baseline	35.94	62.74	52.97

Table 3: Results (macro-F1-scores) for subtask A including short descriptions for each system. A “&” signifies a jointly trained Validity-Novelty-Predictor, a “|” sign validity and novelty predictions independent of each other.

[seanzhou1207]

Literature Review

https://conceptnet.io

1. Assessing the quality of arguments
2. Demonstrated the difficulty of generating an argument that's both valid and novel
3. Best novelty score achieved via the use of ConceptNet, a multilingual semantic network (Speer et al., 2017). The team constructed relationships between premises and conclusions using ConceptNet and ensured their semantic relatedness to the argument via [SBERT] (https://arxiv.org/abs/1908.10084). ACCEPT-1
4. NLP@UIT