search

seanzhou1207 / ArgMining

1 stars 0 forks source link

Notes #1

Open seanzhou1207 opened 7 months ago

seanzhou1207 commented 7 months ago

Shared Task Description (Argmining 2022 on Validity and novelty) Task Description Paper

Team(Paper) Short Description ValNov(code) Validity Novelty
CLTeamL-3 GPT-3Val&Nov+NLIRoBERTaVal&Nov) 45.16 74.64 61.75
AXiS@EdUni-1 FFNNVal&Nov W/ NLIBART & WikiData 43.27 69.80 62.43
ACCEPT-1 SVMVal&Nov W/ ConceptNet & SBERT 43.13 59.20 70.00
CLTeamL-5 GPT-3Val&Nov+ARCRoBERTaVal&Nov 43.10 74.64 58.90
CSS* NLIRoBERTaVal&Nov 42.40 70.76 59.86
AXiS@EdUni-2 FFNNVal|Nov W/ NLIBART & WikiData 39.74 66.69 61.63
CLTeamL-2 NLIRoBERTaVal&Nov 38.70 65.03 61.75
CLTeamL-1 GPT-3Val&Nov 35.32 74.64 46.07
CLTeamL-4 ARCRoBERTaVal&Nov 33.11 56.74 58.95
ACCEPT-3 SVMVal&Nov W/ ConceptNet 30.13 58.63 56.81
ACCEPT-2 SVMVal|Nov W/ ConceptNet & SBERT 29.92 56.80 48.10
NLP@UIT SBERT 25.89 61.72 43.36
Baseline RoBERTaVal|Nov 23.90 59.96 36.12
Harshad BERTVal + novelty := validity 17.35 56.31 39.00
- overall system-average excluding the baseline 35.94 62.74 52.97

Table 3: Results (macro-F1-scores) for subtask A including short descriptions for each system. A “&” signifies a jointly trained Validity-Novelty-Predictor, a “|” sign validity and novelty predictions independent of each other.

seanzhou1207 commented 7 months ago

Literature Review

https://conceptnet.io

  1. Assessing the quality of arguments
  2. Demonstrated the difficulty of generating an argument that's both valid and novel
  3. Best novelty score achieved via the use of ConceptNet, a multilingual semantic network (Speer et al., 2017). The team constructed relationships between premises and conclusions using ConceptNet and ensured their semantic relatedness to the argument via [SBERT] (https://arxiv.org/abs/1908.10084). ACCEPT-1
  4. NLP@UIT