Compositionality and Sentence Meaning: Comparing Semantic Parsing and Transformers on a Challenging Sentence Similarity Dataset

[shirawein]

CL Preprint:

https://direct.mit.edu/coli/article/doi/10.1162/coli_a_00536/124463/Compositionality-and-Sentence-Meaning-Comparing

[xiulinyang]

Interesting paper! I read the abstract and conclusion - it seems the research question (if different types of models can predict human judgements of semantic similarity) is quite theoretical and an AMR parser is taken as a representative of syntax-based parser. Should we tag it under ‘Applications,’ or should we create a new category for this, like ‘Linguistic Insight/Theory’?

[shirawein]

Ooh, good question. This kind of examination of the utility of AMR (in this case, how effectively AMR captures meaning in comparison to human judgments of semantic similarity), I tend to mark as `Evaluation' -- of course, in addition to evaluation metrics for AMR. That's probably what I'd recommend in this case but I agree that these days it is often hard to mark new papers as fitting into one of the existing labels.

[xiulinyang]

Yeah, it makes sense! I will take 'Evaluation' then! Thanks:)