Compositional Semantic Parsing

[wesen]

What topic would you be interested in seeing an article/chapter about?

This technique is an extension of Least-to-Most, and could probably benefit from its own entry (haven't read the full paper yet).

• https://arxiv.org/abs/2209.15003

  . Our best method is based on least-to-most prompting: it decomposes the problem using prompting-based syntactic parsing, then uses this decomposition to select appropriate exemplars and to sequentially generate the semantic parse. This method allows us to set a new state of the art for CFQ while requiring only 1% of the training data used by traditional approaches

Any additional info you have about it:

There is more similar work linked in:

• https://arxiv.org/abs/2302.07842

  This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools

such as:

https://arxiv.org/abs/2212.04092 We introduce ``Successive Prompting'', where we iteratively break down a complex task into a simple task, solve it, and then repeat the process until we get the final solution. Successive prompting decouples the supervision for decomposing complex questions from the supervision for answering simple questions, allowing us to (1) have multiple opportunities to query in-context examples at each reasoning step (2) learn question decomposition separately from question answering, including using synthetic data, and (3) use bespoke (fine-tuned) components for reasoning steps where a large LM does not perform well. The intermediate supervision is typically manually written, which can be expensive to collect. We introduce a way to generate a synthetic dataset which can be used to bootstrap a model's ability to decompose and answer intermediate questions. Our best model (with successive prompting) achieves an improvement of ~5% absolute F1 on a few-shot version of the DROP dataset when compared with a state-of-the-art model with the same supervision.

This could probably be amended to the Least-to-Most section, or again moved to its own. Especially the synthesis of prompts is interesting as scaffolding prompts both for experimentation, practice, finetuning and training turns out to be an actual major roadblock of practical endeavours.

[wesen]

Oh! I actually just finished reading/studying the paper and can do a write up! Forgot I made a ticket for it.