Paper Presentation

[ArtemisDicoTiar]

Faithful Reasoning Using Large Language Models

0. Abstract

1. Introduction

2. Defining a Valid Reasoning Trace

3. Components of a Faithful Reasoning Model

3.1 Selection-Inference: Valid Forward Reasoning

3.1.1. Selection

3.1.2. Inference

3.2. Halting: When to Stop Reasoning?

3.3. Search: Finding the Best Trace

4. Experimental Setup and Evaluation of Components

4.1. Datasets

4.2. Selection-Inference

4.3. Halter

4.4. Search

5. Experiments and Results

5.1. Baselines

5.2. Final Answer Accuracy

5.3. Evaluating Reasoning Trace Accuracy

5.4. Trace Validity

5.4.1. SI produces connected traces

5.4.2. SI produces correct inferences

5.4.3. SI uses its reasoning trace to answer the question

6. Related Work

6.1. Language Models are Not Enough

6.2. Reasoning with Language Models

6.3. Using Search for Reasoning Problems

6.4 The Problem of When to Stop Reasoning

7. Limitations

8. Discussion

A. Appendix

❗️Caution❗️: This section is much longer than any of you expected. (31 pages) This section includes a number of prompts and the model outputs.