To augment language models with the ability to reason, researchers usuallyprompt or finetune them to produce chain of thought reasoning steps beforeproducing the final answer. However, although people use natural language toreason effectively, it may be that LMs could reason more effectively with someintermediate computation that is not in natural language. In this work, weexplore an alternative reasoning approach: instead of explicitly producing thechain of thought reasoning steps, we use the language model's internal hiddenstates to perform implicit reasoning. The implicit reasoning steps aredistilled from a teacher model trained on explicit chain-of-thought reasoning,and instead of doing reasoning "horizontally" by producing intermediate wordsone-by-one, we distill it such that the reasoning happens "vertically" amongthe hidden states in different layers. We conduct experiments on a multi-digitmultiplication task and a grade school math problem dataset and find that thisapproach enables solving tasks previously not solvable without explicitchain-of-thought, at a speed comparable to no chain-of-thought.
URL
Affiliations
Abstract
Translation (by gpt-3.5-turbo)
Summary (by gpt-3.5-turbo)