Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models

[Aidenzich]

Key Observation

• LLMs exhibit a significant decline in reasoning abilities when subjected to strict format restrictions.
• The stricter the format, the greater the performance degradation in reasoning tasks.
• Looser format restrictions tend to improve performance and reduce variance in reasoning tasks.

Prompt Examples

Standard Prompting:

Instruct: Provide your output in the following text format:
Step by step reasoning: ...
Answer: The final answer is ...
Format-Restricting Instructions

JSON Format Prompting:

Instruct: Provide your output in the following valid JSON format:
{
  "step_by_step_reasoning": "...",
  "answer": "..."
}

YAML Format Prompting:

Instruct: Provide your output in the following valid YAML format:
reasoning: |
  <think step by step>,
answer: <answer>

XML Format Prompting:

Instruct: Provide your output in the following valid XML format:
<root>
  <reason>[think step by step]</reason>
  <answer>[answer]</answer>
</root>

Specific Task Prompts

Mathematical Problem-Solving Task:
Given: A mathematical question or problem
Required: A numerical answer only
Role: You are a math tutor assisting students of all levels
Process: Think step by step to solve the problem

[Aidenzich]

Comparison

[Aidenzich]

The observed phenomena regarding the performance of large language models (LLMs) under format restrictions can be attributed to several key factors:

• Cognitive Load: Stricter format constraints, such as those imposed by JSON-mode, may increase the cognitive load on the model, making it difficult for the LLM to maintain coherent reasoning while also adhering to format specifications. This can lead to performance degradation, particularly in reasoning tasks.
• Decoupling of Reasoning and Format: In cases where format-restricting instructions (FRI) are used, the model might be better able to reason out answers in natural language before converting them to the required format. This two-step process allows for more effective reasoning compared to formats with strict constraints that force immediate structured output.
• Impact of Task Type: Different tasks interact differently with format constraints. For example, reasoning-intensive tasks tend to suffer more under strict format rules, while classification tasks might benefit from structured outputs that limit the range of possible answers, thus reducing errors.
• Parsing Errors: While parsing errors were initially thought to be a significant factor, the study found that they were not the primary cause of performance differences. Instead, the inherent restrictions of the format impacted the LLM's generation and reasoning processes more substantially.
• Variability Across Models: Various LLMs exhibit different strengths and weaknesses concerning format adherence, which means that the effectiveness of these formats can vary significantly based on the specific model architecture and training.

[Aidenzich]

My review

I think the observed phenomena can be seen as biases coming from the training dataset. The performance of large language models (LLMs) is constrained by their training data, and biases and inconsistencies in that data may affect the model's performance under specific format restrictions. Additionally, the differences in the model's performance when handling different tasks may reflect its biases toward certain formats or types of responses during training. Therefore, the performance variations under format restrictions may be related to inherent biases in the training dataset.