daveshap / Raspberry

Create an open source toy dataset for finetuning LLMs with reasoning abilities
MIT License
400 stars 28 forks source link

Create self-contained and provable logic puzzles #52

Open daveshap opened 1 month ago

daveshap commented 1 month ago

Overview

Example: "write a 10 word coherent sentence where each word has exactly 1 letter more than its previous word starting with first word having one letter and so on till tenth word having 10 letters"

We as many of these kinds of logic puzzles as possible. The main criteria are:

Abstract Criteria

a) It combines multiple constraints simultaneously. b) It requires balancing form and function. c) It involves pattern recognition and application. d) It necessitates creative problem-solving within strict parameters. e) It tests the ability to optimize multiple variables at once. f) It challenges the solver to find a solution that satisfies all conditions.

Potential Categories:

Number sequences: Create a sequence with a hidden pattern.

Example: What's the next number in the sequence: 1, 11, 21, 1211, 111221, ...? Verification: This is the "look and say" sequence. Each number describes the previous number.

Word puzzles: Create puzzles involving letter manipulation or word patterns.

Example: What 8-letter word can have a letter taken away and still form a word, take another letter away and still form a word, and take a third letter away and still form a word? Verification: The word is "starting". starting → staring → string → sting

Logic puzzles: Create scenarios that require deductive reasoning.

Example: In a room are two men. One always lies, and one always tells the truth. You don't know which is which. You can ask one question to one of them to determine which door leads to freedom. What question do you ask? Verification: Ask either man: "If I asked the other man which door leads to freedom, what would he say?" Then choose the opposite door.

Mathematical puzzles: Create problems that require creative thinking with numbers.

Example: Using only addition, how can you add eight 8's and get the number 1000? Verification: 888 + 88 + 8 + 8 + 8 = 1000

Lateral thinking puzzles: Create scenarios that require "thinking outside the box".

Example: A man pushes his car to a hotel and tells the owner he's bankrupt. Why? Verification: He's playing Monopoly.

Definition of Done

Create a dataset of self-contained logic puzzles. At least 100 samples.

Stretch Goal

Create an automated system that can generate an arbitrary number of provable logic puzzles.

daveshap commented 1 month ago

Some brainstorming with Claude

I apologize for the overly list-heavy format. You're right that it needs to be more comprehensible to those unfamiliar with the project. Let me revise the summary to be more narrative and accessible:

Developing Challenging Brain Teasers: A Creative Approach

Project Overview

This project aims to create a collection of brain teasers that challenge people's reasoning, mathematical, and logical thinking skills. The puzzles are designed to be solvable without external resources, making them ideal for various settings such as classrooms, team-building exercises, or personal entertainment.

The Inspiration

The project was inspired by a unique word puzzle:

"Write a 10-word coherent sentence where each word has exactly 1 letter more than its previous word, starting with the first word having one letter and ending with the tenth word having 10 letters."

This puzzle showcases the core elements we're aiming for: creativity within constraints, pattern recognition, and the balance between form and function.

Our Approach

Initially, we explored various categories of brain teasers, including number sequences, word puzzles, logic problems, mathematical challenges, and lateral thinking exercises. However, as we delved deeper, we realized that the most intriguing puzzles often transcend these categories.

We began to look at puzzle creation more abstractly. The key, we found, was not in the specific type of puzzle, but in the thinking processes it required. The best puzzles combine multiple constraints, require creative problem-solving, and challenge solvers to optimize several variables simultaneously.

Expanding the Concept

With this new perspective, we started to imagine puzzles that could apply these principles across different domains. For instance:

These examples demonstrate how the core concept can be adapted to various fields, always maintaining the balance between rigid structure and creative expression.

The Creative Process

Creating these puzzles is a puzzle in itself. It requires:

  1. Identifying the core elements that make a puzzle challenging and engaging.
  2. Establishing a clear set of rules or constraints.
  3. Ensuring that despite these constraints, there's room for creative solutions.
  4. Balancing difficulty so that the puzzle is challenging but not impossible.

Looking Forward

As this project progresses, we aim to develop a diverse collection of brain teasers that not only entertain but also enhance problem-solving skills and encourage thinking outside the box. These puzzles have the potential to be used in educational settings, cognitive training programs, or simply as a fun way to keep minds sharp and engaged.

By pushing the boundaries of traditional puzzle types, we hope to create a new category of brain teasers that challenges people to think in novel ways, combining creativity with logical reasoning across various domains of knowledge and skill.

DataBassGit commented 1 month ago

We do have some public datasets that we can use for this. These don't seem to be in distribution for training material either.

https://www.kaggle.com/datasets/prgmaz/cs779-brainteaser https://www.kaggle.com/datasets/mathurinache/math-dataset https://www.kaggle.com/datasets/anthonytherrien/single-prompt-medium-to-hard-math-questions https://www.kaggle.com/datasets/thedevastator/arc-grade-school-science-questions https://www.kaggle.com/datasets/radek1/sci-or-not-sci-hypthesis-testing-pack https://www.kaggle.com/datasets/allenai/ai2-science-questions

daveshap commented 1 month ago

https://www.kaggle.com/datasets/prgmaz/cs779-brainteaser https://www.kaggle.com/datasets/mathurinache/math-dataset

These two look like the most useful by far. :D

daveshap commented 1 month ago

I gave the following problem to o1-preview and it did okay.

I need you to generate more problems like the following:

Overview
Example: "write a 10 word coherent sentence where each word has exactly 1 letter more than its previous word starting with first word having one letter and so on till tenth word having 10 letters"

We as many of these kinds of logic puzzles as possible. The main criteria are:

1. Provable (mathematically or logically verifiable)
2. Self-contained (no outside resources required)
3. Require logic, math, planning, lateral thinking
4. No visual elements (alphanumeric only)

Abstract Criteria
a) It combines multiple constraints simultaneously.
b) It requires balancing form and function.
c) It involves pattern recognition and application.
d) It necessitates creative problem-solving within strict parameters.
e) It tests the ability to optimize multiple variables at once.
f) It challenges the solver to find a solution that satisfies all conditions.

Please generate 10 such new questions

Sure! Here are 10 new logic puzzles that meet your criteria:

  1. Sentence Construction Challenge: Write a coherent sentence of exactly eight words where each word has one more syllable than the previous word, starting with a one-syllable word and ending with an eight-syllable word.

  2. Alphabetical Sequence Sentence: Create a sentence of five words where each word starts with a consecutive letter of the alphabet (e.g., A, B, C, D, E), and each word is one letter longer than the previous word.

  3. Number Puzzle: Find a three-digit number such that the sum of its digits equals the product of its digits, and the number is a palindrome.

  4. Magic Square Arrangement: Using the numbers 1 to 9 exactly once, arrange them in a 3x3 grid (magic square) where the sums of each row, column, and diagonal are all equal.

  5. Word Ladder Transformation: Transform the word "HEAD" into "TAIL" by changing one letter at a time, with each intermediate step being a valid English word, and complete the transformation in exactly four steps.

  6. Vowel-Consonant Balance: Find a seven-letter English word where the number of vowels equals the number of consonants, and no letters repeat.

  7. Arithmetic Sequence Challenge: Create a sequence of five positive integers where each number is two more than the previous one, and the sum of the squares of these numbers equals 275.

  8. Equation Formation Puzzle: Using each of the digits 1 to 9 exactly once, fill in the blanks to create a true equation: [ \square \times \square + \square = \square \times \square - \square = \square \times \square - \square ]

  9. Anagram Sentence: Write a coherent sentence using four words that are all anagrams of each other.

  10. Digit Reversal Number: Find a two-digit number such that when you reverse its digits and subtract the smaller number from the larger, the result is a perfect square.

Each of these puzzles combines multiple constraints and requires logical reasoning, pattern recognition, and creative problem-solving to find a solution that satisfies all conditions.