search

open-thought / system-2-research

System 2 Reasoning Link Collection
Apache License 2.0
697 stars 57 forks source link

readme

OpenThought - System 2 Research Links

Here you find a collection of material (books, papers, blog-posts etc.) related to reasoning and cognition in AI systems. Specifically we want to cover agents, cognitive architectures, general problem solving strategies and self-improvement.

The term "System 2" in the page title refers to the slower, more deliberative, and more logical mode of thought as described by Daniel Kahneman in his book Thinking, Fast and Slow.

You know a great resource we should add? Please see How to contribute.

Cognitive Architectures

(looking for additional links & articles and summaries)

Agent Papers

LLM Based

LLM Reasoning Improvements / Training on Synthetic Data

Direct o1 Replication Efforts

Reward Models (ORM/PRM)

RL

MCTS

Minecraft Agents

Massive Sampling / Generate-and-Test

World Models

Neuro-Symbolic Approaches

Math

Active Inference

Prompting Techniques

Negative results

Mechanistic Interpretability

Blog Posts / Presentations

Graph Neural Networks

Complex Logical Query Answering (CQLA)

Answering logical queries over Incomplete Knowledge Graphs. Aspirationally this requires combining sparse symbolic index collation (SQL, SPARQL, etc) and dense vector search, preferably in a differentiable manner.

Inductive Reasoning over Heterogeneous Graphs

Similar to the regular CQLA, but with the emphasis on the "Inductive Setting" - i.e. querying over new, unseen during training nodes, edge types or even entire graphs. The latter part is interesting as it relies on the higher order "relations between relations" structure, connecting KG inference to Category Theory.

Neural Algorithmic Reasoning (NAR)

Initially attempted back in 2014 with general-purpose but unstable Neural Turing Machines, modern NAR approaches limit their scope to making GNN-based "Algorithmic Processor Networks" which learn to mimic classical algorithms on synthetic data and can be deployed on noisy real-world problems by sandwiching their frozen instances inside Encoder-Processor-Decoder architecture.

Grokking

Open-Source Agents & Agent Frameworks

Algorithms

Weak Search Methods

Weak methods are general but don't use knowledge (heuristics) to guide the search process.

Strong Search Methods

Books

Biologically Inspired Approaches

Diverse approaches some of which tap into classical PDE systems of biological NNs, some concentrate on Distibuted Sparse Representations (by default non-differentiable), others draw inspiration from Hippocampal Grid Cells, Place Cells, etc. Biological systems surpass most ML methods for Continual and Online Learning, but are hard to implement efficienly on GPU.

Dense Associative Memory

Dense Associative Memory is mainly represented by Modern Hopfield Networks (MHN), which can be viewed as a generalized Transformers capable of storing queries, keys and values explicitly (as in Vector Databases) and running recurrent retrival by energy minimization (relating them to Diffusion models). Application for Continual Learning is possible when combined with uncertainty quantification and differentiable top-k selection.

Continual Learning

Software Tools & Libraries

Commercial Offerings

Competitions & Benchmarks

Code

Related Projects

Youtube Content

Joscha Bach

Best LLM APIs

Novel model architectures

Philosophy: Nature of Intelligence & Consciousness

Biology / Neuroscience

Workshops

https://s2r-at-scale-workshop.github.io (NeurIPS 2024)

How to contribute

To share a link related to reasoning in AI systems that is missing here please create a pull request for this file. See editing files in the github documentation.