Chapter 2 Connectionist Models of Cognition(page 37-65)

Artificial neural networks (ANN) or connectionist systems wikipedia

Introduction

the use of neural networks 以神经网络计算为核心
inspired by research into how computation works in the brain 灵感来自对大脑如何计算的研究
Processing is characterized by patterns of activation across simpleprocessingunitsconnectedtogetherinto complex networks. 处理的模式是把简单处理单元的激活模式连接到复杂的网络中
Knowledge is stored in the strength of the connections between units 信息存储于单元之间的连接强弱
connectionism 因此被称为联结主义

Background

3个重要早期模型

the Interactive Activation model of letter recognition (McClelland & Rumelhart, 1981; RumelhartandMcClelland,1982) 字母识别的交互激活模型
Rumelhartand McClelland’s (1986) model of the acquisition of the English past tense 英语过去时习得模式
Elman’s (1991) simple recurrent network for ﬁnding structure in time 用于发现结构的简单递归网络

2.1 Historical Context （可输出为时间线）

联结模型的灵感来自于这样一个概念：神经系统的信息处理特性会影响认知理论

神经元的发现成为联结主义的理论来源（19世纪下半页）

19世纪下半页，神经细胞发现（参考：[卡哈尔]画的神经元、高尔基），人们开始思考神经元在思维过程的作用。当时流行的一些心理过程的联想主义理论引出了早期神经网络理论
1929年的Lashley的脑损伤实验数据显示大脑表现仅和脑损伤区域大小，联结主义理论受到质疑。

再次发展人们对用数学技术描述神经细胞网络重新产生兴趣（1930s-1940s）

Warren McCulloch 和 Walter Pitts 于1943年从数学运算的角度描述了二元神经元简单网络的功能，是最早的神经网络模型。
1949年，Hebb在《The Organization of Behavior》一书提出认知的细胞组装理论，其重要观点是：学习过程可能产生特定神经突触连接的变化，是最早的神经元学习规则。（就是著名的赫布学习规则）
10年后，Rosenblatt 制定了两层神经网络的学习规则，从数学上证明了感知器收敛规则，即感知器。它可以调整连接简单神经元的输入层和输出层的权重，前馈神经网络雏形形成。不过，Minsky and Papert 证明两层可计算函数的集合的局限性。电脑的计算能力不够强，也没有算法来学习系统的连接。
Rumelhart, Hinton和Williams在1986发表的一篇重要论文证明了使用反向传播训练网络在解决神经网络面临的关键计算和认知挑战方面的有效性。

1970年代，序列处理和冯·诺依曼计算机隐喻主导了认知心理学。然而，许多研究人员继续研究神经系统的计算特性。(原文后面接了若干模型，就是下图里的内容) 一个简化的示意图，显示神经网络架构的历史演变的简化示意图。 SRN=simple recurrent network.（page 39）（循环神经网络、Elman网络、长短期记忆网络、双向循环网络...）
2006年，深度信念网络（Deep Belief Network） 发表，迎来深度学习的研究热潮。

2.2. Key Properties of Connectionist Models 联结主义的关键属性

联结主义模型的灵感来自神经系统：一组简单的处理单元并行计算，并通过一个加权的网络相互影响激活状态。 Rumelhart, Hinton, and McClelland (1986) identiﬁed seven key features that would deﬁne a general framework for connectionist processing.

第一个特性 处理单元集 ui。比如认知模型中的个体的概念(如字母或单词)。处理单元通常分为输入单元、输出单元和隐藏单元。在关联网络中，输入和输出单元的状态由建模任务定义(至少在训练期间)，隐藏单元是自由参数，其状态可由算法根据需要确定。
第二个特征给定时间(t)的激活状态(a)。单元集的状态通常用实数向量a(t)表示，一般取值[0-1]。
第三个特征连接模式。任何两个单元间的连接强度将决定随后时间点中一个单元的激活状态对另一个单元的激活状态程度的影响。单元i与j之间的连接强度可以用权重值为wij的矩阵W表示。通常单元被安排成层(例如，输入、隐藏、输出)，并且单元的层之间是完全连接的。
第四个特性传播激活状态的规则，适用于整个网络。该规则把发送激活的处理单元的输出值向量a(t)和连接矩阵W结合，生成每个接收单元的总和或净输入。接收单元的净输入由向量和矩阵相乘得到。
第五个特性激活规则，用于指定如何组合给定单元的净输入以生成其新的激活状态。
第六个特征 “根据经验修改连接模式的算法”。其观点是，两个单元之间的权重应该根据单元相关活动的比例改变。例如，如果一个单元ui接收到另一个单元uj的输入，那么如果两个单元都是高度活跃的，那么从uj到ui的权重wij就应该加强。
连接主义网络的最后一个普遍特征是环境对系统的表征。假定它由一组外部提供的事件或用于生成此类事件的函数组成。事件可以是单个模式，如可视输入;相关模式的集合，如拼写一个字及其相应的声音或意义;或输入序列，如句子中的单词。
190102 0012 wjj cut English part
190119 1927 wjj add chapter 2.2
190119 0129 wjj add chapter 2.1

Hinton内容补充

来自剑桥手册

A key paper by Rumelhart, Hinton, and Williams (1986) demonstrated the usefulness of networks trainedusing back propagation for addressing key computational and cognitive challenges facing neural networks.
interactive networks
Rumelhart, Hinton, and McClelland (1986) identiﬁed seven key features that would deﬁne a general framework for connectionist processing.
对符号主义和联结主义的讨论

参考文献

Rumelhart, D. E., Hinton, G. E., & Williams, R. J.(1986).Learninginternalrepresentationsby error propagation. In D. E. Rumelhart, J. L. McClelland and The PDP Research Group, Parallel distributed processing: Explorations in the microstructureof cognition. Volume 1: Foundations (pp. 318–362). Cambridge, MA: MIT Press.
Rumelhart, D. E., Hinton, G. E., & McClelland, J. L. (1986). A general framework for parallel distributed processing. In D. E. Rumelhart, J. L. McClelland, & the PDP Research Group, Parallel distributed processing: Expolrations in the microstructure of congnition.Volume 1: Foundations (pp. 45–76). Cambridge, MA: MIT Press.
Rumelhart, D. E., Smolensky, P., McClelland, J. L., & Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. In J. L. McClelland & D. E. Rumelhart (Eds.), Parallel distributed processing, Vol. 2 (pp. 7–57). Cambridge, MA: MIT Press.
Hinton, G. E., & Sejnowksi, T. (1986). Learning and relearning in Boltzmann machines. In D. Rumelhart & J. McClelland (Eds.), Paralleldistributedprocessing,Vol.1(pp.282–317). Cambridge, MA: MIT Press.
Touretzky, D. S., & Hinton, G. E. (1988). A distributed connectionist production system. Cognitive Science, 12, 423–466.
Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducingthedimensionalityofdatawithneural networks. Science, 313(5786), 504–507.

changelog

190119 1707 wjj add Hinton information from handbook

CHAPTER 5 Declarative/Logic-Based Cognitive Modeling(page 142-180)

1.Introduction

1.1 What Is Logic-Based Computational Cognitive Modeling?

The basic units of such modeling are declarative, or propositional: They are formal objects associated with those particularsentences or expressions in natural languages
Declarative computational cognitive modeling is a top-down, rather than bottom-up, approach. 陈述性计算认知建模是自上而下，而非自下而上。

1.2. Level of Description of LCCM

1.3. The Ancient Roots of LCCM （three challenges）

1.4. LCCM’s Sister Discipline: Logic-Based Human-Level AI （the logicomathematical foundation for LCCM）

1.5. Different Levels of Description （how LCCM addresses the three aforementioned challenges）

1.6. Brief Overview of the Three Challenges （ the future and limitations of computational cognitive modeling ）

1.7. Structure of the Chapter 已添加至各标题括号

2. The Goal of Computational Cognitive Modeling/LCCM

To understand the kind of cognition distinctive of human persons by modeling this cognitionininformationprocessingsystems.

3. Three Challenges Facing Computational Cognitive Modeling

3.1. Challenge 1: Computational Cognitive Modeling Data from Psychology of Reasoning

3.2. Challenge 2: Unify Cognition via a Comprehensive Theoretical Language

3.3. Challenge 3: Computational Cognitive Meodeling Suffers from a Lack of Mathematical Maturity

4. Logic-Based Computational Cognitive Modeling

4.1. Logical Systems

4.2. Sample Declarative Modeling in Conformity to LCCM

4.3. Logic-Based Computer Programming

5. Meeting the Challenges

5.1. Meeting the Challenge of Mechanizing Human Reasoning

5.2. Meeting the Perception/Action Challenge

5.3. Meeting the Rigor Challenge

6. Limitations and the Future

7. Conclusion

CHAPTER 5 Declarative/Logic-Based Cognitive Modeling(page 142-180)

参考

1.Introduction

1.1 What Is Logic-Based Computational Cognitive Modeling?

The basic units of declarative computational cognitive modeling are declarative, or propositional: They are formal objects associated with those particularsentences or expressions in natural languages. 陈述性计算认知模型的基本单元是声明式或命题式的。
The basic process over such units is inference, which may be deductive, inductive, probabilistic, abductive, or analogical. 单元的基本过程是推理，可以是演绎的、归纳的、概率的、外展的或类比的。
Declarative computational cognitive modeling is a top-down, rather than bottom-up, approach. 陈述性计算认知建模是自上而下，而非自下而上。

As Brachman and Levesque (2004) put it, when speaking of declarative computational cognitive modeling within the ﬁeld of artiﬁcial intelligence (AI): It is at the very core of a radical idea about how to understand intelligence: instead of trying to understand or build brains from the bottom up, we try to understand or build intelligent behavior from the top down.Inparticular, we ask what an agent would need to know in order to behave intelligently, and what computational mechanisms could allow this knowledge to be made available to the agent as required.

Logic-based computational cognitive modeling is an interdisciplinary ﬁeld that cuts across cognitive modeling based on certain cognitive architectures(such as ACT-R, Soar, CLARION, Polyscheme, etc.)

1.2. Level of Description of LCCM（略）

1.3. The Ancient Roots of LCCM （three challenges）

Declarative computational cognitivemodeling is the oldest paradigm for modeling the mind. Over 300 years b.c., logic and logic alone was being used to model and predict human cognition. 声明式计算认知模型是最古老的表征思维的范式。公元前300年人们已经开始使用逻辑学来建模和预测。

For example, consider the following argument:（经典三段论） (1) All professors are pusillanimous people. (2) All pusillanimous people are proud. ∴ (3) All professors are proud. Thesymbol∴,oftenreadas“therefore,”says that statement (3) can be logically inferred from statements (1) and (2); or in other wordsthatifstatements(1)and(2)aretrue, then (3) must be true as well.

(1∗) AllAs are Bs. (2∗) AllBs are Cs. ∴ (3∗) AllAs are Cs.

1.4. LCCM’s Sister Discipline: Logic-Based Human-Level AI

1.5. Different Levels of Description

1.6. Brief Overview of the Three Challenges

Challenge 1: Human reasonin
Challenge 2: 自Newell发展起来的很多种不同的计算认知架构，但是他们的计算能力都比较有限。有两个原因：核心机制不足以表征人类思维复杂度。我们需要一种核心机制，这种机制能够实现高维度的推理和元分析以及对外部环境的感知和操作。
Challenge 3: 大多数计算认知结构都没有清晰精确的语法，而计算认知建模还没有像物理数学计算机那样的清晰准确的语言和定理

2. The Goal of Computational Cognitive Modeling/LCCM

To understand the kind of cognition distinctive of human persons by modeling this cognitionininformationprocessingsystems.

3. Three Challenges Facing Computational Cognitive Modeling

3.1. Challenge 1: Computational Cognitive Modeling Data from Psychology of Reasoning

3.2. Challenge 2: Unify Cognition via a Comprehensive Theoretical Language

3.3. Challenge 3: Computational Cognitive Meodeling Suffers from a Lack of Mathematical Maturity

4. Logic-Based Computational Cognitive Modeling

4.1. Logical Systems

4.2. Sample Declarative Modeling in Conformity to LCCM

4.3. Logic-Based Computer Programming

5. Meeting the Challenges

5.1. Meeting the Challenge of Mechanizing Human Reasoning

5.2. Meeting the Perception/Action Challenge

5.3. Meeting the Rigor Challenge

6. Limitations and the Future

7. Conclusion

Chapter 5

一些摘录

Furthermore, the purpose of this chapter is not to introduce a new competitor to extant, mature computational cognitive architectures, such as Soar (Rosenbloom, Laird & Newell, 1993), ACT-R (Anderson 1993; Anderson & Lebiere, 1998; Ander- son, & Lebiere, 2003), CLARION (Sun, 2002), ICARUS (Langley et al., 1991), SNePS (Shapiro & Rapaport, 1987), and Polyscheme Cassimatis, 2002; Cassimatis et al., 2004), nor to declarative computational simulations of parts of human cognition, such as PSYCOP (Rips, 1994), and programs written by Johnson-Laird and others to simulate various aspects of so-called mental models-based reasoning (a review is provided in Bucciarelli & Johnson-Laird, 1999). These systems are all pitched at a level well above LCCM; they can all be derived from LCCM. The formal umbrella used for the systematization herein is to offer a way to understand and rationalize all computational cognitive architectures that are declarative, that is, that are, at least in part, rule-based, explicitly logic-based, predicate-and-argument-based, propositional, and production-rule-based. The ancient roots of this kind of work date back to Aristotle.

此外，本章的目的不是为现存的，成熟的计算认知架构引入一个新的竞争者，如Soar（Rosenbloom，Laird＆Newell，1993），ACT-R（Anderson 1993; Anderson＆Lebiere，1998; Anderson，＆Lebiere，2003），CLARION（Sun，2002），ICARUS（Langley等，1991），SNePS（Shapiro＆Rapaport，1987），和Polyscheme Cassimatis，2002; Cassimatis et al。，2004），也不是人类认知部分的声明性计算模拟，如PSYCOP（Rips，1994），以及Johnson-Laird和其他人为模拟各方面而编写的程序。称为基于心智模型的推理（Bucciarelli＆Johnson-Laird，1999中提供了一篇综述）。这些系统都处于远高于LCCM的水平;它们都可以来自LCCM。用于本文系统化的正式um-brella是提供一种方法来理解和定位所有声明性的计算认知架构，即至少部分基于规则，明确基于逻辑的，基于谓词和论证，命题和生产规则。这种工作的古老根源可以追溯到亚里士多德。

TERM 联结主义

联结主义的基本思想早在20世纪40年代，其基本思想就有表达（McCulloch & Pitts，1943）。但联结主义因未得到重视和资金支持，其研究受到限制而逐渐落入低潮。直到1986年鲁梅尔哈特（Rumelhart）等人提出多层网络中的反向传播（BP）算法，联结主义势头大振，多方面取得重要进展（叶浩生，2005）。

《并行分布加工：认知的微观结构之探索》（Mcclelland & Rumelhart，1986），第一次系统阐述了联结主义的观点和成就，因此这一著作被称为联结主义的里程碑式的著作（叶浩生，2005）。

McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5(4), 115-133. Rumelhart, D. E., & McClelland, J. L. (1986). Parallel distributed processing: explorations in the microstructure of cognition. volume 1. foundations. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. nature, 323(6088), 533. 叶浩生. (2005). 心理学史. 高等教育出版社,p.287.

karanotsingyu / AwesomeCoCoSci