Essential theories for modeling

[karanotsingyu]

• Event Segmentation Model
  • Cognitive Psychology ver.
  • Computation ver.

Proposed in Baldassano et al., 2017.

• Baldassano, C., Chen, J., Zadbood, A., Pillow, J. W., Hasson, U., & Norman, K. A. (2017). Discovering Event Structure in Continuous Narrative Perception and Memory. Neuron, 95(3), 709-721.e5. https://doi.org/10.1016/j.neuron.2017.06.041

[karanotsingyu]

1. Cognitive Psychology ver.

Based on our experiments (described below), we propose that the full life cycle of an event can be described in a unified theory, illustrated in Figure 1. During perception, each brain region along the processing hierarchy segments information at its preferred timescale, beginning with short events in primary visual and auditory cortex and building to multimodal situation models in long-timescale areas, including the angular gyrus(角回) and posterior medial cortex(后内侧皮层). This model of processing requires that (1) all regions driven by audio-visual stimuli should exhibit event-structured activity, with segmentation into short events in early sensory areas and longer events in high-order areas; (2) events throughout the hierarchy should have a nested structure, with coarse event boundaries annotated by human observers most strongly related to long events at the top of the hierarchy; and (3) event representations in long-timescale regions, which build a coarse model of the situation, should be invariant across different descriptions of the same situation (e.g., when the same situation is described visually in a movie or a verbally in a story). We also argue that event structure is reflected in how experiences are stored into episodic memory. At event boundaries in long-timescale areas, the situation model is transmitted to the hippocampus, which can later reinstate the situation model in long-timescale regions during recall. This implies that (4) the end of an event in long-timescale cortical regions should trigger the hippocampus to encode information about the just-concluded event into episodic memory, and (5) stored event memories can be reinstated in long-timescale cortical regions during recall, with stronger reinstatement for more strongly encoded events. Finally, this process can come full circle, with prior event memories influencing ongoing processing, such that (6) prior memory for a narrative should lead to anticipatory reinstatement in long-timescale regions.

Figure 1.

[karanotsingyu]

2. Computational ver.

Figure 2. Event Segmentation Model
(A) Given a set of (unlabeled) time courses from a region of interest, the goal of the event segmentation model is to temporally divide the data into ‘‘events’’ with stable activity patterns, punctuated by ‘‘event boundaries’’ at which activity patterns rapidly transition to a new stable pattern. The number and locations of these event boundaries can then be compared across brain regions or to stimulus annotations. 
 (B) The model can identify event correspondences between datasets (e.g., responses to movie and audio versions of the same narrative) that share the same sequence of event activity patterns, even if the duration of the events is different.
(C) The model-identified boundaries can also be used to study processing evoked by event transitions, such as changes in hippocampal activity coupled to event transitions in the cortex. 
 (D) The event segmentation model is implemented as a modified Hidden Markov Model (HMM, 隐马尔科夫模型) in which the latent state st for each time point denotes the event to which that time point belongs, starting in event 1 and ending in event K. All datapoints during event K are assumed to be exhibit high similarity with an event-specific pattern $m_k$. See also Figures S1, S2, and S3.

(Ah... Latex syntax is not supported in GitHub issues) More details can be seen in the "Event Segmentation Model" part of the original article (HTML ver., or page e2 in PDF ver.)

---

Index of more inf.

• TD...

[karanotsingyu]

The HMM Model will be taught in tutorial 2 on W2D3.