ToDo

[takyamamoto]

I have a lot to do to improve the website.

[takyamamoto]

書く内容について

• 増強シナプスと減衰シナプスと関連して短期記憶に関して書く。
• シナプス伝達に関しては修正必要
• 生理の入門についてはhttps://salad-bowl-of-knowledge.github.io/hp/presentation/2020/04/13/voiceroid_powerpoint.htmlのスライドを元に作成する。
• motor learningについてはOFC, minimum_variance, kalman_lqgなど。実装公開した後に説明追加がよさそう。
• Predictive coding移植

[takyamamoto]

神経回路網の学習則

• 学習則と貢献度分配問題 credit assignment problem
• Hebb則
• STDPとBTDP(burst time-dependent plasticity)
• 誤差逆伝播法 (backpropagation)
• 誤差逆伝播法の適応例：3層NNでZipser Anderson
• target propagation
• feedback alignment
• eligibility trace and BPTT
• RFLO
• 進化的最適化
• 85% learning rule

[takyamamoto]

Energy-based モデル

• Ising model
• Hopfield model
• Boltzmann machine

自己組織化マップと教師なし学習

• orientation maps
• STDP 則と2 層WTA ネットワークによる教師なし学習

Reservoir Computing

• FORCE法

情報理論と最適化原理

• 情報理論の基礎
• Efficient coding仮説
• Infomax原理と独立成分分析

ベイズ脳理論と生成モデル

• スパース符号化：Sparse coding (Olshausen & Field, 1996) model
• 予測符号化：Predictive coding (Rao & Ballard, 1999) model
• VAE

時空間の符号化

• グリッド細胞の発火パターンの可視化
• grid cellのシミュレーション

運動学習

• Motor learning model

強化学習

• Distributional Reinforcement Learningの仕組み

神経細胞の形態と数理モデル

• ニューロンの形態と拡散律速凝集

[takyamamoto]

神経回路網の学習ダイナミクスは神経回路網の数理に置き換えた方が良いか？

• ランダムネットワークからガウス過程
• NTK

[takyamamoto]

神経回路網の構築 (発火率モデル)

• パーセプトロン
• 深層学習
• 畳み込みニューラルネットワーク

神経回路網の構築 (Spikingモデル)

神経回路網の演算処理

• ゲイン調整と四則演算
• 対数正規分布のシナプス重み
• Normalization model (Divisive normalization)
• WTA ネットワーク
• sub-LSTM model

[takyamamoto]

FHNモデル、モデルの１つとして取り扱う https://omedstu.jimdofree.com/2018/06/21/fitzhugh-nagumo%E3%83%A2%E3%83%87%E3%83%AB%E3%82%92%E3%82%A2%E3%83%8B%E3%83%A1%E3%83%BC%E3%82%B7%E3%83%A7%E3%83%B3%E3%81%A7%E8%A6%8B%E3%82%8B/

神経回路網の非線形ダイナミクス：位相場、カオス、同期

• https://en.wikipedia.org/wiki/Dynamical_neuroscience
• http://www-sop.inria.fr/members/Olivier.Faugeras/MVA/Slides10/Lecon4/Jonathan.Touboul.pdf
• https://www.jstage.jst.go.jp/article/biophys/54/4/54_195/_pdf

[takyamamoto]

Event-based Simulation of Spiking Neural Networks in Julia https://github.com/RainerEngelken/JuliaCon2017

位相縮約理論で章を作成 位相方程式によるSNNのシミュレーション

[takyamamoto]

1. HH, FHN https://en.wikipedia.org/wiki/FitzHugh%E2%80%93Nagumo_model Morris–Lecar modelも取り扱うか？ ML model https://en.wikipedia.org/wiki/Morris%E2%80%93Lecar_model https://www.frontiersin.org/articles/10.3389/fnins.2017.00123/full https://ieeexplore.ieee.org/document/7966284
2. LIF, QIF https://www.maths.nottingham.ac.uk/plp/pmzsc/neurodynamics/NonlinearIF.pdf
3. Iz モデル
4. Multicomp
5. 発火率
6. stochastic
7. ISI

↓に行くほど抽象度が高くなるようにする。

https://en.wikipedia.org/wiki/Biological_neuron_model

[takyamamoto]

教師なし学習は 競合学習(Competitive Learning) https://www.frontiersin.org/articles/10.3389/fncom.2015.00099/full

[takyamamoto]

PCAとHebbian

https://twitter.com/samuel_eckmann/status/1285280776007933957?s=20

[takyamamoto]

https://github.com/RainerEngelken/neurotheory-seminar-2019

[takyamamoto]

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2638500/

[takyamamoto]

nonsynaptic plasticityも書く https://en.wikipedia.org/wiki/Nonsynaptic_plasticity

Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits https://www.biorxiv.org/content/10.1101/2020.03.30.015511v1.full

BTDPも書く burst-time dependent plasticity

[takyamamoto]

多重ディスパッチ使おう

[takyamamoto]

Hopfield-model

https://repository.kulib.kyoto-u.ac.jp/dspace/bitstream/2433/95024/1/KJ00004760583.pdf https://www.pnas.org/content/79/8/2554 https://arxiv.org/abs/1606.01164 https://arxiv.org/abs/1701.00939 https://arxiv.org/abs/2008.02217 https://github.com/takyamamoto/Hopfield-Network

[takyamamoto]

Dayan & Abbott, 2001 Poisson process with Gaussian refractory period ISIはEI balanceに影響 https://mfr.ca-1.osf.io/render?url=https://osf.io/rbx2a/?direct%26mode=render%26action=download%26mode=render

[takyamamoto]

Neural sampling

Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002211

Sampling-based probabilistic inference emerges from learning in neural circuits with a cost on reliability https://arxiv.org/pdf/1807.08952.pdf

Neural Variability and Sampling-Based Probabilistic Representations in the Visual Cortex https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5077700/

Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference https://www.nature.com/articles/s41593-020-0671-1

Knill, D. & Richards, W. Perception as Bayesian Inference (Cambridge Univ. Press, 1996).

Perceptual decision-making as probabilistic inference by neural sampling

patio-temporal representations of uncertainty in spiking neural networks https://papers.nips.cc/paper/5343-spatio-temporal-representations-of-uncertainty-in-spiking-neural-networks

sampling-based probabilistic inference

Bayesian Brains without Probabilities https://www.sciencedirect.com/science/article/pii/S1364661316301565

島崎先生の論文 https://arxiv.org/abs/1902.11233 https://arxiv.org/abs/2006.13158

The Hamiltonian Brain: Efficient Probabilistic Inference with Excitatory-Inhibitory Neural Circuit Dynamics https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005186

[takyamamoto]

Ising model https://www.frontiersin.org/articles/10.3389/fncir.2020.00054/full

[takyamamoto]

Hopfield Networks is All You Need https://ml-jku.github.io/hopfield-layers/

[takyamamoto]

https://motorcontrol.jp/mc13/MC2019_2_OptimalControlStochastic.pdf

[takyamamoto]

CCNBook https://grey.colorado.edu/CompCogNeuro/index.php/CCNBook/Main

[takyamamoto]

https://github.com/kylerbrown/python-for-neuroscience

[takyamamoto]

https://alleninstitute.org/what-we-do/brain-science/events-training/summer-workshop-dynamic-brain/swdb-datasets/

[takyamamoto]

https://arxiv.org/abs/2009.01791

[takyamamoto]

Information Theory is abused in neuroscience https://philpapers.org/rec/NIZITI

[takyamamoto]

Network http://www.mibel.cs.tsukuba.ac.jp/~s-tugawa/jikken/model.pdf http://www.metabolomics.jp/wiki/Aritalab:Lecture/NetworkBiology/Erdos-Renyi_Model

Graph Structure of Neural Networks https://arxiv.org/abs/2007.06559

[takyamamoto]

https://www.cell.com/neuron/fulltext/S0896-6273(20)30651-6

[takyamamoto]

https://www.nature.com/articles/s41593-020-0699-2

[takyamamoto]

Path integrals and SDEs in neuroscience

Part1 : http://benlansdell.github.io/statistics/sdes/

Part2 : http://benlansdell.github.io/statistics/sdesII/

[takyamamoto]

Path Integral Methods for Stochastic Differential Equations https://arxiv.org/abs/1009.5966

Path integration https://en.wikipedia.org/wiki/Path_integration

Line integral

Spiking activity propagation in neuronal networks: reconciling different perspectives on neural coding https://www.nature.com/articles/nrn2886

Dynamic representations in networked neural systems https://www.nature.com/articles/s41593-020-0653-3

[takyamamoto]

Third-order nanocircuit elements for neuromorphic engineering https://www.nature.com/articles/s41586-020-2735-5

[takyamamoto]

Artificial Neural Networks for Neuroscientists: A Primer https://www.cell.com/neuron/fulltext/S0896-6273(20)30705-4

[takyamamoto]

①ニューラルネットのパラメータの更新則はランジュバン方程式に従う　⇒　学習プロセスを確率熱力学の言葉で説明できる https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.118.010601 ②ベイズ統計と統計力学の形式的類似性を追求する　⇒　熱容量に対応する「学習容量」を発見した https://journals.aps.org/pre/abstract/10.1103/PhysRevE.99.052140

[takyamamoto]

http://www.offconvex.org/2016/11/03/MityaNN1/

[takyamamoto]

Distributional Reinforcement Learning in the Brain https://www.sciencedirect.com/science/article/pii/S0166223620301983

[takyamamoto]

Cx3Dについての補足 https://www.ini.uzh.ch/~amw/seco/cx3d/

[takyamamoto]

http://neuromorpho.org/index.jsp

[takyamamoto]

https://twitter.com/hbouammar/status/1489215674014703618?s=20&t=D-XJmf15Br6Ph4uo9h_uDw

[takyamamoto]

$$ y=x^2 $$

Note This is a note

Check This is a note

Warning This is a warning