NNFramework

[dmccloskey]

Description

Barebones neural network framework

Objectives

• [ ] SGS with adaptive and random learning rate
• [ ] Cost functions (e.g., 1/2(delta)**2)
• [ ] Mini batch with average update
• [ ] Foward propogation using an adjacency graph
• [ ] Backward propogation using an adjacency graph
• [ ] Node definitions (e.g., ReLU)
• [ ] Integration with Eigen

Objectives (Bonus)

• [ ] GPU distribution

Test

• Autoencoder

Notes

• biases are weights with unitary output (i.e., w*o where o = 1.0)
• apply a function to an eigen matrix: https://stackoverflow.com/questions/33786662/apply-function-to-all-eigen-matrix-element
• eigen tensor: http://eigen.tuxfamily.org/index.php?title=Tensor_support,

[dmccloskey]

Resources:

http://briandolhansky.com/blog/2013/9/27/artificial-neural-networks-backpropagation-part-4 http://briandolhansky.com/blog/2014/10/30/artificial-neural-networks-matrix-form-part-5

[dmccloskey]

GPU matrix operations

• https://developer.nvidia.com/cublas
• apply a function to an eigen matrix: https://stackoverflow.com/questions/33786662/apply-function-to-all-eigen-matrix-element
• eigen tensor: http://eigen.tuxfamily.org/index.php?title=Tensor_support,
• eigen tensor source: https://bitbucket.org/eigen/eigen/src/de7544f256bd/unsupported/Eigen/CXX11/src/Tensor/?at=default
• armadillo and nvblas: https://stackoverflow.com/questions/32679582/armadillonvblas-into-rcpparmadillonvblas
• nvblas: http://docs.nvidia.com/cuda/nvblas/
• armadillo: http://arma.sourceforge.net/faq.html#blas_lapack_replacements

[dmccloskey]

https://eigen.tuxfamily.org/dox/TopicCUDA.html http://eigen.tuxfamily.org/index.php?title=Tensor_support https://bitbucket.org/eigen/eigen/src/default/unsupported/Eigen/CXX11/src/Tensor/README.md?fileviewer=file-view-default