Neural Networks in Cuda from Scratch

This is a multi part tutorial on how to implement neural networks in CUDA.

The tutorials are available as videos on Youtube (Youtube Playlist) or in written+summarized form here on github.

I am planning to cover the following topics

1. Implementing activation layer Youtube Video
2. Linear layer Youtube Video
3. Forward pass with multiple layers Youtube Video
4. Forward pass with multiple inputs Youtube Video
5. Backpropagation
6. Applying gradient descent
7. Training loop with multiple iterations
8. Cost function
9. Scaling

Setup Cuda Development Environment

I will not create a “How to setup CUDA Development Environment” video. Since there are already many out there. I linked some guides below:

Windows/Visual Studio Guide CoffeeBeforeArch

Ubuntu