Tensor API (ML framework front-end)

Project Abstract

This document proposes a Tensor class (multi-dimensional array) written in C++. Tensors are the primary data structure in all modern ML/DL framework and serve as a front-end for manipulating raw data and performing complex computations on data (matmuls, grads, etc.). The tensor class will be a high-level interface that abstracts away memory management, hardware acceleration and complex compute. This Tensor API will only incorporate simple tensor ops to begin with.

Conceptual Design

This simple tensor class will be written in C++ completely from scratch, relying only on the C++ STL. Utilizing C++ utilities such as move-semantics, templates, smart pointers, const expressions, concepts, ranges/views, std::span, etc. The raw memory buffer will be managed by a minimal Storage class containing a T* and a size. Each Tensor object points to an underlying Storage object. This allows different tensors to have different views on the same underlying data. I would also like to incorporate lazy evaluation and wrap it in python if we have time.

Proof of Concept

I plan on using Andrej Karpathy's simple Tensor class written in C as a guide to building our own implementation with added features of modern C++ (There are explicit instructions on how to compile and run in his repo). I also really like what George Hotz is doing with tinygrad, so there is some great stuff we can use from his framework as well (tinygrad is open-source). Although that is an entire DL framework, there are components that can be taken and modified for our simpler use case.

Andrej Karpathy (tensor) - https://github.com/EurekaLabsAI/tensor

George Hotz (tinygrad) - https://github.com/tinygrad/tinygrad tinygrad docs - https://docs.tinygrad.org

Background

This section is sort of redundant as the previous section (Proof of Concept) showcases two sources that I plan on utilizing as a guide for this project. You could also compare this tensor API to that you would see in pytorch or tensorflow (both open-source) just without the autograd engine and accelerator support (basically the ML part).

Required Resources

There is minimal requirement regarding compute resources as I don't plan on incorporating accelerator support or an autograd engine. I would recommend brushing up on C/C++ and playing around with PyTorch to get a feel for what a clean Tensor API feels like within a modern ML framework.

tensor

cis3296f24 / Section3ProjectPresentation

Tensor API (ML framework front-end) #15