Tensor API (ML framework front-end)

[magwinst]

Project Abstract

This document proposes a Tensor class (multi-dimensional array) written in C++. Tensors are the primary data structure in all modern ML/DL framework and serve as a front-end for manipulating raw data and performing complex computations on data (matmuls, grads, etc.). The tensor class will be a high-level interface that abstracts away memory management, hardware acceleration and complex compute. This Tensor API will only incorporate simple tensor ops to begin with.

Conceptual Design

This simple tensor class will be written in C++ completely from scratch, relying only on the C++ STL. Utilizing C++ utilities such as move-semantics, templates, smart pointers, const expressions, concepts, ranges/views, std::span, etc. The raw memory buffer will be managed by a minimal Storage class containing a T* and a size. Each Tensor object points to an underlying Storage object. This allows different tensors to have different views on the same underlying data. I would also like to incorporate lazy evaluation and wrap it in python if we have time.

Proof of Concept

I plan on using Andrej Karpathy's simple Tensor class written in C as a guide to building our own implementation with added features of modern C++ (There are explicit instructions on how to compile and run in his repo). I also really like what George Hotz is doing with tinygrad, so there is some great stuff we can use from his framework as well (tinygrad is open-source). Although that is an entire DL framework, there are components that can be taken and modified for our simpler use case.

Andrej Karpathy (tensor) - https://github.com/EurekaLabsAI/tensor

George Hotz (tinygrad) - https://github.com/tinygrad/tinygrad tinygrad docs - https://docs.tinygrad.org

Background

This section is sort of redundant as the previous section (Proof of Concept) showcases two sources that I plan on utilizing as a guide for this project. You could also compare this tensor API to that you would see in pytorch or tensorflow (both open-source) just without the autograd engine and accelerator support (basically the ML part).

Required Resources

There is minimal requirement regarding compute resources as I don't plan on incorporating accelerator support or an autograd engine. I would recommend brushing up on C/C++ and playing around with PyTorch to get a feel for what a clean Tensor API feels like within a modern ML framework.

[123GetBuckets]

I heard about tinygrad in 2023, this project seems fun

[jayhaych02]

Feedback on Tensor Class Proposal

Why I Find This Project Interesting:

This proposal is exciting because tensors are the foundation of modern ML/DL frameworks, and building a high-level tensor class in C++ with modern utilities like smart pointers and lazy evaluation is both ambitious and essential. The use of move semantics and memory management abstractions will lead to a cleaner and more efficient design.

How I Can Contribute:

I’d love to contribute by exploring performance optimizations, especially in areas like hardware acceleration with CUDA or OpenCL for GPU support. Additionally, I can help optimize memory handling with efficient tensor views and explore multi-threading or SIMD for faster operations.

Suggestions for Improvement:

• Consider diving deeper into lazy evaluation, drawing inspiration from libraries like Eigen for optimized performance.
• Look into parallelization and SIMD for accelerating tensor operations on CPUs.
• Exploring GPU acceleration as a future extension could unlock significant performance improvements, especially for large-scale tensor computations.

[123GetBuckets]

Feedback

What I find Interesting: I prefer working in lower level languages, and this project excites me because it is centered around C++, tensors and optimization. ML/DL in C++ is incredibly efficient but is only made possible through highly efficient and foolproof software design. This project would be a great opportunity to learn about building fast algorithms.

How I can contribute: I don't have much experience in C++, but I know C pretty well and I can understand the concepts for this project. I also have experience writing performance evaluation software, which I think would be useful in measuring the progress and demonstration purposes.

[kripseepatel]

1.  This project stands out as one of the best proposals because it combines deep technical complexity with a clear focus on building fundamental tools used in machine learning frameworks. The idea of constructing a custom Tensor class in C++ from scratch demonstrates a strong commitment to understanding and improving the performance and functionality of the core data structures behind ML and DL frameworks. What makes this project particularly appealing is its grounded, yet ambitious approach, utilizing modern C++ techniques and drawing inspiration from leaders like Andrej Karpathy and George Hotz. Personally, I appreciate the balance between simplicity and scalability in this project, as it allows for a manageable starting point while leaving room for future enhancements.
2.  I can contribute to this project by bringing my knowledge of C++ and system-level programming to refine the memory management and optimization aspects of the Tensor class. Specifically, I can help with the implementation of move semantics, smart pointers, and efficient memory allocation using STL features to ensure the class is both performant and extensible. Additionally, I can work on incorporating lazy evaluation into tensor operations, allowing for more efficient computational pipelines by delaying execution until necessary. I am also interested in exploring multi-threading and concurrency to maximize performance across multi-core processors, utilizing libraries like <thread> to make tensor operations more parallelizable.