CNNFS: Neural Networks from Scratch in C++

CNNFS (Neural Networks from Scratch in C++) is exactly as the name suggests: a neural network library written entirely from scratch in C++ and CUDA. It has no external dependencies, implementing everything from the neural networks themselves to the underlying linear algebra operations. The project began in plain C, using a custom metaprogramming layer to support generic types. After integrating CUDA, I transitioned to C++ to take advantage of features like templates and operator/function overloading.

This project was inspired by the excellent Neural Networks from Scratch book written by YouTuber sentdex.

I took on this challenge for several reasons. First, since my research heavily involves machine learning, I wanted to understand it at a deeper level, and I learn the best by building. It also served as a great introduction to CUDA and how to structure an application to work with either or both the CPU and GPU. In the future, I also plan to accelerate my CPU code paths with SIMD.

Design Highlights

• Unified interface: CPU and GPU code paths share the same API; dispatching is handled under the hood.
• Automatic device management: data objects remember which device they reside on, follow the appropriate code paths, and automatically transfer data when needed.