Neural Network Learning Resources
Mathematics Fundamentals
Linear Algebra
Books
- Linear Algebra and Its Applications - Gilbert Strang
- Linear Algebra Done Right - Sheldon Axler
- Matrix Analysis and Applied Linear Algebra - Carl D. Meyer
Video Courses
Research Papers
- "Fast Linear Algebra is Stable" - arXiv:1405.7903
- "Randomized Numerical Linear Algebra: Foundations & Algorithms" - arXiv:2002.01387
Calculus
Books
- Calculus: Early Transcendentals - James Stewart
- Calculus Made Easy - Silvanus P. Thompson
- Advanced Calculus: A Differential Forms Approach - Harold M. Edwards
Video Courses
Research Papers
- "The Calculus of Deep Learning" - arXiv:2102.01454
Statistics & Probability
Books
- Introduction to Probability - Dimitri P. Bertsekas
- All of Statistics - Larry Wasserman
- Probability Theory: The Logic of Science - E. T. Jaynes
Video Courses
Programming Fundamentals
C Programming
Books
- Modern C - Jens Gustedt
- Expert C Programming: Deep C Secrets - Peter van der Linden
- C Programming: A Modern Approach - K.N. King
- 21st Century C - Ben Klemens
Online Resources
Neural Network Implementation
Foundational Resources
Books
- Neural Networks and Deep Learning - Michael Nielsen
- Deep Learning - Ian Goodfellow, Yoshua Bengio, Aaron Courville
- Pattern Recognition and Machine Learning - Christopher Bishop
Research Papers
- "Training Deep Nets with Sublinear Memory Cost" - arXiv:1604.06174
- "Deep Learning Hardware: Past, Present, and Future" - arXiv:1907.09423
- "Efficient BackProp" - Research Paper Link
Advanced Implementation
Online Courses
Code Repositories
Research Papers
- "Random Search for Hyper-Parameter Optimization" - arXiv:1803.09820
- "A Disciplined Approach to Neural Network Hyper-Parameters" - arXiv:1803.09820
- "An Analysis of Deep Neural Network Models for Practical Applications" - arXiv:1605.07678
Performance Optimization
System Level Optimization
Books
- High Performance Computing
- Optimizing C++
- Computer Systems: A Programmer's Perspective - Bryant and O'Hallaron
Research Papers
- "Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference" - arXiv:1712.05877
- "Mixed Precision Training" - arXiv:1710.03740
Architecture Design
Books
Research Papers
- "A Hardware Architecture for Deep Learning" - arXiv:1908.07748
- "System Design for Large Scale Machine Learning" - Paper Link