links
- Research Groups
- Blogs
- Resources (Notes, Textbooks, Monographs, Classes, etc.)
- High-Dimensional Probability, Geometry, and Functional Analysis
- Learning Theory and Statistics
- Optimal Transport
- Interactive Learning
- Algorithmic Theoretical Machine Learning
- Algorithmic Game Theory
- Theoretical Computer Science
- Optimization
- Graphical Models and Bayesian Inference
- Natural Language Processing
- Neuroscience
- Deep Learning
- Linear Algebra and Matrix Analysis
- Analysis
- Combinatorics and Discrete Math
- Topology
- Algebra
- Functional Programming and Programming Languages
- Physics
- Problem Solving and Olympiad Math
- Writing
- Advice (on academia, productivity tips, how to’s, coding tutorials, etc…)
- Tools
- Other Compilations
- Code
- Fun Links
This page holds links to notes, class webpages, research groups, blogs, code, and other people’s webpages which I have found interesting and helpful. My brother is also involved in machine learning research, check out his website.
Research Groups
- Machine Learning @ Columbia
- Theory @ Columbia
- Dynamic Perception and Memory Lab @ Columbia
- Simons Foundations of Deep Learning
- Unsupervised Learning Research Group @ Princeton
- OptiML Research Group @ Princeton
- Theoretical ML @ Princeton
- NLP @ Princeton
- Theory @ Princeton
- Computational Memory Lab @ Princeton
Blogs
- Off-Convex
- Machine Learning Research Blog
- Arg Min
- I’m a Bandit
- Hunch.net
- Theory of Computing Blog Aggregator
- Windows on Theory
- My Biased Coin
- Theory Matters
- In Theory
- TCS math
- What’s New
- Holden Lee’s Blog
- Columbia Advanced Machine Learning Seminar
- GeomBlog
- TheoryDish
- Minimizing Regret
- Moody Rd
- Bits of DNA
- Jacob Andreas’ Blog
- NLPers
- Rooting for the Machines
- A Butterfly Valley
- Lullaby of Cape Cod
- Approximately Correct
- Math \(\cap\) Programming
- Matt.Might.net
- PG Bovine
- Jean Yang’s blog
- inFERENce
- Seita’s Place
- Power Overwhelming
- Cosma Shalizi’s Homepage
Resources (Notes, Textbooks, Monographs, Classes, etc.)
Note (Sept. 2021): There are some more recent courses/notes/books that I will add when I get a chance.
High-Dimensional Probability, Geometry, and Functional Analysis
- Probability in High Dimension (Ramon van Handel)
- High Dimensional Probability (Roman Vershynin)
- Introduction to the Non-Asymptotic Theory of Random Matrices (Roman Vershynin)
- High Dimensional Probability (Phillipe Rigollet)
- Concentration Inequalities: A Nonasymptotic Theory of Independence (Boucheron, Lugosi, Massart)
- An Introduction to Matrix Concentration Inequalities (Joel Tropp)
- Markov Chains and Mixing Times (Levin, Peres, Wilmer)
- Stochastic Processes on Graphs (Allan Sly)
- Modern Discrete Probability (Allan Sly)
- Modern Discrete Probability: An Essential Toolkit (Roch)
- A Mathematical Introduction to Compressive Sensing (Foucart and Rauhut)
- Elements of Information Theory (Cover and Thomas)
- Analysis of Boolean Functions (Ryan O’Donnell)
- Metric Embeddings and Geometric Inequalities (Assaf Naor)
- Metric Dimension Reduction: A Snapshot of the Ribe Program (Assaf Naor)
- Random Fields and Geometry (Adler and Taylor)
- Information Geometry and its Applications (Shun-ichi Amari)
- An Elementary Introduction to Modern Convex Geometry (Keith Ball)
- Lectures on Discrete Geometry (Matousek)
- Information, Physics, and Computation (Mézard and Montanari)
- The Cauchy-Schwarz Master Class (Steele)
Learning Theory and Statistics
- Theory of Classification (Boucheron, Bousquet, Lugosi)
- Statistical Learning Theory and Applications (Poggio, Rosasco, Rakhlin, Banburski)
- Understanding Machine Learning (Shai [Shalev-Shwartz, Ben-David])
- Foundations of Machine Learning (Mohri, Rostamizadeh, and Talwalkar)
- Theoretical Machine Learning (Rob Schapire)
- Machine Learning (Stanford CS 229)
- Statistical Learning Theory (Percy Liang)
- Patterns, Predictions, and Actions: A Story about Machine Learning (Moritz Hardt and Ben Recht)
- A Course in Machine Learning (Hal Daumé III)
- Statistical Learning Theory (Sham Kakade)
- Introduction to Computational Learning Theory (Rocco Servedio)
- Computational and Statistical Learning Theory (Nati Srebro)
- Boosting: Foundations and Algorithms (Robert Schapire and Yoav Freund)
- Learning with Kernels (Schölkopf and Smola)
- High Dimensional Stats (Phillipe Rigollet)
- The Elements of Statistical Learning (HTF)
- All of Statistics (Larry Wasserman)
- All of Nonparametric Statistics (Larry Wasserman)
- Introduction to Nonparametric Estimation (Tsybakov)
- Mathematical Statistics (Jun Shao)
- Asymptotic Statistics (van der Vaart)
- Statistical Learning with Sparsity (HTW)
- Theory of Statistics (John Duchi)
- Information Theory and Statistics (John Duchi)
- Theoretical Statistics I (Berkeley Stat 210a)
- Theoretical Statistics II (Berkeley Stat 210b)
- Sparsity, Structure, and Inference (Yuxin Chen)
- Inference, Estimation, and Information Processing (Andrea Montanari)
- Large Scale Inference: Empirical Bayes for Estimation, Testing, and Prediction (Efron)
- A Survey on Distribution Testing: Your Data is Big. But is it Blue? (Canonne)
Optimal Transport
- Computational Optimal Transport (Gabriel Peyré and Marco Cuturi)
- Optimal Transport, Old and New (Cédric Villani)
Interactive Learning
- Interactive Learning (Daniel Hsu)
- Online Methods in Machine Learning (Sasha Rakhlin)
- The Algorithmic Foundations of Differential Privacy (Cynthia Dwork and Aaron Roth)
- The Algorithmic Foundations of Adaptive Data Analysis (Aaron Roth)
- Reinforcement Learning: Theory and Algorithms (Alekh Agarwal, Nan Jiang, Sham Kakade, Wen Sun)
- Introduction to Multi-Arm Bandits (Alex Slivkins)
- Reinforcement Learning (Shipra Agrawal)
- Dynamic Programming and Reinforcement Learning (Daniel Russo)
- Learning and Optimization for Sequential Decision Making (Shipra Agrawal)
- Bandits and RL (Alex Slivkins and Alekh Agarwal)
- Multi-Arm Bandit Survey (Bubeck and Cesa-Bianchi)
- Bandit Algorithms (Lattimore and Szepesvari)
- From Bandits to Monte-Carlo Tree Search: The Optimistic Principle Applied to Optimization and Planning (Munos)
- Sébastien Bubeck’s YouTube Channel
- Reinforcement Learning (Sutton and Barto, 2018 edition)
- A Tour of Reinforcement Learning: The View from Continuous Control (Recht)
- Algorithms for Reinforcement Learning (Csaba Szepesvári)
- Deep Reinforcement Learning (Sergey Levine)
Algorithmic Theoretical Machine Learning
- Foundations of Data Science (Blum, Hopcroft, Kannan)
- Algorithmic Aspects of ML (Ankur Moitra)
- Provable Algorithms for Machine Learning Problems (Rong Ge)
- Simons Spring 2017 Program on Theoretical Foundations of Machine Learning
- Simons Fall 2018 Program on Foundations of Data Science
- Non-Convex Optimization for Machine Learning: Design, Analysis, and Understanding (Tengyu Ma)
- Algorithmic Aspects of ML (Rong Ge)
- New Directions in Theoretical ML (Sanjeev Arora)
- Ten Lectures and Forty-Two Open Problems in the Mathematics of Data Science (Afonso Bandeira)
- Theoretical Foundations of Machine Learning (Nika Haghtalab)
- Advanced Machine Learning (Daniel Hsu)
- Linear Dimensionality Reduction: Survey, Insights, Generalizations (Cunningham and Ghahramani)
- Unsupervised Learning (Nakul Verma)
- Unsupervised Learning: Theory and Practice (Elad Hazan)
- Community Detection Survey (Emmanuel Abbe)
- Learning-Augmented Algorithms (Piotr Indyk, Costis Daskalakis)
Algorithmic Game Theory
- Algorithmic Game Theory (Tim Roughgarden)
- Algorithmic Game Theory (Eva Tardos)
- Incentives in Computer Science (Tim Roughgarden)
- Foundations of Blockchains (Tim Roughgarden)
- Topics in Algorithmic Game Theory: Algorithmic Game Theory and Data Science (Costis Daskalakis, Vasilis Syrgkanis)
- Games, Decisions, and Computation (Costis Daskalakis)
- Twenty Lectures on Algorithmic Game Theory (Tim Roughgarden)
- Game Theory, Alive (Anna Karlin and Yuval Peres)
- Algorithmic Game Theory (NRTV)
Theoretical Computer Science
- Introduction to Algorithms (CLRS)
- Algorithm Design (Kleinberg and Tardos)
- Advanced Algorithms (Sanjeev Arora)
- Algorithms through Geometric Lens (Alex Andoni and Ilya Razenshteyn)
- Algorithms through Geometric Lens (2018) (Ilya Razenshteyn)
- Advanced Algorithms (Ankur Moitra)
- Randomized Algorithms (Tim Roughgarden)
- Beyond Worst Case Analysis (Tim Roughgarden)
- The Modern Algorithmic Toolbox (Greg Valiant)
- Communication Complexity for Algorithm Designers (Tim Roughgarden)
- The Structure of Information Networks (Jon Kleinberg)
- A Second Course in Algorithms (Tim Roughgarden)
- Simons Institute Video Lectures
- Mathematics and Computation (Avi Widgerson)
- Spectral Graph Theory (Dan Spielman)
- Spectral Graph Theory and its Applications (Dan Spielman)
- Laplacian Page (Dan Spielman)
- Pseudorandomness (Salil Vadhan)
- A Theorist’s Toolkit (Jon Kelner)
- A Theorist’s Toolkit (Ryan O’Donnell)
- Efficient Algorithms and Intractable Problems (Berkeley CS 170)
- Algorithms and Complexity (Harvard CS 125)
- Lx = b (Nisheeth Vishnoi)
- Lecture Notes on Metric Embeddings (Jirí Matousek)
- P ?= NP (Scott Aaronson)
- Entropy Optimality (James Lee)
- Sketching as a Tool for Numerical Linear Algebra (David Woodruff)
- Sketching Algorithms for Big Data (Jelani Nelson)
- Algorithms for Big Data (David Woodruff)
- The Discrepancy Method: Randomness and Complexity (Bernard Chazelle)
- Geometric Discrepancy (Matousek)
- Randomized Linear Algebra (Michael Mahoney)
- Randomized Algorithms in Numerical Linear Algebra (Ravi Kannan and Santosh Vempala)
- Algorithmic Convex Geometry (Santosh Vempala)
- Approximation Algorithms (David Williamson)
- The Design of Approximation Algorithms (Williamson and Shmoys)
- Approximate Algorithms and Semidefinite Programming (Gärtner and Matousek)
- Integer Programming and Combinatorial Optimization (Dorit Hochbaum)
- Hierarchies of Integer Programming Relaxations (Moses Charikar)
- Discrete Optimization (Gärtner)
- Geometry: Combinatorics and Algorithms (GHW)
- Geometric Random Walks: A Survey (Santosh Vempala)
- Randomness and Computation (Alistair Sinclair)
- Randomized Algorithms and Probabilistic Analysis (James Lee)
- Randomized Algorithms (David Karger)
- Combinatorial Optimization: Exact and Approximate Algorithms (Luca Trevisan)
- Notes on Randomized Algorithms (James Aspnes)
- Applied Algorithms (James Lee)
- Spectral Algorithms (Ravi Kannan and Santosh Vempala)
- Advanced Approximation Algorithms (Anupam Gupta and Ryan O’Donnell)
- Approximate Nearest Neighbor Search in High Dimensions (Andoni, Indyk, Razenshteyn)
- Seminar on Learning Algorithms (Embeddings) (Sanjoy Dasgupta)
- Lecture Notes on Graph Partitioning, Expanders, and Spectral Methods (Luca Trevisan)
- Expander Graphs and their Applications (Hoory, Linial, Widgerson)
- Proofs, Beliefs and Algorithms through the lens of Sum-of-Squares (Boaz Barak and David Steurer)
- Sum-of-Squares Proofs and the Quest Toward Optimal Algorithms (Boaz Barak and David Steurer)
- The Sum-of-Squares Algorithmic Paradigm in Statistics (Tselil Schramm)
- On the Unique Games Conjecture (Subhash Khot)
- Physics and Computation (Boaz Barak and Tselil Schramm)
- Information Theory in Theoretical Computer Science (Omri Weinstein)
- Information Theory in Computer Science (Madhu Sudan)
- Essential Coding Theory (Madhu Sudan)
- Theory of Computing Online Journal
Optimization
- Introductory Lectures on Convex Optimization (Yurii Nesterov)
- Convex Optimization: Algorithms and Complexity (Sébastian Bubeck)
- Linear and Semidefinite Programming (Ryan O’Donnell)
- Convex Optimization (Boyd and Vandenberghe)
- Simons Fall 2017 Program on Bridging Continuous and Discrete Optimization
- Convex Optimization (Laurent El Ghaoui)
- Introduction to Online Convex Optimization (Elad Hazan)
- Interplay between Convex Optimization and Geometry (Yin Tat Lee)
- Convex Optimization and Approximation (Moritz Hardt)
- Lectures on Modern Convex Optimization (Tal and Nemirovski)
- Mathematical Programming I (Damek Davis)
- Semidefinite Optimization (Laurent and Vallentin)
- Convex Optimization II (John Duchi)
- Large Scale Optimization for Data Science (Yuxin Chen)
- Competitive Analysis via Convex Optimization (James Lee)
- Proximal Algorithms (Parikh and Boyd)
- Convex and Conic Optimization (Amir Ali Ahmadi)
- ADMM Survey (BPCPE)
- Numerical Optimization (Nocedal and Wright)
Graphical Models and Bayesian Inference
- Foundations of Graphical Models 2018 (Dave Blei)
- Foundations of Graphical Models 2016 (Dave Blei)
- Bayesian Models for Machine Learning (John Paisley)
- Probabilistic Graphical Models (Koller and Friedman)
- Elements of Causal Inference (Peters, Janzing, and Schölkopf) (note, open access!)
- Graphical Models, Exponential Families, and Variational Inference (Martin Wainwright and Michael Jordan)
- Lecture Notes on Bayesian Nonparametrics (Peter Orbanz)
- Sampling Algorithms (Peter Orbanz)
- Advanced Machine Learning (Peter Orbanz and John Cunningham)
- Machine Learning: A Probabilistic Perspective (Kevin Murphy)
- Gaussian Processes for Machine Learning (Rasmussen and Williams)
- Bayesian Nonparametrics and the Probabilistic Approach to Modeling (Zoubin Ghahramani)
- An Introduction to Hidden Markov Models and Bayesian Networks (Zoubin Ghahramani)
- Fundamentals of Exponential Families (Brown)
- A Unifying Review of Gaussian Linear Models (Roweis and Ghahramani)
- Bayesian Nonparametric Models (Orbanz and Teh)
- Dirichlet Process (Yee Whye Teh)
- Hierarchical Bayesian Nonparametric Models with Applications (Teh and Jordan)
Natural Language Processing
- Natural Language Processing (Michael Collins)
- A Primer on Neural Network Models for Natural Language Processing (Yoav Goldberg)
- Deep Learning for Natural Language Processing (Richard Socher)
- Machine Learning for Natural Language (Sasha Rush and Yoon Kim)
- Neural Networks for NLP (Graham Neubig)
- Machine Translation and Sequence-to-Sequence Models (Graham Neubig)
- Natural Language Processing (Sida Wang)
- Natural Language Processing (Jacob Eisenstein)
Neuroscience
- Computation and the Brain (Santosh Vempala)
- Simons Spring 2018 Program on The Brain and Computation
- Computational Cognitive Science (Tenenbaum)
- Probabilistic Models of Cognition (Tenenbaum)
Deep Learning
- Neural Networks: Theory and Applications (Sebastian Seung)
- Theoretical Foundations of Deep Learning (Sanjeev Arora)
- Science of Deep Learning: Bridging Theory and Practice (Costis Daskalakis and Aleksander Madry)
- Theories of Deep Learning (David Donoho)
- Convolutional Networks for Visual Recognition (Fei-Fei Li)
- Deep Learning (Goodfellow, Bengio, Courville)
- Mathematics of Deep Learning (Joan Bruna)
- Inference and Representation (Joan Bruna)
- Topics in Deep Learning (Joan Bruna)
- Mathematical Aspects of Deep Learning (Elchanan Mossel)
Linear Algebra and Matrix Analysis
- Linear Algebra (Michael Damron and Tasho Kaletha)
- Introduction to Linear Algebra (Strang)
- Thirty-three Miniatures: Mathematical and Algorithmic Applications of Linear Algebra (Matousek)
- Linear Algebra Done Right (Axler)
- Advanced Linear Algebra (Roman)
- Matrix Computations (Golub and Van Loan)
- Matrix Analysis (Horn and Johnson)
- Topics in Matrix Analysis (Horn and Johnson)
Analysis
- Analysis in a Single Variable (Michael Damron and Mark McConnell)
- An Introduction to Analysis (Robert C. Gunning)
- Principles of Mathematical Analysis (Rudin)
- An Introduction to Measure Theory (Terry Tao)
- Probability Theory (Allan Sly)
- Probability: Theory and Examples (Durrett)
- Probability with Martingales (David Williams)
- Probability and Stochastics (Erhan Çinlar)
- Stochastic Calculus, Filtering, and Control (Ramon van Handel)
- Fourier Analysis (Stein and Shakarchi)
- Complex Analysis (Stein and Shakarchi)
- Real Analysis (Stein and Shakarchi)
- Functional Analysis (Stein and Shakarchi)
- Manifolds and Differential Forms (Sjamaar)
Combinatorics and Discrete Math
- Combinatorial Mathematics (van Zwam)
- Generatingfunctionology (Wilf)
- Introduction to Graph Theory (West)
- Topics in Combinatorics: The Probabilistic Method (Noga Alon)
- The Probabilistic Method (Alon and Spencer)
- Modern Graph Theory (Bela Bollobas)
- Introduction to Random Graphs (Frieze and Karonski)
Topology
- Topology 2nd Ed. (Munkres)
- Elements of Algebraic Topology (Munkres)
- Algebraic Topology (Hatcher)
- Using the Borsuk-Ulam Theorem (Matousek)
Algebra
- Group Theory (Mark McConnell)
- Linear Representations of Finite Groups (Serre)
- Galois Theory (Xiaoheng Jerry Wang)
- Basic Algebraic Geometry (Arapura)
- Algebraic Geometry (Gathmann)
- Basic Category Theory (Leinster)
Functional Programming and Programming Languages
- Types and Programming Languages (Pierce)
- Functional Programming (David Walker)
- Programming Languages (David Walker)
- Learn You a Haskell
Physics
- The Feynman Lectures on Physics
- How to become a good theoretical physicist (Hooft)
- The theoretical minimum (Susskind)
- Perimeter Institute Videos
- Concepts in Theoretical Physics: Mathematical Tripos (Daniel Baumann)
- Heidelberg Theoretical Physics Lecture Notes (Hefft)
- Lectures on Theoretical Physics (Tong)
Problem Solving and Olympiad Math
- Problem-Solving Strategies (Engel)
- How To Solve It (Polya)
- Putnam Seminar (Po-Shen Loh)
- Olympiad Notes (Po-Shen Loh)
- Contest Math (Evan Chen)
- Contest Math (and more) (Holden Lee)
Writing
- Mathematical Writing (Knuth, Larabee, Roberts)
- How to Write a Great Research Paper (Simon Peyton Jones)
- Write the Paper First (Jason Eisner)
- 3 Shell Scripts to Improve Your Writing (Matt Might)
- Style: Lessons in Clarity and Grace (Williams and Bizup)
- The Science of Scientific Writing (Gopen and Swan)
- Creative Writing (Brandon Sanderson)
- Manga in Theory and Practice (Hirohiko Araki)
Advice (on academia, productivity tips, how to’s, coding tutorials, etc…)
- Kevin Gimpel’s Advice to Ph.D. Students
- How to get a Ph.D. in Mathematics in a Timely Fashion (Sara C. Billey)
- Manuel Blum’s Advice to a Beginning Graduate Student
- The Bandwagon (Claude Shannon)
- Terry Tao’s Career Advice
- Troubling Trends in Machine Learning Scholarship (Lipton and Steinhardt)
- You and Your Research (Hamming)
- How to be a Mathematician (or Theoretical Computer Scientist) (O’Donnell)
- Theory Life Hacks (Boaz Barak)
- Advice on submitting to Nature, Science, and PNAS (from computer scientists)
- CAREER example proposals
- How to Be a Successful Ph.D. Student (Dredze and Wallach)
- How to Read a Technical Paper (Jason Eisner)
- How to Organize Your Files (Jason Eisner)
- How to Find Research Problems (Jason Eisner)
- The Ph.D. Grind (Phillip Guo)
- Stress in Research (Charles Sutton)
- A Ph.D. is NOT Enough! (Feibelman)
- How to Give Good Academic Talks (Matt Might)
- How to Give a Great Research Talk (Simon Peyton Jones)
- How to get Tenure (Matt Might)
- Parsing BibTeX (Matt Might)
- Successful PhD Students (Matt Might)
- How to Email (Matt Might)
- Grad Student Resolutions (Matt Might)
- Set Up Homebrew on Mac (Belyamini)
- Set Up Python
- Bash by Example (Matt Might)
Tools
- Handbook of Mathematical Functions (Abramowitz and Stegun)
- Matrix Cookbook (Petersen and Pedersen)
- Online Encyclopedia of Integer Sequences (OEIS)
- Theoretical CS StackExchange
- Wolfram Alpha
- DeTeXify
- MathJax tricks
- CombinePDF
- Bash Cheatsheet
- Vim Cheatsheet
Other Compilations
- Tim Roughgarden’s lecture notes
- Tim Roughgarden’s lectures
- Holden Lee’s notes (there are a lot!!!)
- Arun Debray’s notes (there are also a lot!!)
- Karl Stratos’ notes
- Evan Chen’s notes
- An Infinitely Large Napkin (Evan Chen)
- Irene Chen’s notes on papers
- Daniel Takeshi’s notes on papers
- Billy Fang’s notes
- Cosma Shalizi’s notebooks
Code
- Princeton NLP Codebase
- Princeton ML Codebase
- Allen NLP Codebase
- Harvard NLP Codebase
- BrainIAK
- CVXOPT
- OpenAI Gym
- Tensorflow
- Colaboratory
- PyTorch
- HuggingFace
- Jax
- scikit-learn