DS703: Introduction to Statistical Inference and Machine Learning
Core, IIT Bombay, C-MInDS, 2026
Course Title: Introduction to Statistical Inference and Machine Learning
Instructors: Parthe Pandit, Arjun Bhagoji
TA: Sreeranjini T M, Apurva Dhingra
Time: Monday, Tuesday, Thursday (11.35-12.30am)
Room: LT206
Office Hours: Arjun (2.30-3.30pm on Tuesdays in CC120/online), Parthe (4-5pm on Mondays in KR-320)
Course Description
This course is intended to serve as a foundational graduate course in statistics and machine learning for all incoming graduate students in C-MInDS. It will provide them with one part of the unified, necessary background to take further electives. The course will be rigorous, starting with a treatment of statistical inference as a precursor to modern machine learning. Key aspects of theoretical machine learning such as generalisation bounds will be covered.
Intended Audience: The intended audience for this class is graduate students working in machine learning and data science, who are interested in doing research in this area. However, interested final-year undergraduates are welcome to attend as well.
Pre-requisites: Mathematical maturity will be assumed as will the basics of algorithms, probability, linear algebra, and optimisation.
Course Schedule
| Week | Topic | References | Notes | Comments |
|---|---|---|---|---|
| 1 | Measure-theoretic probability | Chapter 1, Durrett | ||
| 2 | Parameter estimation, exponential families | Chapters 2 and 8, Keener; Chapters 6 and 9, Wasserman | ||
| 3 | MLE, Generalised Linear Models | Chapter 9 and 14, Keener; GLMs | ||
| 4 | Linear regression | Chapter 3, Bach (ignore all discussions of excess risk) | ||
| 5 | Kernels | 2.1, 2.2.1, 2.2.2, 2.2.6, 2.2.7, 2.3 from Schölkopf & Smola; 7.1, 7.2, 7.3, 7.4 from Bach | ||
| 6 | Sparse regression | 6.1-6.4 from Beck; 10.1-10.2 from Beck; 8.1, 8.2, 8.3.1 from Bach | ||
| 7 | Confidence Intervals | 6.3 from Wasserman; 9.4-9.5 from Keener | ||
| 8 | Mid-sem week | |||
| 9 | Classification and risk notions | Chapter 2, Bach; Chapter 2, 3, and 4, Shalev-Shwartz & Ben-David; 3.1-3.6, Liang’s notes | ||
| 10 | Support vector machines | Chapter 5, Mohri et al.; Chapter 7, Schölkopf & Smola; Crammer & Singer, 2001 | ||
| 11 | Surrogate losses and logistic regression | 4.7 from Mohri et al., 4.1 from Bach | ||
| 12 | Optimisation for learning | 1.1, 1.2, 1.3, 3.1, 3.2 from Bubeck; Chapter 5, Bach | Bach uses very different proof techniques, mostly intended as advanced reading | |
| 13 | Generalisation bounds, NFL theorems | Chapter 4, Bach | ||
| 14 | Unsupervised learning and dimensionality reduction | |||
| 15 | Advanced Topics: Nonparametric methods, ensemble methods |
Resources
Practice problems
Running list of practice questions
Supplementary Books
Probability
Statistics
- Keener, Theoretical Statistics: Topics for a Core Course
- Wasserman, All of Statistics
- Casella & Berger, Statistical Inference
Optimisation
- Boyd, Convex Optimization: Algorithms and Complexity
- Bubeck, Convex Optimization
- Beck, First-Order Methods in Optimization
Machine Learning
- Mohri et al., Foundations of Machine Learning
- Shalev-Shwartz & Ben-David, Understanding Machine Learning: From Theory to Algorithms
- Bach, Learning Theory from First Principles
- Schölkopf & Smola, Learning with Kernels
General
Helpful Courses
Other references mentioned in class
Grading
Quizzes: 40% (4, each 10%)
Mid-sem: 15%
End-sem: 25%
Class participation+attendance: 20%
All exams allow handwritten cheatsheets of appropriate length. No other aids are permitted.
Attendance Policy
2 unexplained absences are allowed. 3 additional pre-approved absences are allowed.
Accommodations
Students with disabilities and health issues should approach the instructors at any point during the semester to discuss accommodations. The course aim is to learn together and legitimate bottlenecks will be resolved collaboratively.