Optimization Algorithms for Machine Learning
Department of Mathematics · Archive, spring 2019
The lectures used the Julia programming language (version 1.1 or later) and the Plots.jl library. All the material is also available on Dropbox.
Lectures
| Date | Topic | Scripts |
|---|---|---|
| 05/03/2019 | Introduction (PDF) Reading: Ch. 1 of [1]; Ch. 1–2 of [2] |
basic_regression.jl |
| 14/03/2019 | Clustering (PDF) Reading: Ch. 9 of [2] |
exercise1.jl, exercise2.jl Solutions: ex1_solution.jl, ex2_solution.jl |
| 19/03/2019 | Multilayer Neural Networks (PDF) Reading: Ch. 11 of [1] |
homer.jl, T2-ex.jl |
| 26/03/2019 | Classification (PDF) | homer_sol.jl, T3.jl, mlp_example.jl, CNN.jl |
| 02/04/2019 | Digit Classification Challenge (PDF) Leaderboard |
challenge.jl, xor2.jl |
Other scripts: linear_regression.jl, xor.jl.
Textbooks
- T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, 2009 (free PDF).
- C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006 (free PDF).
Readings and links
Machine learning
- TEDx talk by Marco Lübbecke on optimization (YouTube)
- Evolution of randomness in optimization methods for supervised machine learning
- Adversarial attacks on medical machine learning (Science, 2019)
- The softmax function and its derivative
- Runge phenomena: polynomial interpolation at evenly spaced nodes converges if and only if the function is analytic inside a football-shaped region.
Julia and Flux
Useful links
Current course: Operations Research / Optimization Models and Algorithms for Data Science.