Machine Learning

Instructor

Jingli Gao
Associate Profesor
School of Software Engineering
Pingdingshan University

Course goal

This course is an introductory undergraduate course in machine learning. The class will briefly cover topics in linear regression, linear classification, fully connected neural networks, convolutional neural networks, recurrent Neural Networks, deep learning.

Prerequisites: You should understand basic probability and statistics, and college-level algebra and calculus. For the programming assignments, you should have some background in programming, and it would be helpful if you know Python and Pytorch.

1 Introduction

1.1 Computer vision

1.2 Deep learning

1.3 Natural language processing

2 Learning Foundations

2.1 Model evaluation

2.2 Model parameter selection

2.3 Supervised learning

2.4 Unsupervised learning

2.5 Gradient descent optimization

3 Linear Models

3.1 Linear models for regression

3.2 Linear models for classification

4 Fully Connected Neural Networks

4.1 Neural networks basics

4.2 Feed-forward network functions

4.3 Error Backpropagation

4.4 A simple example

5 Convolutional Neural Networks

5.1 Convolutional neural networks basics

5.2 Neural computations in a CNN

5.3 The equations of a forward pass through a CNN

5.4 The equations of backpropagation used to train CNNs

5.5 Classic network architectures

5.6 An example for classification

6 Recurrent Neural Networks

6.1 Recurrent neural networks basics

6.2 The equations of a forward pass through a RNN

6.3 The equations of backpropagation used to train RNNs

6.4 RNNs architectures

6.5 An example for natural language processing

• Assignments 50%
• Final project 50%

References

• [Lv21] 吕云翔, 刘卓然. PyTorch深度学习实战：微课视频版, 清华大学出版社, 2021.
• [xu22] 徐亦达. Machine Learning Class.
• [Gon18] Rafael C. Gonzalez, Richard E. Woods. Digital Image Processing, 4th Edition, Pearson, 2018.
• [Gut22] Michael U. Gutmann. Pen and Paper Exercises in Machine Learning, 2022.
• [Alp14] Alpaydin, E. Introduction to Machine Learning, 3Ed. The MIT Press, 2014.
• [Fag21] Fabio A. González. Machine Learning, 2021.
• [Mur12] Murphy, Kevin P. Machine learning: a probabilistic perspective. The MIT Press, 2012.
• [Barber2013] Barber, David, Bayesian Reasoning and Machine Learning, Cambridge University Press, 2013.
• [Bis06] Bishop, C. Pattern Recognition and Machine Learning. Springer-Verlag, 2006.
• [HTF09] Hastie, T. and Tibshirani, R. and Friedman. The elements of statistical learning: data mining, inference, and prediction, Springer, 2009
• [GBC2016] Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
• [zhang2021dive] Aston Zhang, Zack C. Lipton, Mu Li, Alex J. Smola. Dive into Deep Learning, 2021.
• [Mit97] Mitchell, T. M. 1997 Machine Learning. 1st. McGraw-Hill Higher Education.
• [DHS00] Duda, R. O., Hart, P. E., and Stork, D. G. 2000 Pattern Classification (2nd Edition). Wiley-Interscience.
• [SC04] Shawe-Taylor, J. and Cristianini, N. 2004 Kernel Methods for Pattern Analysis. Cambridge University Press.
• [SS02] Scholkopf, B. and Smola, A.J., 2002, Learning with kernels, MIT Press.
• [OCW-ML] 6.867 Machine Learning, Fall 2006, MIT OpenCourseWare.
• [STANFD-ML] Andrew Ng, CS229 Machine Learning, Stanford University

Additional resources

• SciPy: scientific, mathematical, and engineering package for Python
• scikit-learn: machine learning Scipy add-on
• Kaggle: datascience competition, many interesting data sets and different competitions with prizes.
• CMU Deep Learning Systems: Algorithms and Implementation: taught by Tianqi Chen and Zico Kolter.
• Stanford Deep Learning for Computer Vision Course: one of the best computer vision course taught by Fei-Fei Li.
• Coursera Machine Learning Course: one of the first (and still one of the best) machine learning MOOCs taught by Andrew Ng.
• Stanford Statistical Learning Course: an introductory course with focus in supervised learning and taught by Trevor Hastie and Rob Tibshirani.
• Introduction to Machine Learning: the course content comes from specific chapters of Pattern Recognition and Machine Learning by Chris Bishop.