Overview
This block studies structure and representation in data. We begin with Principal Component Analysis and clustering, then transition to an introduction to neural networks as models that learn nonlinear transformations of the original predictors.
The 20 October lesson connects the feature-transformation ideas in PCA to learned hidden-layer representations in neural networks. The lesson introduces forward propagation, activation functions, loss functions, gradient descent, backpropagation, and Keras.
Topics Covered
- Principal Component Analysis
- Variance explained and dimensionality reduction
- K-Means clustering
- Hierarchical clustering
- DBSCAN
- Feature extraction and representation
- Neural-network structure and notation
- Hidden layers and learned nonlinear features
- Sigmoid and ReLU activation functions
- Forward propagation
- Cost functions, gradient descent, and backpropagation
- Cross-entropy loss
- Fitting multilayer neural networks with Keras
Kaggle Competition 2: Spoken Digit Data Competition
Due: 16 Oct 2026 NLT 2359
Competition 2 began in the Non-Parametric Models block with Support Vector Machines. The 13 October PCA lesson provides an additional perspective on feature representation and dimensionality, and the final competition submission is due on 20 October.
Key Topics, Lesson Objectives, Readings, and Competition Work
| Date | Key Topics | Lesson Objectives | Readings | Competition Work |
|---|---|---|---|---|
| 13 Oct 2026 | PCA & Unsupervised Structure |
Understand PCA as a linear transformation.
Understand principal directions of variation. Interpret proportion of variance explained. Understand K-Means and hierarchical clustering. Evaluate whether unsupervised structure or dimensionality reduction can provide useful representations for downstream modeling. |
ISLP
Chapter 12: Unsupervised Learning §12.1 The Challenge of Unsupervised Learning §12.2 Principal Components Analysis §12.2.1 What Are Principal Components? §12.2.2 Another Interpretation of Principal Components §12.2.3 Proportion of Variance Explained §12.4 Clustering Methods §12.4.1 K-Means Clustering §12.4.2 Hierarchical Clustering §12.4.3 Practical Issues in Clustering Géron Chapter 8: Dimensionality Reduction Principal Component Analysis Explained Variance Ratio Choosing the Right Number of Dimensions Chapter 9: Unsupervised Learning Techniques K-Means Limits of K-Means DBSCAN |
Continue Competition 2.
Spoken Digit Data Competition Due: 16 Oct 2026 NLT 2359 Open / Submit to the Spoken Digit Data Competition Test whether PCA or other representation changes improve a supervised model under the same validation procedure. Finalize the competition submission and repository before the 16 October deadline. |
| 20 Oct 2026 | Introduction to Neural Networks |
Lesson notes:
20 October Neural Network Lesson Notes
.
Understand the structure and notation of a multilayer neural network. Given inputs and the weights and biases of a multilayer neural network, calculate the predicted outputs for the network. Understand how the cost function and gradient descent are used to update weights and biases through backpropagation. Use functions in Python's Keras module to fit multilayer neural networks. |
Course Notes
MA289 AY2027 Neural Network Lesson ISLP Chapter 10: Deep Learning §10.1 Single Layer Neural Networks §10.2 Multilayer Neural Networks §10.7 Fitting a Neural Network §10.7.1 Backpropagation §10.7.2 Regularization and Stochastic Gradient Descent Géron Chapter 10: Introduction to Artificial Neural Networks with Keras The Perceptron The Multilayer Perceptron Backpropagation Activation Functions Regression and Classification MLPs Building Neural Networks with Keras Related prior notes: Gradient Descent Notes |
Complete the neural-network forward-pass and
backpropagation examples from the lesson notes.
Fit a small multilayer neural network in Keras and connect the implementation to the mathematical notation. Compare the neural-network representation with the linear transformations used in PCA and the feature-based classification workflow used in the Spoken Digit competition. |
Resources
- ISLP Chapter 10: Deep Learning
- ISLP Chapter 12: Unsupervised Learning
- Géron Chapter 8: Dimensionality Reduction
- Géron Chapter 9: Unsupervised Learning Techniques
- Géron Chapter 10: Introduction to Artificial Neural Networks with Keras
- 20 October Neural Network Lesson Notes
- Kaggle Competition Hub