Book contents and learning paths
This is a general machine learning textbook with a substantial deep learning sequence. It develops the ability to understand a method, derive its main equations, implement it, evaluate it, and choose when to use it. The scope includes supervised learning, unsupervised learning, probabilistic models, neural networks, and the practical work that makes experiments trustworthy.
Deep learning belongs in the long-term book, but the classical machine learning route is useful on its own. Examples will span tabular data, text, images, recommendations, and time series. Gait, rehabilitation, sensing, and robotics may supply optional case studies; they do not determine the curriculum. Random Forest is one ensemble method among the model families covered.
How to read this contents page
The outline contains 84 planned chapter positions in 13 parts, plus front matter, appendices, and references. It is a long-term authoring map, not a claim that all chapters have been written or a requirement to read everything in one course.
- Available: Chapters 1–14 have source chapters, linked below. The current revision and troubleshooting chapter covers that material only.
- Planned: Chapters 15–83 and the full-book expansion of Chapter 84 remain to be authored. Their titles are plain text rather than links to nonexistent pages.
- Extension: A chapter marked “extension” provides optional depth beyond the main learning routes. It still receives a substantive treatment when authored.
- Numbering: Numbers below describe the eventual sequence. The book navigation numbers only the chapters actually included in the current release; the current revision chapter therefore has an earlier number than its eventual position 84.
The order is pedagogical, not historical, and does not rank algorithms by performance. Prerequisites are local: a reader can begin neural networks after the relevant foundations without first studying every classical method.
Front matter consists of the Preface, this contents page, reading paths, notation conventions, and instructions for running examples.
Part I — Orientation and computing
Start here with elementary arithmetic. Programming and data-handling ideas are introduced before they are used in complete modeling examples.
| No. | Chapter | Main coverage |
|---|---|---|
| 1 | Introduction to machine learning | Learning problems; observations, features, and targets; supervised, unsupervised, and reinforcement learning; generalization and intended use. |
| 2 | Python for machine learning | Variables, control flow, functions, files, exceptions, debugging, and readable numerical programs. |
| 3 | NumPy and numerical computing | Arrays, shapes, indexing, numerical precision, matrix operations, explicit loops, and vectorization. |
| 4 | Working with data and plots | Tables, missing values, joins, descriptive summaries, visualizations, units, and row alignment. |
Part II — Essential mathematics
These chapters build the mathematical language used throughout the book. Readers may revisit them as needed. Computational examples use Parts I and II together.
| No. | Chapter | Main coverage |
|---|---|---|
| 5 | Algebra, functions, and logarithms | Equations, functions, powers, exponentials, logarithms, and summation notation. |
| 6 | Vectors and matrices | Dot products, norms, projections, matrix multiplication, rank, linear systems, and dimensions. |
| 7 | Probability | Random variables, distributions, conditional probability, Bayes’ rule, expectation, variance, and an introduction to entropy. |
| 8 | Statistical estimation | Sampling, estimators, likelihood, covariance, correlation, confidence intervals, and bootstrap uncertainty. |
| 9 | Calculus for learning | Derivatives, partial derivatives, gradients, the chain rule, curvature, and numerical checks. |
| 10 | Numerical optimization | Objectives, gradient descent, learning rates, convexity, constraints, stopping rules, and numerical stability. |
Part III — The machine learning workflow
Study these chapters before conducting model experiments. They establish the evaluation and preprocessing rules reused by every model family.
| No. | Chapter | Main coverage |
|---|---|---|
| 11 | Problem framing and exploratory data analysis | Intended use, observation units, target definitions, provenance, data quality, sampling, and exploration. |
| 12 | Training, evaluation, and data leakage | Losses and metrics, baselines, train/validation/test roles, grouped and temporal splits, cross-validation, overfitting, and bias–variance intuition. |
| 13 | Data preparation and pipelines | Imputation, scaling, encoding, transformations, training-only fitting, and reproducible preprocessing pipelines. |
Part IV — Core supervised learning
The workflow and essential mathematics are prerequisites. Begin with regression and regularization, then compare several approaches to classification. Calculus is needed for gradient-based derivations; probability supports generative classifiers. Constrained optimization is developed further within the SVM treatment.
| No. | Chapter | Main coverage |
|---|---|---|
| 14 | Linear regression | Least squares, normal equations and their conditions, numerical solvers, gradient descent, polynomial features, residuals, and diagnostics. |
| 15 | Ridge regression | L2 regularization, collinearity, shrinkage, scaling, penalty selection, and the bias–variance tradeoff. |
| 16 | Lasso and elastic net | L1 and combined penalties, sparse coefficients, coordinate descent, correlated predictors, and feature-selection stability. |
| 17 | Logistic regression | Log-odds, sigmoid probabilities, likelihood, cross-entropy, gradients, regularization, multiclass models, and decision thresholds. |
| 18 | k-nearest neighbors | Distance-based regression and classification, scaling, neighborhood size, dimensionality, and prediction costs. |
| 19 | Naive Bayes | Generative classification, conditional independence, smoothing, and Gaussian, multinomial, and Bernoulli variants. |
| 20 | Linear and quadratic discriminant analysis | Gaussian class models, shared versus class-specific covariance, decision boundaries, covariance estimation, and the discriminant-projection view. |
| 21 | Support vector machines | Margins, hinge loss, soft-margin constraints, support vectors, primal and dual viewpoints, and linear implementations. |
| 22 | Kernels and nonlinear models | Feature maps, positive-semidefinite kernels, the kernel trick, nonlinear SVMs, support vector regression, and kernel ridge regression. |
Part V — Decision trees and ensembles
Use the workflow from Part III and the regression/classification language from Part IV. Tree splitting does not require calculus. Boosting derivations additionally use losses and optimization. Tree → bagging → forest is one conceptual sequence within the book; boosting has a different training mechanism.
| No. | Chapter | Main coverage |
|---|---|---|
| 23 | Decision trees | Classification and regression splits, impurity, recursive construction, stopping, pruning, interpretability, and instability. |
| 24 | Bagging | Bootstrap samples, averaging and voting, variance and learner correlation, out-of-bag estimates, and their limits. |
| 25 | Random Forest | Feature subsampling, tree diversity, classification and regression, tuning, computational costs, and importance caveats. |
| 26 | AdaBoost | Weighted training examples, weak learners, stagewise updates, margins, and sensitivity to label noise. |
| 27 | Gradient boosting | Additive models, residuals and negative gradients, learning rates, tree complexity, early stopping, and regularization. |
| 28 | Gradient boosting in practice: XGBoost, LightGBM, and CatBoost — extension | Separate treatments of each framework’s objectives, tree construction, categorical-data handling, controls, and resource tradeoffs; aligned comparisons. |
| 29 | Voting, stacking, and ensemble comparison | Hard and soft voting, out-of-fold predictions, meta-models, matched tuning budgets, and measured accuracy/runtime tradeoffs. |
Part VI — Reliable model development
Return to these topics after fitting a few supervised models. Each example requires the particular model being studied, rather than an arbitrary prerequisite such as Random Forest. Basic evaluation is already required by Part III; this part deepens it.
| No. | Chapter | Main coverage |
|---|---|---|
| 30 | Feature engineering and selection | Domain features, transformations, filter/wrapper/embedded methods, leakage prevention, and selection stability. |
| 31 | Hyperparameter optimization | Search spaces, grid and random search, resource budgets, early stopping, and an introduction to Bayesian optimization. |
| 32 | Model evaluation and comparison in depth | Nested validation, repeated/grouped/temporal evaluation, learning curves, uncertainty, dependent observations, and multiple comparisons. |
| 33 | Imbalanced learning, calibration, and decisions | Class weights, resampling within training folds, precision–recall and ROC analysis, probability calibration, costs, and threshold selection. |
| 34 | Predictive uncertainty and conformal prediction — extension | Prediction intervals and sets, uncertainty sources, coverage, exchangeability, and limitations under distribution shift. |
| 35 | Interpreting and explaining models | Coefficients, permutation importance, partial dependence, accumulated local effects, local explanations, and correlated-feature pitfalls. |
| 36 | Responsible and robust machine learning | Subgroup evaluation, fairness criteria, distribution shift, stress testing, privacy, data consent, documentation, and limits of intended use. |
Part VII — Unsupervised learning and representations
Use linear algebra, probability, optimization, and preprocessing as needed. The opening chapter supplies eigenvalue and singular-value decompositions before PCA. Clustering and anomaly-detection evaluation are taught explicitly because their goals differ from supervised prediction.
| No. | Chapter | Main coverage |
|---|---|---|
| 37 | Eigenvectors, singular values, and low-rank approximation | Covariance geometry, eigenvalue decomposition, SVD, numerical rank, and reconstruction. |
| 38 | Principal component analysis | Centering, variance, projections, reconstruction, component selection, and the limits of variance as a learning criterion. |
| 39 | k-means clustering | Centroids, alternating updates, initialization, inertia, cluster validity, stability, and geometric assumptions. |
| 40 | Hierarchical clustering | Linkage choices, dendrograms, distance matrices, cutting a hierarchy, and sensitivity to scaling. |
| 41 | Density-based clustering: DBSCAN and HDBSCAN | Core/border/noise points, density connectivity, parameter sensitivity, and a separately developed optional density-hierarchy extension. |
| 42 | Gaussian mixtures and expectation-maximization | Latent membership, responsibilities, likelihood, EM updates, covariance choices, degeneracy, and local solutions. |
| 43 | Nonnegative matrix factorization — extension | Additive factors, reconstruction objectives, optimization, identifiability, and interpretability limits. |
| 44 | Independent component analysis — extension | Whitening, independence, non-Gaussianity, identifiability, and synthetic signal separation. |
| 45 | Embedding visualization: t-SNE and UMAP — extension | Separate neighborhood-based constructions, optimization, hyperparameters, stability, and misleading visual patterns. |
| 46 | Anomaly and novelty detection | Isolation Forest, Local Outlier Factor, an optional one-class SVM treatment, score thresholds, and evaluation with limited labels. |
| 47 | Semi-supervised and active learning — extension | Self-training, label propagation, unlabeled-data assumptions, querying labels, and evaluation under limited annotation budgets. |
Part VIII — Further statistical and probabilistic learning
This is an extension branch, not a prerequisite block for all neural-network chapters. GLMs build on regression and likelihood; Bayesian models build on probability and estimation; Gaussian processes additionally use kernels. Causal reasoning requires explicit assumptions beyond predictive accuracy.
| No. | Chapter | Main coverage |
|---|---|---|
| 48 | Generalized linear models — extension | Exponential-family motivation, link functions, count responses, dispersion, and selecting an appropriate response model. |
| 49 | Splines and generalized additive models — extension | Basis functions, smoothness penalties, additive effects, interactions, and interpretation. |
| 50 | Bayesian inference and regression — extension | Priors, likelihoods, posteriors, posterior prediction, Bayesian linear regression, and introductory computational approximations. |
| 51 | Gaussian processes — extension | Covariance functions, posterior predictions, hyperparameters, uncertainty assumptions, and scaling costs. |
| 52 | Causal reasoning for machine learning — extension | Confounding, causal graphs, interventions, selection bias, identification, and the distinction between prediction and causal claims. |
Part IX — Deep learning foundations
The direct entry route is Python/NumPy, linear algebra, probability, calculus, optimization, the modeling workflow, linear regression, and logistic regression. Review regularization from Chapters 15–16 as needed. Trees, clustering, Gaussian processes, and the rest of Part VIII are not prerequisites for this part.
| No. | Chapter | Main coverage |
|---|---|---|
| 53 | Perceptrons and multilayer neural networks | Neurons, activations, layers, forward computation, representations, losses, and expressive limitations. |
| 54 | Backpropagation and automatic differentiation | Computation graphs, the chain rule, explicit network gradients, gradient flow, and numerical gradient checks. |
| 55 | Training and regularizing neural networks | Minibatches, SGD, momentum, adaptive optimizers, initialization, normalization, dropout, weight decay, schedules, and debugging. |
| 56 | A deep learning framework and complete training loop | Tensors, automatic differentiation, data loaders, training/evaluation modes, device handling, checkpoints, and reproducible evaluation. |
The framework will be selected and its APIs verified when this part is authored. This outline does not assume a framework is installed. Core experiments should run on a Windows laptop; larger training runs are optional.
Part X — Deep learning architectures and representation learning
Part IX is the common prerequisite. Sequence models introduce the sequence supervision they use; the transformer chapter develops attention and masking explicitly. Readers can choose image, sequence/text, graph, or representation branches without completing every other branch. For transfer learning (58), first study CNNs (57) or the architecture being adapted. Language modeling (62) builds on transformers (61). The PCA comparison in autoencoders (64) also requires Chapters 37–38, including for readers taking the direct deep-learning route.
| No. | Chapter | Main coverage |
|---|---|---|
| 57 | Convolutional neural networks | Convolution, padding, stride, pooling, receptive fields, parameter counts, shape tracing, and small image classifiers. |
| 58 | Transfer learning and fine-tuning | Pretrained representations, freezing, fine-tuning, augmentation, limited-data experiments, and domain mismatch. |
| 59 | Recurrent neural networks, LSTMs, and GRUs | Recurrent state, variable-length sequences, backpropagation through time, gating, masking, and gradient limitations. |
| 60 | Temporal convolutional networks — extension | Causal and dilated convolution, receptive fields, padding, and information timing; builds on Chapter 57. |
| 61 | Attention and transformers | Query/key/value operations, self- and cross-attention, positional information, masks, encoder/decoder structures, and computational costs. |
| 62 | Language modeling and pretrained language models — extension | Tokenization, embeddings, autoregressive and masked objectives, pretraining versus adaptation, small language models, and task-specific evaluation. |
| 63 | Graph neural networks — extension | Graph representations, message passing, aggregation, node/edge/graph tasks, and evaluation splits. |
| 64 | Autoencoders and learned representations | Bottlenecks, reconstruction, denoising, representation evaluation, and comparison with PCA. |
| 65 | Self-supervised representation learning — extension | Contrastive objectives, augmentations, masked prediction, downstream evaluation, and separation of pretraining and test data. |
Part XI — Specialized learning problems and advanced models
Choose these branches by task. Classical forecasting and recommendation do not require deep learning. HMMs use probability and latent-variable estimation; Kalman filtering uses linear algebra and Gaussian models. Neural generative models use Part IX. For variational autoencoders (70), first study Bayesian inference (50) and autoencoders (64); latent-variable and variational ideas are then developed in the chapter. Deep reinforcement learning follows the tabular foundation (73).
| No. | Chapter | Main coverage |
|---|---|---|
| 66 | Time-series forecasting | Forecast horizons, autocorrelation, stationarity, lag/rolling features, statistical baselines, backtesting, and optional neural comparisons. |
| 67 | Recommendation systems and ranking | Explicit/implicit feedback, neighborhood methods, matrix factorization, cold start, ranking metrics, and user/time-aware evaluation. |
| 68 | Hidden Markov models — extension | Hidden states, forward/backward calculations, Viterbi decoding, likelihood, and parameter estimation. |
| 69 | State-space models and Kalman filtering — extension | Prediction/update steps, covariance propagation, linear-Gaussian assumptions, and an introduction to nonlinear extensions. |
| 70 | Variational autoencoders — extension | Latent-variable generation, variational approximation, KL divergence, the evidence lower bound, and reparameterization. |
| 71 | Generative adversarial networks — extension | Generator/discriminator objectives, adversarial training, instability, mode collapse, and evaluation beyond visual plausibility. |
| 72 | Diffusion models — extension | Forward corruption, denoising objectives, reverse sampling, small educational examples, and compute costs. |
| 73 | Reinforcement learning foundations — extension | Markov decision processes, returns, Bellman equations, dynamic programming, exploration, and tabular Q-learning. |
| 74 | Deep reinforcement learning — extension | Value approximation, policy gradients, actor–critic methods, training stability, and evaluation across runs. |
A classical forecast uses the workflow, regression, and temporal evaluation chapters; a neural forecast also requires its architecture chapter. Recommendation adds matrix decomposition and optimization. Formal variational treatments in Chapter 72 refer back to Chapter 70 rather than assuming the derivation is already known.
Part XII — End-to-end applications and deployment
Each project starts with a problem statement, data/label audit, baseline, and evaluation protocol. It reuses the relevant methods and adds the practical decisions needed to combine them. Projects use small, documented data; larger datasets and compute-intensive extensions remain optional.
| No. | Chapter | Main coverage |
|---|---|---|
| 75 | Tabular regression and classification projects | Data audit, baselines, feature pipelines, contrasting model families, tuning, error analysis, and final held-out evaluation. |
| 76 | Image learning projects | Image classification, augmentation, transfer learning, error analysis, and optional detection/segmentation extensions with their own metrics. |
| 77 | Text learning projects | Text preprocessing, sparse features and classical baselines, learned representations, document classification, and evaluation of a small language-model application. |
| 78 | Time-series and sensor projects | Forecasting or sequence classification, synchronization, sampling, windows, temporal features, group/time boundaries, and model comparison. |
| 79 | Deployment, monitoring, and maintenance | Packaging, versioning, prediction interfaces, latency, training/serving consistency, drift, feedback, retraining, and reproducibility. |
| 80 | Reproducible research and technical reporting | Experiment records, ablations, uncertainty, negative results, external validation, limitations, dataset/model documentation, and defensible conclusions. |
For Chapter 75, start with Parts III–VI. Chapter 76 adds CNNs and transfer learning; Chapter 77 starts with classical text features and Naive Bayes or logistic regression before any neural extension. Chapter 78 uses Chapter 66 for forecasting or the chosen classification model. Deployment and reporting can accompany any completed project.
Optional case studies may include demand forecasting, document classification, recommendation, gait or wearable sensing, and simulated robotics. A gait example must define whether the target is identity, activity, walking condition, or a defensible clinical label. Rehabilitation evidence and robotic control require their own domain assumptions; their earlier proposed treatments remain available in the archived planning document. No case study is required for learning the general methods.
Part XIII — Revision, algorithm selection, and interviews
These chapters consolidate material already taught. They do not replace the worked exercises and revision summary within each algorithm chapter.
| No. | Chapter | Main coverage |
|---|---|---|
| 81 | Selecting and comparing algorithms | Task-based selection, assumptions, sample and feature properties, runtime/memory budgets, diagnostics, and fair comparisons. |
| 82 | Mathematical and algorithm revision sheets | Essential equations, conditions, dimensions, mechanics, hyperparameters, costs, and frequent mistakes. |
| 83 | Interview questions and worked answers | Conceptual, mathematical, coding, debugging, and experimental-design questions with explanations. |
| 84 | Book synthesis and troubleshooting | Complete learning maps, symptom-to-check tables, reproducibility checks, and next-step guidance. The current version covers only the authored milestone. |
Appendices and references
The appendices are planned reference material; the bibliography page already exists.
| Appendix | Coverage |
|---|---|
| A. Notation and glossary | Symbols, dimensions, terminology, units, and abbreviations. |
| B. Python and numerical reference | Array shapes, common operations, stable arithmetic, debugging, and API reminders. |
| C. Mathematical and statistical reference | Matrix identities, derivatives, distributions, probability identities, and the conditions under which they apply. |
| D. Datasets and evaluation protocols | Provenance, licenses, labels, observation units, split records, and reproducible access instructions. |
| E. Environment and publishing guide | Verified dependencies, reproducibility records, HTML/PDF rendering, and troubleshooting. |
| References | Verified papers, books, dataset documentation, and official library documentation cited by authored chapters. |
Suggested learning paths
A complete first course in classical machine learning
Study Chapters 1–19, 23–25, 27, 30–33, 35–39, and 75, then use 79–84 alongside the completed project. This route covers foundations, regression, regularization, classification, trees/ensembles, evaluation, PCA, clustering, practical work, and revision. Chapter 37 supplies the matrix decompositions needed for PCA. Readers who already know the computing or mathematics material may review it through exercises.
This is the ML-only route. It requires no neural-network framework. Extend it with discriminant analysis, SVMs/kernels, the other clustering methods, or a probabilistic model when those comparisons become useful.
Machine learning followed by deep learning
Complete the first-course route or demonstrate its prerequisites, then study 53–59, 61, and 64. Choose an image or text project from 76–77, adding Chapter 62 for language-model work. Study 65 for self-supervised representations and 70–72 only when pursuing generative models.
A reader whose immediate aim is neural networks may take the direct prerequisite route specified in Part IX. Every route retains valid data splitting, preprocessing, evaluation, and baseline comparisons.
Reference, research, and interview use
Begin with the relevant task or model and follow its prerequisite links. Use 30–36 to examine the experiment design and 79–83 for operational, reporting, selection, and revision questions. Time-series readers add 66 and 78; recommendation readers add 67. The advanced branches are choices, not an obligatory checklist.