Read it, then run it
This page is the read-only chapter intro. To actually execute the cells (load the dataset, train a model, render GradCAM heatmaps on your own images), open the Colab notebook in a free T4 runtime. The curriculum is designed to be executed, not read.
Chapter 5: Training, Validation & Hyperparameter Tuning¶
Learning objectives: - Understand the training loop: forward pass, loss, backpropagation, weight update - Build a reusable training function with progress tracking - Train and compare DenseNet121, ViT-Small, and an SGD variant - Visualize and interpret training curves
Clinical context: Training a model is like teaching a radiology resident - you show them cases (forward pass), tell them what they got wrong (loss), explain how to adjust their reasoning (backpropagation), and they improve case by case (weight update). The number of teaching sessions (epochs) and how aggressively you correct them (learning rate) are critical choices.
The Training Loop: Step by Step¶
Every epoch (one pass through the training data) repeats four steps:
| Step | What happens | Clinical analogy |
|---|---|---|
| 1. Forward pass | Show the model an image, get its prediction | Resident reads an image and gives their diagnosis |
| 2. Loss computation | Measure how wrong the prediction was | Attending says "that's wrong, here's the correct answer" |
| 3. Backward pass | Figure out which weights contributed to the error | Resident reflects on which reasoning steps led them astray |
| 4. Optimizer step | Adjust the weights to reduce the error | Resident updates their diagnostic approach |
After every epoch, we evaluate on the validation set to check for overfitting - just like periodically testing a resident on cases they haven't studied from.
Key Hyperparameters¶
These are settings YOU choose before training - the model cannot learn them from data:
| Hyperparameter | What it controls | Analogy |
|---|---|---|
| Learning rate | How big each correction step is | Too large → overshoots the right answer. Too small → barely learns. |
| Batch size | How many cases are reviewed before updating | Reviewing 300 cases before adjusting vs. 1 at a time |
| Epochs | How many passes through the full dataset | How many times the resident reviews the entire case library |
| Optimizer | The strategy for weight updates | Different learning styles - Adam (adaptive) vs. SGD (steady) |
Note: We use
max_epochs=4for speed in this tutorial. In real clinical AI research, you'd train for 50–200 epochs with early stopping - automatically halting when validation performance plateaus.
Hyperparameter Tuning: Does the Optimizer Matter?¶
One of the most impactful hyperparameter choices is the optimizer. We trained DenseNet121 with Adam (adaptive learning rates per parameter). Let's try SGD with momentum - a simpler optimizer with a single global learning rate.
Key differences: - Adam: adapts the learning rate for each parameter individually. Often converges faster. - SGD + momentum: uses a single learning rate for all parameters, with momentum to smooth updates. Often generalizes better when tuned carefully.
Important: We must create a fresh model instance - we can't reuse the Adam-trained model, as its weights have already been optimized. That would be like comparing two teaching methods but starting one group with pre-existing knowledge.