Use training and validation behavior to diagnose whether a model is too simple or too complex.
The core idea
Underfitting occurs when a model cannot capture the useful pattern, so it performs poorly on training and validation data. Overfitting occurs when a model fits training details that do not generalize, causing a gap between training and validation performance.
A practical way to approach it
Plot learning curves, compare a simple baseline with more complex models, and inspect errors by segment. For overfitting, consider regularization, more representative data, simpler features, or early stopping. For underfitting, revisit features and model capacity.
Try it in a small project
Train a decision tree at several depths and compare train and validation scores. The exercise shows why the best training score is not necessarily the best model for unseen data.
Common mistakes to avoid
Do not tune endlessly to a single validation set. Use cross-validation when appropriate and reserve a final test set for an unbiased estimate.
Key takeaway
Keep the first version small, make your assumptions explicit, and verify the result with a repeatable test. For production use, check the current official documentation for the tools and versions you choose.