Machine Learning: Bias vs Variance

The bias-variance tradeoff is bias–variance interesting dilemma, and it’s important to understand if you ever want to do something useful in machine learning.

Let’s recap simple supervised machine learning first.

Supervised learning

To understand the bias-variance tradeoff, let’s discuss supervised machine learning for one second before moving on. When doing supervised learning, the idea is to learn to receive input and predict the correct output.

Gif by author.

One example of this task is to take an image of a handwritten number and then get the machine to recognize what number it is.

Gif by author.

For the computer to learn, you must provide sufficient data examples. One data point is an input and its corresponding output. This is represented below with a gray cube.

Image by author.

The dataset must contains multiple data points, to achieve a good machine learning model.