Understanding Machine Learning Models with MNIST

This is one of the simplest Machine Learning models you'll see.

Most people have heard about MNIST, but only a few can explain how this code works.

Let's go together line by line and understand what's happening here: pic.twitter.com/JNRzuUb9pY

— Santiago (@svpino) June 8, 2023

Writing changed my life.

One morning, I started learning how to communicate my ideas effectively. The rest is history.

Look at the "Writing for Data Scientists" ebook. It may transform your life too.https://t.co/LbvEYQkCu7

Thanks to @themwiti for sponsoring this thread! pic.twitter.com/DbTfdiQzch

— Santiago (@svpino) June 8, 2023

Let's start by loading the MNIST dataset.

It contains 70,000 28x28 images showing handwritten digits.

The function returns the dataset split into train and test sets. pic.twitter.com/jwguMawyxp

— Santiago (@svpino) June 8, 2023

x_train and x_test contain our train and test images.

y_train and y_test contain the target values: a number between 0 and 9 indicating the digit shown in the corresponding image.

We have 60,000 images to train the model and 10,000 to test it. pic.twitter.com/qx3S7nTzPm

— Santiago (@svpino) June 8, 2023

When dealing with images, we need a tensor with 4 dimensions: batch size, width, height, and color channels.

x_train is (60000, 28, 28). We must reshape it to add the missing dimension ("1" because these images are grayscale.) pic.twitter.com/Bdtm3wpgrP

— Santiago (@svpino) June 8, 2023

Each pixel goes from 0 to 255. Neural networks work much better with smaller values.

Here we normalize pixels by dividing them by 255. That way, each pixel will go from 0 to 1. pic.twitter.com/hwXzXIDuhL

— Santiago (@svpino) June 8, 2023

Target values go from 0 to 9 (the value of each digit.)

This line one-hot encodes these values.

For example, this will transform a value like 5 in an array of zeros with a single 1 corresponding to the fifth position:

[0, 0, 0, 0, 0, 1, 0, 0, 0, 0] pic.twitter.com/xxyFL8C4r8

— Santiago (@svpino) June 8, 2023

Let's now define our model.

There are several ways to create a model in Keras. This one is called the "Sequential API."

Our model will be a sequence of layers we will define individually. pic.twitter.com/J3z42z8vmD

— Santiago (@svpino) June 8, 2023

A lot is going on with this first line.

First, we define our model's input shape: a 28x28x1 tensor (width, height, channels.)

This is exactly the shape we have in our train dataset. pic.twitter.com/vcJVmszPv5

— Santiago (@svpino) June 8, 2023

Then we define our first layer: a Conv2D layer with 32 filters and a 3x3 kernel.

This layer will generate 32 different representations using the training images. pic.twitter.com/mnAmTwQbjW

— Santiago (@svpino) June 8, 2023

We must also define the activation function used for this layer: ReLU.

You'll see ReLU everywhere. It's a popular activation function.

It will allow us to solve non-linear problems, like recognizing handwritten digits. pic.twitter.com/8LvxWjrDV9

— Santiago (@svpino) June 8, 2023

After our Conv2D layer, we have a max pooling operation.

The goal of this layer is to downsample the amount of information collected by the convolutional layer.

We want to throw away unimportant details and retain what truly matters. pic.twitter.com/0hThPlIzwm

— Santiago (@svpino) June 8, 2023

We are now going to flatten the output. We want everything in a continuous list of values.

That's what the Flatten layer does. It will give us a flat tensor. pic.twitter.com/zgNLRCRc9k

— Santiago (@svpino) June 8, 2023

Finally, we have a couple of Dense layers.

Notice how the output layer has a size of 10, one for each of our possible digit values, and a softmax activation.

The softmax ensures we get a probability distribution indicating the most likely digit in the image. pic.twitter.com/7g9uVGOrTa

— Santiago (@svpino) June 8, 2023

After creating our model, we compile it.

I'm using Stochastic Gradient Descent (SGD) as the optimizer.

The loss is categorical cross-entropy because this is a multi-class classification problem.

We want to record the accuracy as the model trains. pic.twitter.com/bxCxlov56m

— Santiago (@svpino) June 8, 2023

Finally, we fit the model. This starts the training process.

A couple of notes:

• I'm using a batch size of 32 images.
• I'm running 10 total epochs.

When fit() is done, we'll have a fully trained model! pic.twitter.com/ssFBwYgQTL

— Santiago (@svpino) June 8, 2023

Let's now test the model.

This gets a random image from the test set and displays it.

Notice that we want the image to come from the test set, containing data the model didn't see during training. pic.twitter.com/u5aqcDOOcq

— Santiago (@svpino) June 8, 2023

We can't forget to reshape and normalize the image as we did before. pic.twitter.com/thxx11UblJ

— Santiago (@svpino) June 8, 2023

Finally, I predict the value of the image.

Remember that the result is a one-hot-encoded vector. That's why I take the argmax value (the position with the highest probability), and that's the result. pic.twitter.com/m98AYcoUmw

— Santiago (@svpino) June 8, 2023