Practical: RNN vs CNN with Fashion-MNIST

Anastasia Giachanou, Tina Shahedi

Machine Learning with Python - Utrecht Summer School

In this practical, we'll explore and understand the application of advanced deep learning models, including Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), using the Fashion-MNIST dataset.

!pip install scikeras[tensorflow] > /dev/null 2>&1     # gpu compute platform
!pip install scikeras[tensorflow-cpu] > /dev/null 2>&1
!pip install scikeras > /dev/null 2>&1
!pip install pydot graphviz > /dev/null 2>&1
!pip uninstall -y scikit-learn
!pip install scikit-learn==1.5.2

from scikeras.wrappers import KerasClassifier

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import tensorflow as tf
import seaborn as sns
import random
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.utils import plot_model
from tensorflow.keras.layers import Dense, Flatten, BatchNormalization, Dropout, Conv2D, MaxPooling2D, LSTM
from tensorflow.keras.regularizers import l2
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
from tensorflow.keras.optimizers import *
from tensorflow.keras.optimizers import Adam, SGD, RMSprop
from sklearn.model_selection import RandomizedSearchCV
from sklearn.metrics import classification_report, confusion_matrix

Run the following lines to prepare the data for the models (the code was also used in the previous practical session).

# Load the dataset
fashion_mnist = tf.keras.datasets.fashion_mnist
(sample_images, sample_labels), (test_images, test_labels) = fashion_mnist.load_data()

# Set a random seed for reproducibility
np.random.seed(100)

# Randomly choose 15,000 indices from the range of train_images length
indices = np.random.choice(sample_images.shape[0], 15000, replace=False)

# Use these indices to sample images and labels
train_images = sample_images[indices]
train_labels = sample_labels[indices]

# Now sample_images and sample_labels contain your 15,000 samples
print("train_images shape:", train_images.shape)
print("train_labels shape:", train_labels.shape)

# Flatten the image data and convert it to a DataFrame
# The images are reshaped from 28x28 to 784 per image
train_images_flattened = train_images.reshape(train_images.shape[0], -1)
test_images_flattened = test_images.reshape(test_images.shape[0], -1)

# Create DataFrames
train_df = pd.DataFrame(train_images_flattened)
test_df = pd.DataFrame(test_images_flattened)

# Add labels to the DataFrames
train_df['label'] = train_labels
test_df['label'] = test_labels

# Normalize the pixel values to be between 0 and 1
X_train= train_images.astype('float32') / 255.0
X_test = test_images.astype('float32') / 255.0

# One-hot encode the labels
y_train = tf.keras.utils.to_categorical(train_labels, num_classes=10)
y_test = tf.keras.utils.to_categorical(test_labels, num_classes=10)

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz
29515/29515 ━━━━━━━━━━━━━━━━━━━━ 0s 0us/step
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz
26421880/26421880 ━━━━━━━━━━━━━━━━━━━━ 0s 0us/step
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz
5148/5148 ━━━━━━━━━━━━━━━━━━━━ 0s 1us/step
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz
4422102/4422102 ━━━━━━━━━━━━━━━━━━━━ 0s 0us/step
train_images shape: (15000, 28, 28)
train_labels shape: (15000,)

Lets refresh our memory regarding the structure and characteristics of the dataset.

fig, ax = plt.subplots(6, 6, figsize=(8, 8))
fig.suptitle('Fashion images and labels', fontsize=14)
ax = ax.ravel()  # This line ensuring ax is a flat array

for i in range(36):
    sample_n = random.randint(0, X_train.shape[0] - 1)
    ax[i].imshow((X_train[sample_n]).reshape(28, 28), cmap='Greys')
    ax[i].get_xaxis().set_visible(False)
    ax[i].get_yaxis().set_visible(False)
    label_index = np.argmax(y_train[sample_n])
    ax[i].set_title(label_index, fontsize=12)


plt.subplots_adjust(hspace=0.3)

As we can see, each training example is assigned to one of the following labels:

1. T-shirt/top
2. Trouser
3. Pullover
4. Dress
5. Coat
6. Sandal
7. Shirt
8. Sneaker
9. Bag
10. Ankle boot

Let's begin!

Recurrent Neural Networks

A Recurrent Neural Network (RNN) is typically used for sequential data like text. However, it can be adapted for image data such as the Fashion MNIST dataset by processing rows or columns of pixels as sequences. This allows the RNN to capture spatial information over sequences of the image. Long-short term memory (LSTMs), a specialized kind of RNN, are particularly effective for this purpose. They selectively remember patterns over long sequences, which can be useful for learning the nuances of fashion item images.

1. Reshape the X_train and X_test variables. Use the shape (number of samples, 28, 28).

2. Build a Recurrent Neural Network (RNN) using the Keras Sequential API to classify images from the Fashion-MNIST dataset. The model should include: 1. Two LSTM layers: one with 64 units (returning sequences), the next with 32 units 2. A Dropout layer (rate 0.25) after each LSTM to prevent overfitting. 3. A Dense layer with 16 units and ReLU activation 4. A final Dense output layer with 10 units (softmax) for multi-class classification Compile the model using the Adam optimizer and categorical crossentropy as the loss function.

3. Output the summary of the model and visualize the architecture of RNN model by using the plot_model function from keras.utils.

4. Now it's time to train! Train (with fit()) the RNN model for 5 epochs and a batch size of 64, using x_train_rnn and y_train, and validate using x_test_rnn and y_test. Save the training history.

5. Plot the model's accuracy and loss over epochs. You can use the same function as in the previous practical

6. Calculate the RNN model's accuracy on the training and test datasets.

7. Train the RNN model for 10 epochs.

8. Plot the results of the model with the 10 epochs.

Convolutional neural networks

Convolutional Neural Networks (CNNs), often referred to as convnets, have significantly transformed the landscape of machine learning, especially in image classification and computer vision. Their ability to autonomously extract and learn features from images has set new benchmarks in how machines interpret visual information.

9. Reshape Fashion MNIST images to include a channel dimension for CNN input, converting 2D arrays (28x28) into 3D arrays (28x28x1).

10. Build a CNN model with Keras for the Fashion-MNIST dataset. Reshape images for Conv2D compatibility, add a MaxPooling2D (pool size 2) and a Dropout layer (rate 0.25), flatten for dense layers, then use a 32-unit Dense layer and a 10-unit 'softmax' output layer. Compile with Adam optimizer and categorical_crossentropy.

11. Print the summary and use the plot_model function from keras.utils to create a visual representation of your CNN model's architecture.

12. Fit the model in 5 epochs with a batch size of 64. Save the training process in a variable named history.

13. Evaluate the accuracy of the model on the training and test data.

Hyperparameter Optimization

14. Define a function named create_model that constructs a CNN model for the Fashion MNIST dataset. The function should accept the number of filters, kernel size, and dense layer size (embedding size) as input arguments. Utilize the structure of your prior CNN model, ensuring these parameters dynamically define the model's convolutional layers and dense layer accordingly.

15. Use a Python dictionary to define a set of hyperparameters for your CNN model, including the number of filters, kernel size, and dense layer size. Implement this in your create_model function, allowing for dynamic model configuration.

16. Use the KerasClassifier from scikeras and set model to create_model function. Set epochs = 5 and batch_size=64

17. Use RandomizedSearchCV together with the KerasClassifier model you created and the predefined hyperparameter grid. Set it up for 5-fold cross-validation. Set n_iter to 3 (we do that so the fitting part will not take a lot of time during the practical.)

18. Fit the RandomizedSearchCV instance with x_train_cnn and y_train, initiating the search for the best hyperparameters.

19. Identify and note the best score and hyperparameters obtained from the RandomizedSearchCV results.

20. Evaluate the tuned model on the test set (X_test, y_test) to measure its final performance.

Predict

21. Predict the labels for the test dataset using the trained model.

Evaluating Model

22. Evaluate the performance of the model using a classification report and confusion matrix. You can use the fashiol labels if you want to add the categories of the items on the confusion matrix (fashion_labels = ["T-shirt/top", "Trouser", "Pullover", "Dress", "Coat", "Sandal", "Shirt", "Sneaker", "Bag", "Ankle boot"] )

Optional. Visualize predictions of a model on the Fashion MNIST dataset by plotting images with predicted and true labels, highlighting correct and incorrect predictions.

End of practical!