Walk with fastai revisited - Project

[manisnesan]

Label: Current

The goal is to create a hf space showcasing a deep learning application as part of Walk with fastai course project. Personal goal is to get myself comfortable with the MidLevel API.

Project Ideas

• [Finalized] FashionMNIST - use only low level fastai api
• ~Spot the mask - use only low level fastai api~

Tasks

• [x] Download & Load the fashion_mnist dataset using HuggingFace Dataset.
• [x] Create fastai dataloaders from hf dataset using Mid Level API such as Datasets from DataCore.
• [x] Train a model and Predict items
• [x] Add gradio interface and Deploy to HF space
• [x] Add a blog post

[manisnesan]

Code - Google Colab

[manisnesan]

HuggingFace Space for Fashion MNIST Classifier

[manisnesan]

Idea

From fastbook Chapter 11 - Data Munging with fastai's Mid-Level API

What can we do when the data block API is not flexible enough to accommodate our particular use case? .

For this, we need to use fastai's mid-level API for processing data. The data block API is built on top of that layer, so it will allow you to do everything the data block API does, and much much more.

It encompasses Transforms, Pipeline, TfmdLists, Datasets, Callbacks and General Optimizer

Context

• Dataset: Fashion MNIST classification dataset containing black white images of size 28x28 of apparels along with the label such as t-shirt, ankle boot containing 10 classes.
• Goal is to create a image classifier predicting the target class given an black white image of a fashion apparel.

End to end workflow

Load Dataset -> Create Dataloaders -> Train Model -> Predict Item -> Deploy with Gradio to HF space

Dataset

• Download and load fashion_mnist from https://huggingface.co/datasets/fashion_mnist using HuggingFace datasets library.

Fashion-MNIST is a dataset of Zalando's article images—consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes.

• Explore the dataset using load_dataset_builder inspecting the dataset info such as description, features, splits etc
• Load the dataset specifying the name.
• This is a DatasetDict containing a train and test dataset dictionary within this object.
• Inspecting the individual item within train and test and the different labels.

HF Dataset to fastai DataLoaders

• Inspecting the type of features and label
• type(train[0]['image']) is PIL.PngImagePlugin.PngImageFile
• type(train[0]['label') is an int
• Combine the train and test into a single list of items with index from 60000 to 70000 set aside as validation set.
• In order to inform fastai on how to turn the data into DataLoaders (???) object , 4 key pieces of info are needed. 1. the kind of data used for independent variable ie item and dependent variable ie target 2. Getters for the list of items 3. Labeling the items 4. Validation set creation
• https://github.com/fastai/fastbook/blob/master/02_production.ipynb

  
  def get_image_attr(x): return x['image']
  def get_label_attr(x): return x['label']

dblock = DataBlock(blocks=(ImageBlock(cls=PILImageBW), CategoryBlock), get_x=get_image_attr, get_y=get_label_attr, splitter=IndexSplitter(valid_idx=L(range(60000, 70000))))

- Run the `DataBlock.summary` to understand how fastai set up the data pipeline and perform the necessary transforms. 

2 datasets of sizes 60000,10000 Setting up Pipeline: get_image_attr -> image2tensor -> PILBase.create Setting up Pipeline: get_label_attr -> Categorize -- {'vocab': None, 'sort': True, 'add_na': False}

- Get the image attribute from the item, Convert the image to tensor. But fastai also adds the `PILBase.create` since we specified ImageBlock as our independent variable. This caused an issue `KeyError: ((1, 1, 28), '|u1')` due to `Image.fromarray` function used in `PILBase.create`.  From https://stackoverflow.com/questions/57621092/keyerror-1-1-1280-u1-while-using-pils-image-fromarray-pil
> Pillow's fromarray function can only do a MxNx3 array (RGB image), or an MxN array (grayscale).     
- Essentially we want to convert the Image of size [28 ,28] into [1, 28, 28] as our end goal and decided to perform the item transforms using Mid Level API instead.
- `Datasets` in MidLevel API provides more flexibility and full control over the individual item transforms performed.

splits = IndexSplitter(valid_idx=L(range(60000, 70000)))(concat_dsets) dsets = Datasets(concat_dsets, [[get_image_attr], [get_target_attr, Categorize]], splits=splits)

- Pass the items, followed a list of transforms in a tuple (first one specifying the transforms perform on item, second one specifying the transforms on the target)
- We will also create a vocabulary of our target classes. Because integer `label` are not that helpful.

classLabel = ClassLabel(names=['T - shirt / top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot'], id=None)

def add_target(x:dict): x['target'] = classLabel.int2str(x['label']) return x

train = train.map(lambda x: add_target(x)) valid = test.map(lambda x: add_target(x))

- Convert the `datasets` to `dataloaders` along with the set of item transforms specified using `after_item` and set of batch transforms specified using `after_batch`.

def img2tensor(im: Image.Image): return TensorImageBW(array(im)).unsqueeze(0)

item_tfms = [img2tensor] # convert PILImage to tensors batch_tfms = [IntToFloatTensor] # convert the int tensors from images to floats, and divide every pixel by 255 dls = dsets.dataloaders(after_item=item_tfms, after_batch=batch_tfms, bs=64, num_workers=8)

- Show batch of items on the dataloaders
- Check individual item and target from a single batch
- The item is of shape [64,1, 28, 28] ie batch size of 64, single channel, height as 28, width as 28. 
- Check the number of classes and target vocabulary.

## Train the model

- We will use resnet architecture as our backbone and linear layer as our head.
- We will set `pretrained` as False since fashion mnist dataset is not similar to imagenet dataset used in the pretrained model.
- Check the model input layer and since it accepts in_channels as 3 (RGB) but since the dataset is a greyscale image. 

model = resnet34(num_classes=dls.c).cuda() print(model.conv1)

- Let's change it in_channels of size 1.
`model.conv1 = nn.Conv2d(in_channels=1, out_channels=64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)`
- Create the learner using dataloaders, model architecture and the metrics as accuracy
- Get the learning rate using the learning rate finder
- Train the model using `fit_one_cycle` for 6 epochs and using the learning rate from previous step.
- Model is trained with an accuracy of 92.3% 
- Export the trained model

## Inference

- Load the learner from the exported model
- Predict the item using `Learner.predict`. This performs the necessary transforms on the item using the learner.

### Deployment using Gradio to HF space

- Create a new space in HF Space
- Upload the exported model
- Move all the necessary functions used as part of the transforms for the inference. This includes all the getters.
- Add all the dependencies to `requirements.txt`
- Create a gradio interface passing the `classify` function, specifying the inputs(Image) and outputs(Label)
- See the complete code https://huggingface.co/spaces/msivanes/fashion-mnist-classifier/blob/main/app.py
- HuggingFace Space https://huggingface.co/spaces/msivanes/fashion-mnist-classifier

Note: This is my project write up for WalkWithFastai revisited course as one of my goal for this course is to get comfortable with low level API, debug issues diving into the source, uncovering the fastai magic. Thanks to Zach Mueller for an excellent course.  

## Outline
- Idea
- Context
- Objective
- Code Walkthrough of the Classifier application
- Troubleshooting
- Takeaways
- References
  - https://store.walkwithfastai.com/walk-with-fastai-revisited 
  - https://walkwithfastai.com/MNIST 
  - https://github.com/fastai/fastbook/blob/master/11_midlevel_data.ipynb 

[manisnesan]

Blog Checklist

• [x] Draft
• [x] Edit & Rewrite
• [x] Review & Publish

[manisnesan]

Published post

Minor Rendering issues

• yaml showing up

toc: true badges: true comments: true categories: [image-classification, machine-learning] image: images/chart-preview.png

• Change to bulleted

  Steps involved to prepare the data

• Change to bulleted

Steps involved to prepare the data

• Formatting issue

Datasets need the following pieces of information - raw items - the list of transforms that builds our inputs from the raw items - the list of transforms that builds our targets from the raw items - the split for training and validation

• Random code shown

::: {.cell vscode='{"languageId":"python"}'}

learn.export(fname='export.pkl')
:::