AI For Filmmaking

Code for the blog post AI for Filmmaking. Detect cinematic shot types in images using a pre-trained ResNet-50.

This model recognises 6 shot types:

1. Extreme Wide Shot

2. Long Shot

3. Medium Shot

4. Medium Close Up

5. Close Up

6. Extreme Close Up

In the not so distant future, it will also recognise:

Wide Shots

Medium Long Shots

Requirements

fastai — Installation instructions here. You can use this code without a GPU.

Usage

After downloading the directory, run bash get_data_model.sh to download the model and the validation set. A dummy training set is downloaded too to enable the generation of heatmaps.

Predict Shot Types

python get-preds.py                     \
    --path_base  ~/shot-type-classifier \
    --path_img   ~/images               \
    --path_preds ~/images/preds         \

Where path_base is the directory path, path_img the path to the images you want to evaluate, and path_preds where you'd like to store the predictions (.csv files). The script create the ~/images/preds folder if it doesn't exist.

Heatmaps

python get-heatmaps.py                 \
    --path_base ~/shot-type-classifier \
    --path_img  ~/images               \
    --path_hms  ~/images/heatmaps      \
    --alpha 0.8

Where path_base is the directory path, path_img the path to the images you want to evaluate, path_hms where you'd like to store the heatmaps, and alpha the blending value of the heatmap with the original image. An alpha value of 1.0 produces the heatmap only. The script create the ~/images/heatmaps folder if it doesn't exist.

License

This repository is released under the Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0). See here for more details.