Create a template to generate an end-to-end training

[ai-fast-track]

🚀 Feature

Is your feature request related to a problem? Please describe. Create a script that generates an an end-to-end training code snippet:

method signature: generate_e2e_training(url: str, dataset_name: str, annotation_type="voc", model="efficientdet", backbone=None)

Describe the solution you'd like Mandatory Inputs: url: URL pointing to a dataset dataset_name: name of a dataset annotation_type: select coco or voc annotation

Optional Inputs: By default, we choose EfficientDet as model and tf_efficientdet_lite0 as a backbone model: efficientdet, faster_rcnn or mask_rcnn. By default, we choose efficientdet backbone: Corresponding backbone. For more details, check EfficientDet and Faster/Mask RCNN backbone lists

If the model is efficientdet, backbone = "tf_efficientdet_lite0" If the model is faster_rcnn or mask_rcnn, the default backbone is: backbone = backbones.resnet_fpn.resnet50(pretrained=True)

For efficientdet, check out the quickstart example For faster_rcnn, check out the getting_started example For faster_rcnn, check out the mask_rcnn_pennfudan example

Output: A ready to train end-to-end training code.

Example: My Awesome Dataset

Inputs:

url  = "https://path/to/dataset/MyAwesomeDataset2020.zip"
dataset_name = "my_awesome_dataset"
annotation_type = "voc":    # select `coco` or `voc` annotation

Output:

# pip install icevision[all] icedata

from icevision.all import *

url = "https://path//to/dataset/MyAwesomeDataset2020.zip"
dest_dir = "my_awesome_dataset"

# Loading Data
data_dir = icedata.load_data(url, dest_dir)

# Parser
class_map = ClassMap(["ADD", "YOUR", "LABELS", "LIST", "HERE"])
parser = parsers.voc(annotations_dir=data_dir / "my_awesome_dataset/annotations",  # Point to the annotations folder
                    images_dir=data_dir / "my_awesome_dataset/images", # Point to the images folder
                    class_map=class_map)

# Records
train_records, valid_records = parser.parse()
show_records(train_records[:3], ncols=3, class_map=class_map)

# Transforms
train_tfms = tfms.A.Adapter([*tfms.A.aug_tfms(size=384, presize=512), tfms.A.Normalize()])
valid_tfms = tfms.A.Adapter([*tfms.A.resize_and_pad(384), tfms.A.Normalize()])

# Datasets
train_ds = Dataset(train_records, train_tfms)
valid_ds = Dataset(valid_records, valid_tfms)

# DataLoaders
train_dl = efficientdet.train_dl(train_ds, batch_size=16, num_workers=4, shuffle=True)
valid_dl = efficientdet.valid_dl(valid_ds, batch_size=16, num_workers=4, shuffle=False)

# Model and Metrics
model = efficientdet.model(model_name="tf_efficientdet_lite0", num_classes=len(class_map), img_size=size)
metrics = [COCOMetric(metric_type=COCOMetricType.bbox)]

# Training using Fastai
learn = efficientdet.fastai.learner(dls=[train_dl, valid_dl], model=model, metrics=metrics)
learn.fine_tune(50, 1e-2, freeze_epochs=20)

# Inference
# DataLoader
infer_dl = efficientdet.infer_dl(valid_ds, batch_size=8)
# Predict
samples, preds = efficientdet.predict_dl(model, infer_dl)
# Show samples
imgs = [sample["img"] for sample in samples]
show_preds(
    imgs=imgs[:6],
    preds=preds[:6],
    class_map=class_map,
    denormalize_fn=denormalize_imagenet,
    ncols=3,
)

[FraPochetti]

Not relevant anymore