Train and Finetune

[IbrahimSobh]

Thank you very much for the impressive work

It is not clear to me:

1) How to fine-tune the existing models on another dataset? 
2) How to train a deep privacy model from scratch? 

In both cases, what are the exact steps, scripts, formats, etc

Regards

[hukkelas]

1. How to fine-tune the existing models on another dataset?

I haven't done this myself and haven't made any functionality for this. But, you could do it in a "hacky" way. I'm going to create this example for our largest face model trained on the FDF dataset (see config here).

• First, you need to create a new config for your new model with the new dataset config. Let's in this example place this under configs/fdf/finetuned_model.py. Also, I'm going to create a "dummy" example where we want to finetune a model on the CelebA-HQ dataset which was originally trained for FDF:

__base_config__ = "512.py" # Inherit all settings from your finetune model

dataset_type = "CelebAHQDataset"
data_root = os.path.join("data", "celebA-HQ")
data_train = dict( # Overwrite all dataset config. For celebA-HQ, you have to do this.
    dataset=dict(
        type=dataset_type,
        dirpath=os.path.join(data_root, "train"),
        percentage=1.0,
        is_train=True
    ),
    transforms=[
        dict(type="RandomFlip", flip_ratio=0.5),
        dict(type="FlattenLandmark")

    ],
)
data_val = dict(
    dataset=dict(
        type=dataset_type,
        dirpath=os.path.join(data_root, "val"),
        percentage=.2,
        is_train=False
    ),
    transforms=[
        dict(type="FlattenLandmark")
    ],
)

• When your config is defined, you need to copy the checkpoints to the new checkpoint folder. Sadly, I have no checkpoints that can be finetuned from the FDF dataset, as we slimmed the checkpoints before uploading them to our model zoo :/ So you will have to retrain the model.:. cp -r outputs/fdf/512/checkpoints outputs/fdf/finetuned_model

2. How to train a deep privacy model from scratch? Face detection and keypoint models are already pre-trained, so you only need to train a new model. You just need to download the FDF dataset (see here), and place this under data/fdf. Then it should be possible to train a model with the line python train.py configs/fdf/512.py

[SanderKlomp]

Regarding 2: I have been trying to train from scratch as well, but running into the issue that the FDF dataset is split into different sizes when downloaded (8x8, 16x16, 32x32, 64x64, 128x128) with corresponding scaled bounding boxes and keypoints files, while configs/fdf/512 simply expects a single folder of images and one bounding box and one keypoints file. Of course I could simply use only the 128 scale, but I'm not sure if that is intended for reproducing your (highly impressive) results. The 512.py config thus does not apply any progressive training (which matches "progressive=False" in its config). Is the progressive training indeed not required for the best fdf model?

[hukkelas]

You only require the 128 scale. We removed progressive training with MSG-GAN in our recent paper, see https://arxiv.org/abs/2011.01077 for details :)

[SanderKlomp]

Thanks for the answer and the link to the most recent paper. The results are already looking pretty decent here after just a few hours of training on a single GPU. Will definitely check out your the new paper as well :)