[NEW!] checkpoint release: https://drive.google.com/drive/folders/11V1XJCORcm3XQ_AfaOEfRaKFZpjMLyVm?usp=drive_link (if it expired, contact me via qying@nvidia.com)

A light-weight e2e dl framework for all papers

This framework is developed on the basis of source code from "Invertible image decolorization", which also uses invertible network for image-to-image translation.

In separate research works, we usually encounter repeated codes such as training loop, launching DDP and so on. So I developed this framewotk and the scripts can be called via routers, i.e., using "opt" to define options and using "mode" to do different experiments/ablations for a same project.

Learning to Immunize Images for Tamper Localization and Self-Recovery

Please see README_PAMI.md

DefensiveRAW: Robust Image Authentication via Transferable RAW Protection

Mode Define

• mode=0: generating protected images (val)
• mode=1: tampering localization on generating protected images (val)
• mode=2: regular training, including ISP, RAW2RAW and localization (train)
• mode=3: regular training for ablation (RGB protection), including RAW2RAW and localization (train)
• mode=4: OSN performance (val)
• mode=5: train a ISP using restormer for validation (train)
• mode=6: train passive image manipulation detection networks (train)

  Issues

• restormer cannot be loaded simultaneously with OSN network, because they share the same variable localizer
• now you need to specify the path where each model locates

How to test OSN in the code?

• run bash ./run_ISP_OSN.sh (mode==4)

• Line 141 of Modified_invISP.py, modify the model as that of OSN network

• specify Line 1319-2323 which provides the tamper source and mask

• the setting file is train_ISP_OSN.yml. If you want to do automatic copy-move, set inference_tamper_index=2 and inference_load_real_world_tamper=False

• using_which_model_for_test decides using which model for testing. discriminator_mask is our method, localizer is OSN.

• The average F1 score will be printed in the console.

• The main loop loops over the training set. Therefore you should manually kill the process when all the validate images are runned.

• Voila! the flow is optimize_parameters_router -> get_performance_of_OSN

How to Test the baseline (RGB protection)?

• run bash ./run_ISP_OSN.sh (mode==4)
• the setting file is train_ISP_OSN.yml. Set test_baseline: true and task_name_customized_model: ISP_alone, load_customized_models: 64999(which loads the trained baseline model from that location)
• Voila! the flow is optimize_parameters_router -> get_performance_of_OSN

How to use Restormer during inference?

• whatever the option file yml is, set test_restormer: true, and the model localizer will be Restormer.
• Note: you cannot use OSN and Restormer at once!

Logs

1003

• test is now ok. Use mode=0, which would contain protected image generation and tampering localization.

1012

• update option and introduce base option

1024

• the project has been restarted.

1030

• including cropping in testing. See Line 1373-1375.