I implemented pytorch iCNN codes in which you can change the scaling method or the encoder by configuration files.
The main features are listed below:
Support CLIP model
changed FeatureExtractor so that it can deal with inputs/outputs given as tuples (outputs of attention layers and inputs of all types of layers)
added CLIP similarity loss
Added data augmentations
you can apply image augmentation/modification for both types of losses, i.e., image encoder activation loss and CLIP similarity loss
Additional requirements:
natsort
clip (if you want to use CLIP models)
NOTE:
I removed image clipping for images generated by a generator since I'm not sure of its importance. Instead, images will be interpolated before fed into encoders used in losses.
I have not tested optimization in the pixel space, or other generators.
The main reconstruction script is recon_icnn_using_class.py. You can reconstruct images by running python recon_icnn_using_class.py --recon_conf <path to configuration file>.
TODO:
Currently DA for image encoder activation loss works correctly only for CLIP ViT. I have to normalize image into [0, 1] before augmentations since korniaexpects inputs in that range.
Several dirty solutions, mainly due to inconsistent labels in the emotional movie dataset, still exist.
I implemented pytorch iCNN codes in which you can change the scaling method or the encoder by configuration files. The main features are listed below:
Additional requirements:
NOTE:
The main reconstruction script is
recon_icnn_using_class.py. You can reconstruct images by runningpython recon_icnn_using_class.py --recon_conf <path to configuration file>.TODO:
korniaexpects inputs in that range.