ERROR when changed to my own image in the demo

[blancozhang]

when I change the ori_img to my own images it will show: "view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.", also I have change my own image to the same resolution as the sample one which is 256x256, but still have the same error.

and question for simswap, from the demo results for SimSwap, looks like there was no desruptions, does that mean it is not effective to face swapping ?

Thank you

[ZOMIN28]

I am honored that you are interested in our work. Regarding your two questions:

(1) If you can provide more error details, it will be helpful for debugging, such as the error location. In fact, we provide a parameter process in the Image2tensor() method. Setting it to True can automatically transform the shape to adapt to the model.

(2) Regarding the interruption of face swapping (SimSwap in our work), we have a detailed introduction in Section IV-A-2) of the paper: "For face swapping, we employ SimSwap as the target model. Given the importance of protecting personal identity, we aim to protect source facial images. Each source face is swapped onto five different target images." This means that our defense goal is to make the forged image unrecognizable from the source face. So, as you asked, why is the result like this? We discussed this issue in detail in Section IV-B of the paper: "This is because the perturbations used to protect the source image focus on perturbing the facial feature extractor." The destruction against the facial feature extractor directly leads to the disruption of high-level facial identity information.

[blancozhang]

Thank you so much for your timely reply! below is the errror:

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) Cell In[77], line 21 18 img_att = processorg_simswap()[0] 19 save_image(denorm(adv_attack(ori_img,img_att=img_att,model=simG,device=device,faketype="simswap",ComG=ComG,ComG_woj=ComG_woj,comgan=False)), 20 "data/images/adv_256_Sim/"+images_list[2]) ---> 21 save_image(denorm(adv_attack(ori_img,img_att=img_att,model=simG,device=device,faketype="simswap",ComG=ComG,ComG_woj=ComG_woj,comgan=True,balance=1.0)), 22 "data/images/adv_256_Sim/"+images_list[3]) File c:\Users\zclxs\Desktop\york_project\artifact\DF_RAP-main\attack.py:170, in adv_attack(img, ComG, ComG_woj, model, device, epsilon, c_trg, img_att, faketype, comgan, balance) 168 x_real = img 169 gen_noattack = simswap_fake(img_att,x_real,model) --> 170 *,perturb = pgdattack.perturb(X_nat=img,y=gen_noattack,img_att=img_att,faketype=faketype,comgan=comgan) 171 X = img + perturb 174 return X File c:\Users\zclxs\Desktop\york_project\artifact\DF_RAP-main\attack.py:126, in LinfPGDAttack.perturb(self, X_nat, y, c_trg, img_att, latend_id, faketype, comgan) 124 self.ComG.zero_grad() 125 img_id_downsample1 = F.interpolate(self.ComG(X), size=(112,112)) --> 126 latend_id1 = self.model.netArc(img_id_downsample1) 127 latend_id1 = latend_id1 / torch.norm(latend_id1, p=2, dim=1, keepdim=True) 128 output1 = self.model(self.ComG(X), img_att, latend_id1, latend_id1, True) File ~\AppData\Roaming\Python\Python312\site-packages\torch\nn\modules\module.py:1553, in Module._wrapped_call_impl(self, args, *kwargs) 1551 return self._compiled_call_impl(args, kwargs) # type: ignore[misc] --> 126 x = x.view(x.size(0), -1) 127 x = self.fc(x) 128 x = self.bn3(x) RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.**

I tried set process = True, but still have the same error.

I will definitely take a deep dive in your paper!

[ZOMIN28]

I got it. You seem to meet the same problem at https://github.com/ZOMIN28/DF_RAP/issues/3. This might have been caused by the version of Arc_face. You can temporarily solve your problem according to the solution in this question. Of course, I will update the provided arc_face parameter file as soon as possible to completely solve this problem. Please pay attention to our updates.

[blancozhang]

yes I did modified the code according to that post but still came across this error =-=

[blancozhang]

also if I run this directly:

ComG_model = torch.load('checkpoints/ComGAN/ComG_model.pt')
ComG = ComG_model['ComG'].to(device)
ComG.eval()
ComG_woj = ComG_model['ComG_woj'].to(device)
ComG_woj.eval()

starG = stargan_model()
opt = TestOptions().parse()
simG = simswap_model(opt)`

I will have this error:

usage: ipykernel_launcher.py [-h]
ipykernel_launcher.py: error: unrecognized arguments: --f=c:\Users\zclxs\AppData\Roaming\jupyter\runtime\kernel-v32f9a890a9de87c7e375ffc2a0ef8c9e6d9c37571.json
An exception has occurred, use %tb to see the full traceback.

so instead I added this before runnning:

import os
from types import SimpleNamespace
opt = SimpleNamespace(
    # Experiment setup
    name='people',
    gpu_ids='0',
    checkpoints_dir="./SimSwap/checkpoints",

    # Normalization and other options
    norm='batch',
    use_dropout=False,
    data_type=32,
    verbose=False,
    fp16=False,
    local_rank=0,
    isTrain=False,  # Changed to False because the experiment is for testing

    # Input/output sizes
    batchSize=8,
    loadSize=1024,
    fineSize=512,
    label_nc=0,
    input_nc=3,
    output_nc=3,

    # Input data settings
    dataroot='./datasets/cityscapes/',
    resize_or_crop='scale_width',
    serial_batches=False,
    no_flip=False,
    nThreads=2,
    max_dataset_size=float("inf"),

    # Display settings
    display_winsize=512,
    tf_log=False,

    # Generator settings
    netG='global',
    latent_size=512,
    ngf=64,
    n_downsample_global=3,
    n_blocks_global=6,
    n_blocks_local=3,
    n_local_enhancers=1,
    niter_fix_global=0,

    # Instance-wise feature options
    no_instance=False,
    instance_feat=False,
    label_feat=False,
    feat_num=3,
    load_features=False,
    n_downsample_E=4,
    nef=16,
    n_clusters=10,
    image_size=224,
    norm_G='spectralspadesyncbatch3x3',
    semantic_nc=3,

    # Additional attributes specific to your use case
    ntest=float("inf"),
    results_dir='./results/',
    aspect_ratio=1.0,
    phase='test',
    which_epoch='latest',
    how_many=50,
    cluster_path='features_clustered_010.npy',
    use_encoded_image=False,
    export_onnx=None,
    engine=None,
    onnx=None,
    Arc_path=os.path.join("./", 'SimSwap', 'arcface_model', 'arcface_checkpoint.tar'),
    pic_a_path='faceswap_dataset/256_source/0000.jpg',
    pic_b_path='faceswap_dataset/256_source/0001.jpg',
    pic_specific_path='faceswap_dataset/source/0000.jpg',
    multisepcific_dir='faceswap_dataset/source',
    video_path='path/to/video.mp4',
    temp_path='./temp_results',
    output_path='./output/',
    id_thres=0.03,
    no_simswaplogo=False,
    use_mask=False,
    crop_size=224
)

[blancozhang]

i try change back to the img used in the demo, actually it is not working either for the "defending against deepfakes" part.

RuntimeError                              Traceback (most recent call last)
Cell In[106], [line 14](vscode-notebook-cell:?execution_count=106&line=14)
     [11](vscode-notebook-cell:?execution_count=106&line=11) img_att = processorg_simswap()[0]
     [12](vscode-notebook-cell:?execution_count=106&line=12) save_image(denorm(adv_attack(ori_img,img_att=img_att,model=simG,device=device,faketype="simswap",ComG=ComG,ComG_woj=ComG_woj,comgan=False)),
     [13](vscode-notebook-cell:?execution_count=106&line=13)            "data/images/adv/"+images_list[2])
---> [14](vscode-notebook-cell:?execution_count=106&line=14) save_image(denorm(adv_attack(ori_img,img_att=img_att,model=simG,device=device,faketype="simswap",ComG=ComG,ComG_woj=ComG_woj,comgan=True,balance=1.0)),
     [15](vscode-notebook-cell:?execution_count=106&line=15)            "data/images/adv/"+images_list[3])

File c:\Users\zclxs\Desktop\york_project\artifact\DF_RAP-main\attack.py:170, in adv_attack(img, ComG, ComG_woj, model, device, epsilon, c_trg, img_att, faketype, comgan, balance)
    [168](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:168)         x_real = img
    [169](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:169)         gen_noattack = simswap_fake(img_att,x_real,model)
--> [170](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:170)     _,perturb = pgd_attack.perturb(X_nat=img,y=gen_noattack,img_att=img_att,faketype=faketype,comgan=comgan)
    [171](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:171)     X = img + perturb
    [174](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:174) return X

File c:\Users\zclxs\Desktop\york_project\artifact\DF_RAP-main\attack.py:126, in LinfPGDAttack.perturb(self, X_nat, y, c_trg, img_att, latend_id, faketype, comgan)
    [124](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:124) self.ComG.zero_grad()
    [125](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:125) img_id_downsample1 = F.interpolate(self.ComG(X), size=(112,112))
--> [126](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:126) latend_id1 = self.model.netArc(img_id_downsample1)
    [127](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:127) latend_id1 = latend_id1 / torch.norm(latend_id1, p=2, dim=1, keepdim=True)
    [128](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/attack.py:128) output1 = self.model(self.ComG(X), img_att, latend_id1, latend_id1, True)

File ~\AppData\Roaming\Python\Python312\site-packages\torch\nn\modules\module.py:1553, in Module._wrapped_call_impl(self, *args, **kwargs)
   [1551](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/~/AppData/Roaming/Python/Python312/site-packages/torch/nn/modules/module.py:1551)     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
...
--> [126](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/SimSwap/models/arcface_models.py:126) x = x.view(x.size(0), -1)
    [127](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/SimSwap/models/arcface_models.py:127) x = self.fc(x)
    [128](file:///C:/Users/zclxs/Desktop/york_project/artifact/DF_RAP-main/SimSwap/models/arcface_models.py:128) x = self.bn3(x)

RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.    

so I changed the code to this, using the data I collected, and they have been resized to 256x256.

ori_img = Image2tensor("faceswap_dataset/256_source/0000.jpg",process= True)
print(f"Original image shape: {ori_img.shape}")

c_trg = torch.tensor([[1.,0.,0.,1.,1.]]).to(device)
images_list = ["stargan_none_adv.png","stargan_com_adv.png","simswap_none_adv.png","simswap_com_adv.png"]

output_dir_stargan = "data/images/adv_256_StarGan/"
output_dir_simswap = "data/images/adv_256_Sim/"
os.makedirs(output_dir_stargan, exist_ok=True)
os.makedirs(output_dir_simswap, exist_ok=True)

# StarGAN attacks
for i in range(2):
    comgan = i == 1
    result = adv_attack(ori_img, c_trg=c_trg, model=starG, device=device, faketype="StarGAN", ComG=ComG, ComG_woj=ComG_woj, comgan=comgan, balance=1.0)
    print(f"StarGAN attack result shape (comgan={comgan}): {result.shape}")
    save_image(denorm(result), f"{output_dir_stargan}{images_list[i]}")

# SimSwap attacks
img_att = processorg_simswap()[0]
print(f"img_att shape: {img_att.shape}")

for i in range(2):
    comgan = i == 1
    print(f"\nStarting SimSwap attack with comgan={comgan}")
    try:
        result = adv_attack(ori_img, img_att=img_att, model=simG, device=device, faketype="simswap", ComG=ComG, ComG_woj=ComG_woj, comgan=comgan, balance=1.0)
        print(f"SimSwap attack result shape (comgan={comgan}): {result.shape}")
        save_image(denorm(result), f"{output_dir_simswap}{images_list[i+2]}")
    except Exception as e:
        print(f"Error occurred during SimSwap attack (comgan={comgan}):")
        print(str(e))
        print("Tensor shapes:")
        print(f"ori_img shape: {ori_img.shape}")
        print(f"img_att shape: {img_att.shape}")
        if 'ComG' in locals():
            print(f"ComG output shape: {ComG(ori_img).shape}")
        if 'ComG_woj' in locals():
            print(f"ComG_woj output shape: {ComG_woj(ori_img).shape}")

        # Additional debugging for netArc
        try:
            netArc_input = ComG(ori_img) if comgan else ori_img
            netArc_input_resized = F.interpolate(netArc_input, size=(112, 112), mode='bilinear', align_corners=False)
            print(f"netArc input shape (after resize): {netArc_input_resized.shape}")
            netArc_output = simG.netArc(netArc_input_resized)
            print(f"netArc output shape: {netArc_output.shape}")
        except Exception as netArc_error:
            print(f"Error in netArc: {str(netArc_error)}")    

this would give me this information:

Original image shape: torch.Size([1, 3, 256, 256])
StarGAN attack result shape (comgan=False): torch.Size([1, 3, 256, 256])
StarGAN attack result shape (comgan=True): torch.Size([1, 3, 256, 256])
img_att shape: torch.Size([1, 3, 256, 256])

Starting SimSwap attack with comgan=False
SimSwap attack result shape (comgan=False): torch.Size([1, 3, 256, 256])

Starting SimSwap attack with comgan=True
Error occurred during SimSwap attack (comgan=True):
view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
Tensor shapes:
ori_img shape: torch.Size([1, 3, 256, 256])
img_att shape: torch.Size([1, 3, 256, 256])
ComG output shape: torch.Size([1, 3, 256, 256])
ComG_woj output shape: torch.Size([1, 3, 256, 256])
netArc input shape (after resize): torch.Size([1, 3, 112, 112])
Error in netArc: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

which indicates that whenever comgan = True there will be tensor size mismatch

[ZOMIN28]

We have updated the parameter files for SimSwap (of course, Arcface is also included). You can download them from SimSwap old and put them in the corresponding location according to the README.