search

garyfeng / Global-Flow-Local-Attention

The source code for paper "Deep Image Spatial Transformation for Person Image Generation"
https://renyurui.github.io/GFLA-web
Other
1 stars 0 forks source link

No Canny Edges as input #3

Open garyfeng opened 3 years ago

garyfeng commented 3 years ago

The current implement uses canny edges as input for inference. As implemented below, by default it first extracts the facial keypoints, then augments it with the canny features from the current frame. This second step only makes sense if the source image and the driving video are from the same scene. In the case they are different people and scenes, we shouldn't use canny features.

https://github.com/garyfeng/Global-Flow-Local-Attention/blob/a1f3b895f7ed26945c923e2fbf23d72ff2ba9001/data/face_dataset.py#L147-L160

So in the colab notebook file, we should disable the canny feature by starting with the --no_canny_edge parameter.

garyfeng commented 3 years ago

with the following parameter, 

!python demo.py \
--name=face_checkpoints \
--model=face \
--attn_layer=2,3 \
--kernel_size=2=5,3=3 \
--gpu_id=0 \
--no_canny_edge \
--dataset_mode=face \
--dataroot=./dataset/FaceForensics \
--results_dir=./demo_results/face

we get the following error message:

dataset [FaceDataset] of size 123 was created
val images = 123
Network [FaceGenerator] was created. Total number of parameters: 16.9067 million. To see the architecture, do print(network).
Network [ResDiscriminator] was created. Total number of parameters: 0.6646 million. To see the architecture, do print(network).
Network [ResDiscriminator] was created. Total number of parameters: 0.6655 million. To see the architecture, do print(network).
model resumed from latest iteration
Pretrained network G has excessive layers; Only loading layers that are used
Pretrained network D has fewer layers; The following are not initialized:
[]
Pretrained network D_V has fewer layers; The following are not initialized:
['block0']
model [Face] was created
Traceback (most recent call last):
  File "demo.py", line 18, in <module>
    for i, data in enumerate(dataset):
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 637, in __next__
    return self._process_next_batch(batch)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 658, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
TypeError: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 138, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/content/Global-Flow-Local-Attention/data/face_dataset.py", line 97, in __getitem__
    Ai, Li = self.get_face_image(A_path, transform_scaleA, transform_label, B_size, B_img)
  File "/content/Global-Flow-Local-Attention/data/face_dataset.py", line 163, in get_face_image
    label_tensor = transform_L(part_labels) * 255.0
  File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py", line 49, in __call__
    img = t(img)
  File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py", line 175, in __call__
    return F.resize(img, self.size, self.interpolation)
  File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/functional.py", line 189, in resize
    raise TypeError('img should be PIL Image. Got {}'.format(type(img)))
TypeError: img should be PIL Image. Got <class 'numpy.ndarray'>