search

hcmlab / GANonymization

A GAN-based Face Anonymization Framework for Preserving Emotional Expressions
https://hcmlab.github.io/GANonymization/
MIT License
12 stars 6 forks source link

Running in Google Colab with Python 3.10: `libpng error: Invalid IHDR data` #4

Closed Gordon-4389 closed 7 months ago

Gordon-4389 commented 7 months ago

Hi there,

I am using your project for an undergrad thesis project, and I've ran into some issues when trying to use your pre-trained model (25 Epoch Model) for anonymization in Google Colab.

Although I've had some issues with dependencies, I've been able to resolve them and run the model in Google Colab, but after your update a couple of weeks ago, I ran into the following issue:

2024-02-25 09:50:16.311600: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-02-25 09:50:16.311665: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-02-25 09:50:16.313114: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-02-25 09:50:17.637995: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-02-25 09:50:20.330596: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:47] Overriding orig_value setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
libpng warning: Invalid image width in IHDR
libpng warning: Image width exceeds user limit in IHDR
libpng warning: Invalid image height in IHDR
libpng warning: Image height exceeds user limit in IHDR
libpng error: Invalid IHDR data

The only things I have changed to install your code in Google Colab are:

  1. Commenting out 'git+https://github.com/wiktorlazarski/head-segmentation@main' from requirements.txt and installing it separately as a pip package
  2. Changing 'statsmodels==0.13.5' to 'statsmodels>=0.14.0' in requirements.txt

Do you happen to know how I can resolve this error? Specifically, what may have changed to induce the libpng error? If required, I can provide the Colab notebook for further information.

Kind Regards

Gordon-4389 commented 7 months ago

Hi @FHellmann,

I would like to ask for some assistance regarding this issue. Do you require anything for me send to reproduce this issue?

FHellmann commented 7 months ago

Hi, what image size do you use as input? It seems that the image is too big:

libpng warning: Invalid image width in IHDR
libpng warning: Image width exceeds user limit in IHDR
libpng warning: Invalid image height in IHDR
libpng warning: Image height exceeds user limit in IHDR

For further assistance, I need a more precise error message. Otherwise, I can only guess what the error might cause.

Gordon-4389 commented 7 months ago

Hi @FHellmann,

Thanks for responding. My input image dimensions are 256x256 pixels, as I am trying to run the Anonymisation model on the FDF256 dataset from DeepPrivacy2.

I was running the following code in the Colab notebook:

!python /content/GANonymization/main.py anonymize_image --model_file /content/GANonymization_Data/GANonymization_25.ckpt --input_file /content/GANonymization_Data/fdf256_GANonymization/000032.png --output_file /content/anon_000032.png

This was my exact output:

2024-02-25 09:50:16.311600: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-02-25 09:50:16.311665: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-02-25 09:50:16.313114: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-02-25 09:50:17.637995: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-02-25 09:50:20.330596: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:47] Overriding orig_value setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
libpng warning: Invalid image width in IHDR
libpng warning: Image width exceeds user limit in IHDR
libpng warning: Invalid image height in IHDR
libpng warning: Image height exceeds user limit in IHDR
libpng error: Invalid IHDR data

The first set of warnings/errors appear to be issue with CUDA on Colab, but I've gotten that error before and I was able to run Face Anonymisation properly.

If needed, I can email you the link to my Colab Notebook.

Also, I want to replicate your results with the Cosine Distance metric, is it possible for you to upload the evaluation models?

Regards

FHellmann commented 7 months ago

You can run the code with the BugFix (#5) I just committed.

Here's an example of the code to run in Google Colab. When opening a new Notebook in Google Colab, go to Menu -> Runtime -> Change Runtime Type -> select a GPU.

!git clone https://github.com/hcmlab/GANonymization.git
!pip install git+https://github.com/hcmlab/GANonymization

import os
os.makedirs("/content/data", exist_ok=True)
os.makedirs("/content/model", exist_ok=True)

from urllib.request import urlretrieve
urlretrieve("https://mediastore.rz.uni-augsburg.de/get/NsLjQYey65/", "/content/model/GANonymization_25.ckpt")
urlretrieve("https://upload.wikimedia.org/wikipedia/commons/8/8d/President_Barack_Obama.jpg", "/content/data/President_Barack_Obama.jpg")

!python /content/GANonymization/main.py anonymize_image --model_file /content/model/GANonymization_25.ckpt --input_file /content/data/President_Barack_Obama.jpg --output_file /content/output/President_Barack_Obama.jpg
Gordon-4389 commented 7 months ago

Hi @FHellmann,

Thank you very much addressing this issue. I've ran the code and it's working, but there are a couple of slight issues/questions:

  1. When running !pip install git+https://github.com/hcmlab/GANonymization, I am getting the following dependency conflict: 

    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
plotnine 0.12.4 requires statsmodels>=0.14.0, but you have statsmodels 0.13.5 which is incompatible.

    which was why I changed statsmodels to >=0.14.0 in my initial code. Is there a reason why version 0.13.5 is used over version 0.14.0 as I do not really see any meaningful difference between the versions?

  2. I am running the Colab Notebook on the v100 GPU runtime, but I am still getting the warnings:

    2024-03-13 00:39:56.376005: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-03-13 00:39:56.376053: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-03-13 00:39:56.377429: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-03-13 00:39:58.043849: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-03-13 00:40:01.086600: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:47] Overriding orig_value setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

    If you are not getting this message, do you mind if I ask what GPU you might be using?

(EDIT: Final Question)

  1. Finally, is there a limit on directory size? When running on a folder of 2400 samples: 
    !python /content/GANonymization/main.py anonymize_directory --model_file /content/model/GANonymization_25.ckpt --input_directory /content/GANonymization_Data/fdf256_GANonymization --output_directory /content/FDF_25_results

    I get the following error:

    Running Pretrained Model with 25 epochs
mkdir: cannot create directory ‘FDF_25_results’: File exists
2024-03-13 00:58:27.894110: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-03-13 00:58:27.894165: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-03-13 00:58:27.896212: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-03-13 00:58:29.752533: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Anonymizing from /content/GANonymization_Data/fdf256_GANonymization:   0% 0/2400 [00:00<?, ?it/s]2024-03-13 00:58:32.936643: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:47] Overriding orig_value setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Anonymizing from /content/GANonymization_Data/fdf256_GANonymization:   0% 12/2400 [00:25<1:25:12,  2.14s/it]
Traceback (most recent call last):
File "/content/GANonymization/main.py", line 202, in <module>
fire.Fire()
File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 143, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 477, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 693, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/content/GANonymization/main.py", line 198, in anonymize_directory
anonymize_image(model_file, input_file, output_file, img_size, align, device)
File "/content/GANonymization/main.py", line 173, in anonymize_image
img = FaceCrop(align)(img)[0]
IndexError: list index out of range

    I saw the IndexError and assumed that it might have been the fault of the directory size, but I now think that it may be a result of not finding the face. Please advise.

Regards

FHellmann commented 7 months ago
  1. I also received the error message. After checking this package's usage, I could remove it from the requirements list.
  2. I also Receive errors and warnings regarding CUDA. However, I'm unsure how to fix it, as it is only an issue with Google Colab.
  3. This issue occurred because no face was detected in an image. I created a bugfix (#6) to fix this issue and make it possible to anonymize all detected faces in an image. However, falsely detected faces can also be anonymized; accordingly, the resulting image looks thoroughly off.
Gordon-4389 commented 7 months ago

Much appreciated for addressing these issues. I will be adjusting the dataset used to ensure that there are faces within the images, and will be closing this issue.

Thank you very much.