search

tomas-gajarsky / facetorch

Python library for analysing faces using PyTorch
Apache License 2.0
503 stars 45 forks source link

Stack on AU inference when `return_img_data=True` #56

Closed sobassy closed 11 months ago

sobassy commented 11 months ago

Thank for exciting repository!

I tried to run example .ipynb on locally. First, warmup code run correctly.

response = analyzer.run(
        path_image=path_img_input,
        batch_size=cfg.batch_size,
        fix_img_size=cfg.fix_img_size,
        return_img_data=False,
        include_tensors=True,
        path_output=path_img_output,
    )
{"asctime": "2023-11-27 14:09:19,209", "levelname": "INFO", "message": "Running FaceAnalyzer"}
{"asctime": "2023-11-27 14:09:19,209", "levelname": "INFO", "message": "Reading image", "path_image": "./test.jpg"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:09:19,289", "levelname": "INFO", "message": "Detecting faces"}
{"asctime": "2023-11-27 14:09:21,654", "levelname": "INFO", "message": "Number of faces: 4"}
{"asctime": "2023-11-27 14:09:21,655", "levelname": "INFO", "message": "Unifying faces"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:09:21,679", "levelname": "INFO", "message": "Predicting facial features"}
{"asctime": "2023-11-27 14:09:21,679", "levelname": "INFO", "message": "Running FacePredictor: embed"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:09:21,816", "levelname": "INFO", "message": "Running FacePredictor: verify"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:09:30,575", "levelname": "INFO", "message": "Running FacePredictor: fer"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:09:30,835", "levelname": "INFO", "message": "Running FacePredictor: au"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:09:31,206", "levelname": "INFO", "message": "Running FacePredictor: deepfake"}
{"asctime": "2023-11-27 14:09:31,675", "levelname": "INFO", "message": "Running FacePredictor: align"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:09:31,966", "levelname": "INFO", "message": "Utilizing facial features"}
{"asctime": "2023-11-27 14:09:31,967", "levelname": "INFO", "message": "Running BaseUtilizer: align"}
{"asctime": "2023-11-27 14:09:31,976", "levelname": "INFO", "message": "Running BaseUtilizer: draw_boxes"}
{"asctime": "2023-11-27 14:09:32,005", "levelname": "INFO", "message": "Running BaseUtilizer: draw_landmarks"}

But, change return_img_data=False to return_img_data=True, hung up on AU inference.

# warmup
response = analyzer.run(
        path_image=path_img_input,
        batch_size=cfg.batch_size,
        fix_img_size=cfg.fix_img_size,
        return_img_data=True,
        include_tensors=True,
        path_output=path_img_output,
    )
{"asctime": "2023-11-27 14:23:03,506", "levelname": "INFO", "message": "Running FaceAnalyzer"}
{"asctime": "2023-11-27 14:23:03,506", "levelname": "INFO", "message": "Reading image", "path_image": "./test.jpg"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:23:03,541", "levelname": "INFO", "message": "Detecting faces"}
{"asctime": "2023-11-27 14:23:04,294", "levelname": "INFO", "message": "Number of faces: 4"}
{"asctime": "2023-11-27 14:23:04,294", "levelname": "INFO", "message": "Unifying faces"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:23:04,296", "levelname": "INFO", "message": "Predicting facial features"}
{"asctime": "2023-11-27 14:23:04,296", "levelname": "INFO", "message": "Running FacePredictor: embed"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:23:04,356", "levelname": "INFO", "message": "Running FacePredictor: verify"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:23:25,887", "levelname": "INFO", "message": "Running FacePredictor: fer"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(
{"asctime": "2023-11-27 14:23:28,681", "levelname": "INFO", "message": "Running FacePredictor: au"}
C:\Users\sobassy\.conda\envs\jupyter\lib\site-packages\torchvision\transforms\functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
  warnings.warn(

Is there a solution? or How to get verbose information on inference?

Verbose information

(jupyter) C:\Users\sobassy>pip list
Package                  Version
------------------------ --------------------
antlr4-python3-runtime   4.9.3
anyio                    3.5.0
appdirs                  1.4.4
argon2-cffi              21.3.0
argon2-cffi-bindings     21.2.0
asttokens                2.0.5
attrs                    22.1.0
Babel                    2.11.0
backcall                 0.2.0
beautifulsoup4           4.12.0
black                    23.9.1
bleach                   4.1.0
brotlipy                 0.7.0
certifi                  2022.12.7
cffi                     1.15.1
charset-normalizer       2.0.4
click                    8.1.7
codetiming               1.4.0
colorama                 0.4.6
comm                     0.1.2
contourpy                1.1.0
cryptography             39.0.1
cycler                   0.11.0
debugpy                  1.5.1
decorator                5.1.1
defusedxml               0.7.1
dill                     0.3.7
entrypoints              0.4
executing                0.8.3
facetorch                0.3.0
fastjsonschema           2.16.2
filelock                 3.12.2
flit_core                3.8.0
fonttools                4.40.0
fsspec                   2023.4.0
gdown                    4.7.1
gitdb                    4.0.10
GitPython                3.1.31
googleapis-common-protos 1.61.0
hydra-core               1.3.2
idna                     3.4
importlib-metadata       6.0.0
importlib-resources      5.12.0
ipykernel                6.19.2
ipython                  8.12.0
ipython-autotime         0.3.2
ipython-genutils         0.2.0
ipywidgets               8.0.4
isort                    5.12.0
jedi                     0.18.1
Jinja2                   3.1.2
joblib                   1.3.2
json5                    0.9.6
jsonschema               4.17.3
jupyter                  1.0.0
jupyter_client           8.1.0
jupyter-console          6.6.3
jupyter_core             5.3.0
jupyter-server           1.23.4
jupyterlab               3.5.3
jupyterlab-pygments      0.1.2
jupyterlab_server        2.22.0
jupyterlab-widgets       3.0.5
kiwisolver               1.4.4
lxml                     4.9.2
MarkupSafe               2.1.1
matplotlib               3.7.1
matplotlib-inline        0.1.6
mistune                  0.8.4
mkl-fft                  1.3.1
mkl-random               1.2.2
mkl-service              2.4.0
mpmath                   1.3.0
mypy-extensions          1.0.0
nbclassic                0.5.4
nbclient                 0.5.13
nbconvert                6.5.4
nbformat                 5.7.0
nest-asyncio             1.5.6
networkx                 3.1
notebook                 6.5.3
notebook_shim            0.2.2
numpy                    1.23.5
omegaconf                2.3.0
opencv-python            4.7.0.72
packaging                23.0
pandas                   2.0.2
pandocfilters            1.5.0
parso                    0.8.3
pathspec                 0.11.2
pbr                      6.0.0
pickleshare              0.7.5
Pillow                   9.5.0
pip                      23.0.1
platformdirs             3.11.0
ply                      3.11
pooch                    1.4.0
prometheus-client        0.14.1
prompt-toolkit           3.0.36
protobuf                 4.25.1
psutil                   5.9.0
pure-eval                0.2.2
pycparser                2.21
Pygments                 2.11.2
pylatexenc               2.10
pyOpenSSL                23.0.0
pyparsing                3.1.0
PyQt5                    5.15.7
PyQt5-sip                12.11.0
pyrsistent               0.18.0
PySocks                  1.7.1
python-dateutil          2.8.2
python-json-logger       2.0.7
pytz                     2022.7
pywin32                  305.1
pywinpty                 2.0.10
PyYAML                   6.0
pyzmq                    23.2.0
qiskit                   0.45.0
qiskit-aer               0.13.0
qiskit-terra             0.45.0
qtconsole                5.4.0
QtPy                     2.2.0
requests                 2.28.1
rustworkx                0.13.2
scikit-learn             1.3.1
scipy                    1.10.1
seaborn                  0.12.2
Send2Trash               1.8.0
setuptools               65.6.3
sip                      6.6.2
six                      1.16.0
smmap                    5.0.0
sniffio                  1.2.0
soupsieve                2.4
stack-data               0.2.0
stevedore                5.1.0
sympy                    1.12
terminado                0.17.1
thop                     0.1.1.post2209072238
threadpoolctl            3.2.0
tinycss2                 1.2.1
toml                     0.10.2
tomli                    2.0.1
torch                    2.1.1+cu118
torchaudio               2.1.1+cu118
torchvision              0.16.1+cu118
tornado                  6.2
tqdm                     4.65.0
traitlets                5.7.1
typing_extensions        4.8.0
tzdata                   2023.3
ultralytics              8.0.121
urllib3                  1.26.15
wcwidth                  0.2.5
webencodings             0.5.1
websocket-client         0.58.0
wheel                    0.38.4
widgetsnbextension       4.0.5
win-inet-pton            1.1.0
wincertstore             0.2
zipp                     3.11.0
import torch

print(torch.cuda.is_available())
print(torch.cuda.device_count())
print(torch.cuda.current_device())
print(torch.cuda.get_device_name(0))
True
2
0
NVIDIA GeForce RTX 3080

Thank, you.

wangyuchi369 commented 11 months ago

Hi, have you solved this problem? I met the same situation no matter whether True or False for return_img_data

tomas-gajarsky commented 11 months ago

I was able to reproduce this problem in the Colab notebook. It happens even if I disable all the other predictors, so only the AU model is problematic. However, the AU predictor did not get stuck when I ran it in the provided Docker container on the Nvidia RTX 3090. I thought that new versions of packages might be causing the problem, but when I updated them, it was still running just fine inside Docker.

The logging level can be adjusted in the logger section of the downloaded config file.

tomas-gajarsky commented 11 months ago

The bug was fixed by specifying the versions of the torch and torchvision packages for me. You should try to change the pip install command in the Install dependencies section as follows:

!python -m pip install "facetorch>=0.2.1" "torch==1.11.0+cu113" "torchvision==0.12.0+cu113" --extra-index-url https://download.pytorch.org/whl/cu113

sobassy commented 11 months ago

@tomas-gajarsky

Thanks for useful information and report! It worked well on CUDA v11.3 !!

I use Windows 11, but cuda11.3 is for Windows 10. but worked on Win11 :)

So, google Colab environment use cuda v11.3 or under? Maybe we should deploy local env yml (or requirements.txt) for user running locally.

Thanks very much for advice!

tomas-gajarsky commented 11 months ago

So, google Colab environment use cuda v11.3 or under? Maybe we should deploy local env yml (or requirements.txt) for user running locally.

Google Colab uses CUDA 12.0 as can be seen after executing the first nvidia-smi cell, so the problem was not only the CUDA version but perhaps the torch version as well.

The environment file and the lock file for conda are provided for reproducibility and show which packages were tested for the specific facetorch version. I recommend using the provided Docker images to ensure that facetorch runs, however, I know it is not possible for each use case.