serengil / deepface

A Lightweight Face Recognition and Facial Attribute Analysis (Age, Gender, Emotion and Race) Library for Python
https://www.youtube.com/watch?v=WnUVYQP4h44&list=PLsS_1RYmYQQFdWqxQggXHynP1rqaYXv_E&index=1
MIT License
10.85k stars 1.91k forks source link

[BUG]: Alignment process may not always work as expected #1244

Closed PhilippeRo closed 1 month ago

PhilippeRo commented 1 month ago

Before You Report a Bug, Please Confirm You Have Done The Following...

DeepFace's version

0.0.91

Python version

3.11

Operating System

Fedora 40

Dependencies

absl-py==2.1.0 aiohttp==3.9.1 aiosignal==1.3.1 aiostream==0.5.2 annotated-types==0.6.0 anyio==3.7.1 appdirs==1.4.4 argcomplete==3.3.0 astunparse==1.6.3 attrs==23.1.0 Authlib==1.3.0 Babel==2.15.0 backoff==2.2.1 bcrypt==4.0.1 Beaker==1.12.1 beautifulsoup4==4.12.3 black==23.11.0 blinker==1.7.0 blivet==3.10.0 blivet-gui==2.5.0 bodhi-client==8.1.0 boto3==1.34.99 botocore==1.34.99 bottle==0.12.25 Brlapi==0.8.5 Brotli==1.1.0 cachetools==5.3.2 CCColUtils==1.5 certifi==2023.11.17 cffi==1.16.0 chardet==5.2.0 charset-normalizer==3.3.2 click==8.1.7 colorama==0.4.6 construct==2.10.70 contourpy==1.2.0 cryptography==41.0.7 cupshelpers==1.0 cycler==0.11.0 dasbus==1.7 dataclasses-json==0.6.3 dbus-python==1.3.2 decorator==5.1.1 deepface==0.0.90 defusedxml==0.7.1 Deprecated==1.2.14 diskcache==5.6.3 distro==1.9.0 dm-tree==0.1.8 dnf==4.19.2 docutils==0.20.1 dropbox==11.36.2 duplicity==2.2.3 fastapi==0.104.1 fasteners==0.19 fedora-third-party==0.10 fedpkg==1.44 file-magic==0.4.0 filelock==3.13.1 fire==0.5.0 Flask==3.0.2 flatbuffers==23.5.26 fonttools==4.50.0 fros==1.1 frozenlist==1.4.0 fs==2.4.16 fsspec==2023.12.1 gast==0.5.4 gbinder-python==1.1.2 gdown==5.1.0 ghp-import==2.1.0 gitdb==4.0.10 GitPython==3.1.40 google-api-core==2.11.1 google-api-python-client==2.120.0 google-auth==2.29.0 google-auth-httplib2==0.2.0 google-pasta==0.2.0 googleapis-common-protos==1.63.0 Grammalecte-fr==2.1.2 greenlet==3.0.2 griffe==0.38.0 grpcio==1.62.0 grpcio-status==1.48.4 gssapi==1.7.3 gunicorn==21.2.0 h11==0.14.0 h5py==3.10.0 httpcore==1.0.2 httplib2==0.21.0 httpx==0.25.2 humanize==3.13.1 idna==3.7 importlib-metadata==6.8.0 iniconfig==2.0.0 iso639==0.1.4 itsdangerous==2.1.2 jaraco.classes==3.3.0 jedi==0.19.1 jeepney==0.8.0 Jinja2==3.1.4 jmespath==1.0.1 joblib==1.3.2 keras==3.0.5 kerberos==1.3.0 keyring==24.3.0 kitchen==1.2.6 kiwisolver==1.4.5 koji==1.34.0 krb5==0.5.1 langtable==0.0.66 libclang==16.0.6 libcomps==0.1.20 libdnf==0.73.1 llama-index==0.9.13 llama_cpp_python==0.2.25 lockfile==0.12.2 louis==3.28.0 lxml==5.1.0 Mako==1.2.3 Markdown==3.5.2 markdown-it-py==3.0.0 MarkupSafe==2.1.3 marshmallow==3.20.1 matplotlib==3.8.4 mdurl==0.1.2 mergedeep==1.3.4 meson==1.4.0 mkdocs==1.5.3 mkdocs-autorefs==0.5.0 mkdocs-material==9.4.12 mkdocs-material-extensions==1.3.1 mkdocstrings==0.24.0 mkdocstrings-python==1.7.5 ml-dtypes==0.3.2 more-itertools==10.1.0 mtcnn==0.1.1 multidict==6.0.4 munch==2.5.0 mutagen==1.47.0 mypy-extensions==1.0.0 namex==0.0.7 nest-asyncio==1.5.8 nftables==0.1 nh3==0.2.14 nltk==3.8.1 numpy==1.26.2 oauth2client==4.1.3 olefile==0.47 openai==1.3.8 opencv-python==4.9.0.80 openidc-client==0.7.0 opt-einsum==3.3.0 packaging==23.2 paginate==0.5.6 pandas==2.1.4 paramiko==3.4.0 parso==0.8.3 Paste==3.7.1 pathspec==0.11.2 pexpect==4.9.0 pid==2.2.3 pillow==10.3.0 pkginfo==1.9.6 platformdirs==4.0.0 pluggy==1.3.0 ply==3.11 pooch==1.8.1 productmd==1.38 progressbar2==3.53.2 protobuf==4.25.3 ptyprocess==0.7.0 pwquality==1.4.5 pyalsa==1.2.7 pyasn1==0.5.1 pyasn1-modules==0.3.0 pybeam==0.7 pycairo==1.25.1 pyclip==0.7.0 pycparser==2.20 pycrypto==2.6.1 pycryptodomex==3.20.0 pycups==2.0.4 pycurl==7.45.2 pydantic==2.5.2 pydantic-settings==2.1.0 pydantic_core==2.14.5 PyDrive2==1.19.0 pyenchant==3.2.2 pygit2==1.14.0 Pygments==2.17.2 PyGObject==3.48.2 pyinotify==0.9.6 pykickstart==3.52 pymdown-extensions==10.4 PyNaCl==1.5.0 pyOpenSSL==23.2.0 pyparsing==3.1.2 pyparted==3.13.0 pyroute2==0.7.3 PySocks==1.7.1 pyspnego==0.10.2 pytest==7.4.3 python-augeas==1.1.0 python-bugzilla==3.2.0 python-dateutil==2.8.2 python-dotenv==1.0.0 python-fedora==1.1.1 python-meh==0.51 python-pam==2.0.2 python-utils==3.7.0 pytz==2024.1 pyudev==0.24.1 pyxdg==0.27 PyYAML==6.0.1 pyyaml_env_tag==0.1 readme-renderer==42.0 regex==2024.4.16 requests==2.31.0 requests-file==2.0.0 requests-ftp==0.3.1 requests-gssapi==1.2.3 requests-kerberos==0.14.0 requests-toolbelt==1.0.0 retina-face==0.0.14 rfc3986==2.0.0 rich==13.7.0 rpkg==1.66 rpm==4.19.1.1 rpmautospec==0.6.3 rpmautospec-core==0.1.4 rpmconf==1.1.10 rpmlint==2.5.0 rsa==4.9 s3transfer==0.10.1 SciPy==1.11.3 SecretStorage==3.3.3 selinux @ file:///builddir/build/BUILD/libselinux-3.6/src sepolicy @ file:///builddir/build/BUILD/selinux-3.6/python/sepolicy setools==4.5.1 setuptools==69.0.3 simpleaudio==1.0.4 simpleline==1.9.0 six==1.16.0 smmap==5.0.0 sniffio==1.3.0 sos==4.7.1 soupsieve==2.5 SQLAlchemy==2.0.23 sse-starlette==1.8.2 starlette==0.27.0 starlette-context==0.3.6 stone==3.2.1 systemd-python==235 Tempita==0.5.2 templated-dictionary==1.4 tenacity==8.2.3 tensorboard==2.16.2 tensorboard-data-server==0.7.2 tensorflow==2.16.0rc0 termcolor==2.4.0 tiktoken==0.5.2 tomli_w==1.0.0 tqdm==4.66.1 twine==4.0.2 typing-inspect==0.9.0 typing_extensions==4.9.0 tzdata==2023.3 uritemplate==4.1.1 urllib3==1.26.18 uvicorn==0.24.0.post1 watchdog==3.0.0 websockets==12.0 Werkzeug==3.0.1 wheel==0.42.0 wrapt==1.16.0 xkbregistry==0.3 yarl==1.9.4 yt-dlp==2024.5.27 zipp==3.17.0 zstandard==0.22.0

Reproducible example

Using the test image provided below, in python console try:

faces=DeepFace.extract_faces(img_path='Path to test image.png',detector_backend="retinaface")
plt.imshow(faces[1]["face"])
plt.show()

Note: the example uses faces[1] but it is even worse with faces[0].

Relevant Log Output

No response

Expected Result

This happens only with faces which are close to the upper boundary of an image and which are slightly tilted one way or another. I created the following example : test image

In this type of situation, we should get the whole face...

What happened instead?

But we only get the lower part of the face: face

Currently, after determining the proper rotation angle using the eyes' coordinates, the whole image is rotated then the coordinates of the face are updated using the same angle and finally the face is extracted from the rotated image. The problem is that after the rotation of the whole image the upper y coordinate of the face is negative.

It seems that, at least in this case (when a face is close to the upper boundary of the image and tilted), the face should be first extracted then rotated by the correct angle. But I may be wrong about the right solution.

Additional Info

No response

serengil commented 1 month ago

good catch! will sort this soon.