search

JustinShenk / fer

Facial Expression Recognition with a deep neural network as a PyPI package
MIT License
326 stars 77 forks source link

Two issues - one with mtcnn one with face_rectangles #29

Closed DaddyWesker closed 2 years ago

DaddyWesker commented 2 years ago

Hello.

Thanks for your lib. I'm currently trying to use your fer to detect emotions on webcam video. I've installed your fer using pip install fer. But i've met two issues:

  1. When i'm initializing your detector as detector = FER(mtcnn=True) i'm getting AttributeError: module 'keras.utils.generic_utils' has no attribute 'to_snake_case' tensorflow 2.5.0, keras 2.4.3

  2. SInce i wanted only to use your model as emotion detector not face detector, i wanted to test your detector.detect_emotions with parameter face_rectangles, so code looks like this

    video_capture = cv2.VideoCapture(0)
detector = FER()
while True:
_, frame = video_capture.read() 
box = detector.find_faces(frame)
emotions  = detector.detect_emotions(frame, face_rectangles=box)

    So, i'm trying to use fer's find_faces to test how it performs with only emotion detect. Unfortunately, on the line emotions = detector.detect_emotions(frame, face_rectangles=box) i'm getting ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() on the line if not face_rectangles: in the file fer.py.

My understanding that it should be this if instead if (face_rectangles is None): With this if it works fine. Could you please change this line in your project so other people, installed fer with pip, could use it with their own face detect?

DaddyWesker commented 2 years ago

Oh, yet another problem.

  1. When using detector.top_emotion(frame) there is a chance that there won't be any face detected and you'll receive an error IndexError: list index out of range on the line top_emotion = top_emotions[0] in the file fer.py. Probably, you should add some sort of 'if' here?
JustinShenk commented 2 years ago

Thank you, first off, for the suggestions!

The first issue is related to https://github.com/ipazc/mtcnn/issues/114, which FER depends on. Try locking tensorflow to 2.4.1 and keras to 2.4.3 or this proposed solution: https://stackoverflow.com/a/67797729/6256888.

Regarding the second issue and third issues - good find! These have been fixed in v21.0.1. Please upgrade with pip install -U fer.

DaddyWesker commented 2 years ago

Upgraded fer. THanks

ABout first issue. I've downgraded tf to 2.4.1 (keras already is 2.4.3). Now i've got error


Traceback (most recent call last):
  File "/home/daddywesker/EmotionRecognition/FER/testing_fer.py", line 9, in <module>
    detector = FER(mtcnn=True)
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/fer/fer.py", line 98, in __init__
    from mtcnn.mtcnn import MTCNN
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/mtcnn/__init__.py", line 26, in <module>
    from mtcnn.mtcnn import MTCNN
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/mtcnn/mtcnn.py", line 37, in <module>
    from mtcnn.network.factory import NetworkFactory
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/mtcnn/network/factory.py", line 26, in <module>
    from keras.layers import Input, Dense, Conv2D, MaxPooling2D, PReLU, Flatten, Softmax
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/keras/__init__.py", line 20, in <module>
    from . import initializers
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/keras/initializers/__init__.py", line 124, in <module>
    populate_deserializable_objects()
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/keras/initializers/__init__.py", line 49, in populate_deserializable_objects
    LOCAL.GENERATED_WITH_V2 = tf.__internal__.tf2.enabled()
AttributeError: module 'tensorflow.compat.v2.__internal__' has no attribute 'tf2'

Process finished with exit code 1
JustinShenk commented 2 years ago

Can you report back if pip install tensorflow-estimator==2.1.0 fixes it? It appears related to https://github.com/tensorflow/tensorflow/issues/37525.

DaddyWesker commented 2 years ago

Well, i guess it wants me to further downgrade tensorflow?

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.4.1 requires tensorflow-estimator<2.5.0,>=2.4.0, but you have tensorflow-estimator 2.1.0 which is incompatible.
tensorflow-gpu 2.4.1 requires tensorflow-estimator<2.5.0,>=2.4.0, but you have tensorflow-estimator 2.1.0 which is incompatible.
JustinShenk commented 2 years ago

Otherwise trying this? https://exerror.com/module-tensorflow-compat-v2-internal-has-no-attribute-tf2/

DaddyWesker commented 2 years ago

Well, here is the problem. What the suggest 

!pip install -U -q segmentation-models
!pip install -q tensorflow==2.1
!pip install -q keras==2.3.1
!pip install -q tensorflow-estimator==2.1.

PRoblem - 

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
fer 21.0.1 requires keras==2.4.3, but you have keras 2.3.1 which is incompatible.

DOwngrading fer will return me to the the issues 2-3 =)

linguist89 commented 2 years ago

Hi all, @justinshenk Could you perhaps run pip freeze whichever machine you're running it on and post that along with which CUDA version, OS specs you have. I suspect it has something to do with the CUDA version and not the tensorflow version so much because I've noticed a lot of people are struggling with this now and I think it might be due to CUDA updates. I want to get the information for a system that you're running and try recreate it because it was working fine end of last year with the GPU, but now I can't get it to work either. I can detect my GPU fine and fer runs fine, it just cannot detect the GPU, so it's always using the CPU.

JustinShenk commented 2 years ago

Hey guys, some of these errors should have been fixed in the newly released mtcnn v0.1.1, now required in fer v21.0.2. This mtcnn version replaced all keras calls with tensorflow. Please pip install -U fer and let me know it goes.

@linguist89, I don't have a GPU in my computer unfortunately, so if you can't get the issue to show in a colab notebook, then hopefully others can help debug it.

JustinShenk commented 2 years ago

https://colab.research.google.com/github/justinshenk/fer/blob/master/fer-video-demo.ipynb

absl-py==0.12.0
alabaster==0.7.12
albumentations==0.1.12
altair==4.1.0
appdirs==1.4.4
argon2-cffi==20.1.0
arviz==0.11.2
astor==0.8.1
astropy==4.2.1
astunparse==1.6.3
async-generator==1.10
atari-py==0.2.9
atomicwrites==1.4.0
attrs==21.2.0
audioread==2.1.9
autograd==1.3
Babel==2.9.1
backcall==0.2.0
beautifulsoup4==4.6.3
bleach==3.3.0
blis==0.4.1
bokeh==2.3.2
Bottleneck==1.3.2
branca==0.4.2
bs4==0.0.1
CacheControl==0.12.6
cached-property==1.5.2
cachetools==4.2.2
catalogue==1.0.0
certifi==2021.5.30
cffi==1.14.5
cftime==1.5.0
chardet==3.0.4
click==7.1.2
cloudpickle==1.3.0
cmake==3.12.0
cmdstanpy==0.9.5
colorcet==2.0.6
colorlover==0.3.0
community==1.0.0b1
contextlib2==0.5.5
convertdate==2.3.2
coverage==3.7.1
coveralls==0.5
crcmod==1.7
cufflinks==0.17.3
cupy-cuda101==9.1.0
cvxopt==1.2.6
cvxpy==1.0.31
cycler==0.10.0
cymem==2.0.5
Cython==0.29.23
daft==0.0.4
dask==2.12.0
datascience==0.10.6
debugpy==1.0.0
decorator==4.4.2
defusedxml==0.7.1
descartes==1.1.0
dill==0.3.4
distributed==1.25.3
dlib==19.18.0
dm-tree==0.1.6
docopt==0.6.2
docutils==0.17.1
dopamine-rl==1.0.5
earthengine-api==0.1.272
easydict==1.9
ecos==2.0.7.post1
editdistance==0.5.3
en-core-web-sm==2.2.5
entrypoints==0.3
ephem==4.0.0.2
et-xmlfile==1.1.0
fa2==0.3.5
fastai==1.0.61
fastdtw==0.3.4
fastprogress==1.0.0
fastrlock==0.6
fbprophet==0.7.1
feather-format==0.4.1
fer==21.0.2
filelock==3.0.12
firebase-admin==4.4.0
fix-yahoo-finance==0.0.22
Flask==1.1.4
flatbuffers==1.12
folium==0.8.3
future==0.16.0
gast==0.4.0
GDAL==2.2.2
gdown==3.6.4
gensim==3.6.0
geographiclib==1.52
geopy==1.17.0
gin-config==0.4.0
glob2==0.7
google==2.0.3
google-api-core==1.26.3
google-api-python-client==1.12.8
google-auth==1.32.1
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.4
google-cloud-bigquery==1.21.0
google-cloud-bigquery-storage==1.1.0
google-cloud-core==1.0.3
google-cloud-datastore==1.8.0
google-cloud-firestore==1.7.0
google-cloud-language==1.2.0
google-cloud-storage==1.18.1
google-cloud-translate==1.5.0
google-colab==1.0.0
google-pasta==0.2.0
google-resumable-media==0.4.1
googleapis-common-protos==1.53.0
googledrivedownloader==0.4
graphviz==0.10.1
greenlet==1.1.0
grpcio==1.34.1
gspread==3.0.1
gspread-dataframe==3.0.8
gym==0.17.3
h5py==3.1.0
HeapDict==1.0.1
hijri-converter==2.1.3
holidays==0.10.5.2
holoviews==1.14.4
html5lib==1.0.1
httpimport==0.5.18
httplib2==0.17.4
httplib2shim==0.0.3
humanize==0.5.1
hyperopt==0.1.2
ideep4py==2.0.0.post3
idna==2.10
imageio==2.4.1
imagesize==1.2.0
imbalanced-learn==0.4.3
imblearn==0.0
imgaug==0.2.9
importlib-metadata==4.6.0
importlib-resources==5.2.0
imutils==0.5.4
inflect==2.1.0
iniconfig==1.1.1
install==1.3.4
intel-openmp==2021.3.0
intervaltree==2.1.0
ipykernel==4.10.1
ipython==5.5.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.6.3
itsdangerous==1.1.0
jax==0.2.13
jaxlib==0.1.66+cuda110
jdcal==1.4.1
jedi==0.18.0
jieba==0.42.1
Jinja2==2.11.3
joblib==1.0.1
jpeg4py==0.1.4
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.3.5
jupyter-console==5.2.0
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.0
kaggle==1.5.12
kapre==0.3.5
Keras==2.4.3
keras-nightly==2.5.0.dev2021032900
Keras-Preprocessing==1.1.2
keras-vis==0.4.1
kiwisolver==1.3.1
korean-lunar-calendar==0.2.1
librosa==0.8.1
lightgbm==2.2.3
llvmlite==0.34.0
lmdb==0.99
LunarCalendar==0.0.9
lxml==4.2.6
Markdown==3.3.4
MarkupSafe==2.0.1
matplotlib==3.2.2
matplotlib-inline==0.1.2
matplotlib-venn==0.11.6
missingno==0.4.2
mistune==0.8.4
mizani==0.6.0
mkl==2019.0
mlxtend==0.14.0
more-itertools==8.8.0
moviepy==0.2.3.5
mpmath==1.2.1
msgpack==1.0.2
mtcnn==0.1.1
multiprocess==0.70.12.2
multitasking==0.0.9
murmurhash==1.0.5
music21==5.5.0
natsort==5.5.0
nbclient==0.5.3
nbconvert==5.6.1
nbformat==5.1.3
nest-asyncio==1.5.1
netCDF4==1.5.7
networkx==2.5.1
nibabel==3.0.2
nltk==3.2.5
notebook==5.3.1
numba==0.51.2
numexpr==2.7.3
numpy==1.19.5
nvidia-ml-py3==7.352.0
oauth2client==4.1.3
oauthlib==3.1.1
okgrade==0.4.3
opencv-contrib-python==4.1.2.30
opencv-python==4.1.2.30
openpyxl==2.5.9
opt-einsum==3.3.0
osqp==0.6.2.post0
packaging==20.9
palettable==3.3.0
pandas==1.1.5
pandas-datareader==0.9.0
pandas-gbq==0.13.3
pandas-profiling==1.4.1
pandocfilters==1.4.3
panel==0.11.3
param==1.10.1
parso==0.8.2
pathlib==1.0.1
patsy==0.5.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==7.1.2
pip-tools==4.5.1
plac==1.1.3
plotly==4.4.1
plotnine==0.6.0
pluggy==0.7.1
pooch==1.4.0
portpicker==1.3.9
prefetch-generator==1.0.1
preshed==3.0.5
prettytable==2.1.0
progressbar2==3.38.0
prometheus-client==0.11.0
promise==2.3
prompt-toolkit==1.0.18
protobuf==3.17.3
psutil==5.4.8
psycopg2==2.7.6.1
ptyprocess==0.7.0
py==1.10.0
pyarrow==3.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycocotools==2.0.2
pycparser==2.20
pyct==0.4.8
pydata-google-auth==1.2.0
pydot==1.3.0
pydot-ng==2.0.0
pydotplus==2.0.2
PyDrive==1.3.1
pyemd==0.5.1
pyerfa==2.0.0
pyglet==1.5.0
Pygments==2.6.1
pygobject==3.26.1
pymc3==3.11.2
PyMeeus==0.5.11
pymongo==3.11.4
pymystem3==0.2.0
PyOpenGL==3.1.5
pyparsing==2.4.7
pyrsistent==0.18.0
pysndfile==1.3.8
PySocks==1.7.1
pystan==2.19.1.1
pytest==3.6.4
python-apt==0.0.0
python-chess==0.23.11
python-dateutil==2.8.1
python-louvain==0.15
python-slugify==5.0.2
python-utils==2.5.6
pytz==2018.9
pyviz-comms==2.1.0
PyWavelets==1.1.1
PyYAML==3.13
pyzmq==22.1.0
qdldl==0.1.5.post0
qtconsole==5.1.1
QtPy==1.9.0
regex==2019.12.20
requests==2.23.0
requests-oauthlib==1.3.0
resampy==0.2.2
retrying==1.3.3
rpy2==3.4.5
rsa==4.7.2
scikit-image==0.16.2
scikit-learn==0.22.2.post1
scipy==1.4.1
screen-resolution-extra==0.0.0
scs==2.1.4
seaborn==0.11.1
semver==2.13.0
Send2Trash==1.7.1
setuptools-git==1.2
Shapely==1.7.1
simplegeneric==0.8.1
six==1.15.0
sklearn==0.0
sklearn-pandas==1.8.0
smart-open==5.1.0
snowballstemmer==2.1.0
sortedcontainers==2.4.0
SoundFile==0.10.3.post1
spacy==2.2.4
Sphinx==1.8.5
sphinxcontrib-serializinghtml==1.1.5
sphinxcontrib-websupport==1.2.4
SQLAlchemy==1.4.20
sqlparse==0.4.1
srsly==1.0.5
statsmodels==0.10.2
sympy==1.7.1
tables==3.4.4
tabulate==0.8.9
tblib==1.7.0
tensorboard==2.5.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.5.0
tensorflow-datasets==4.0.1
tensorflow-estimator==2.5.0
tensorflow-gcs-config==2.5.0
tensorflow-hub==0.12.0
tensorflow-metadata==1.1.0
tensorflow-probability==0.13.0
termcolor==1.1.0
terminado==0.10.1
testpath==0.5.0
text-unidecode==1.3
textblob==0.15.3
Theano-PyMC==1.1.2
thinc==7.4.0
tifffile==2021.7.2
toml==0.10.2
toolz==0.11.1
torch==1.9.0+cu102
torchsummary==1.5.1
torchtext==0.10.0
torchvision==0.10.0+cu102
tornado==5.1.1
tqdm==4.41.1
traitlets==5.0.5
tweepy==3.10.0
typeguard==2.7.1
typing-extensions==3.7.4.3
tzlocal==1.5.1
uritemplate==3.0.1
urllib3==1.24.3
vega-datasets==0.9.0
wasabi==0.8.2
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wordcloud==1.5.0
wrapt==1.12.1
xarray==0.18.2
xgboost==0.90
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yellowbrick==0.9.1
zict==2.0.0
zipp==3.4.1

nvcc --version:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0

!nvidia-smi: NVIDIA-SMI 470.42.01 Driver Version: 460.32.03 CUDA Version: 11.2

DaddyWesker commented 2 years ago

Hey guys, some of these errors should have been fixed in the newly released mtcnn v0.1.1, now required in fer v21.0.2. This mtcnn version replaced all keras calls with tensorflow. Please pip install -U fer and let me know it goes.

Hello. I've updated mtcnn and fer and after that i'm getting this


Traceback (most recent call last):
  File "/home/daddywesker/EmotionRecognition/FER/testing_fer.py", line 7, in <module>
    detector = FER(mtcnn=True)
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/fer/fer.py", line 120, in __init__
    self.__emotion_classifier = load_model(emotion_model, compile=compile)
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/save.py", line 146, in load_model
    return hdf5_format.load_model_from_hdf5(filepath, custom_objects, compile)
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/hdf5_format.py", line 166, in load_model_from_hdf5
    model_config = json.loads(model_config.decode('utf-8'))
AttributeError: 'str' object has no attribute 'decode'

Process finished with exit code 1

Deleting .decode('utf-8') part helps but i'm editing keras code this way. And there are some more places with such a problem. DOn't know if my solution is right.

JustinShenk commented 2 years ago

The issue is caused by having h5py>=3.0.0 with older version TensorFlow (2.3 or below). TensorFlow 2.3 and below require h5py==2.10.0. So either try pip install -U tensorflow or 

pip uninstall h5py
pip install h5py==2.10.0

I think this has been fixed in fer==21.0.3. Please try uninstalling tensorflow and pip install -U fer.

DaddyWesker commented 2 years ago

It is weird, but i can't install fer 21.0.3, pip doesn't see such a version.

JustinShenk commented 2 years ago

Sorry, there was an issue with the automatic deployment. Please try again with 21.0.3.

On Mon, Jul 19, 2021 at 9:35 AM DaddyWesker @.***> wrote:

It is weird, but i can't install fer 21.0.3, pip doesn't see such a version.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/justinshenk/fer/issues/29#issuecomment-882316083, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOLMZAHS2JFUJKM2XE4ATDTYPIVVANCNFSM5ACDLWPA .

DaddyWesker commented 2 years ago

Now it works. THanks.