hsemotion inference not working

[archaludon]

The hsemotion.py example script always gives the same output no matter the input. Have changed the 'lenna.png' default image to a different image and the output is the same. Change to video input for webcam and the result is still the same for every frame. The output is always:

INFO hsemotion.py (213) : emotion_class_count=4 INFO hsemotion.py (218) : + idx=0 INFO hsemotion.py (219) : category=7 [ Surprise ] INFO hsemotion.py (223) : prob=0.36632421612739563 INFO hsemotion.py (218) : + idx=1 INFO hsemotion.py (219) : category=3 [ Fear ] INFO hsemotion.py (223) : prob=0.22524544596672058 INFO hsemotion.py (218) : + idx=2 INFO hsemotion.py (219) : category=5 [ Neutral ] INFO hsemotion.py (223) : prob=0.12056519836187363 INFO hsemotion.py (218) : + idx=3 INFO hsemotion.py (219) : category=4 [ Happiness ] INFO hsemotion.py (223) : prob=0.10977587103843689

[kyakuno]

I got correct result for each images. What version of ailia SDK did you use? And which backend did you use? CPU or CUDA?

kyakuno@mbakk hsemotion % python3 hsemotion.py -i lenna.png 
 INFO arg_utils.py (13) : Start!
 INFO arg_utils.py (163) : env_id: 2
 INFO arg_utils.py (166) : MPSDNN-Apple M2
 INFO model_utils.py (86) : ONNX file and Prototxt file are prepared!
 INFO hsemotion.py (121) : lenna.png
 INFO hsemotion.py (132) : Start inference...
 INFO hsemotion.py (146) : emotion_class_count=4
 INFO hsemotion.py (150) : + idx=0
 INFO hsemotion.py (151) :   category=5 [ Neutral ]
 INFO hsemotion.py (153) :   prob=0.6251291632652283
 INFO hsemotion.py (150) : + idx=1
 INFO hsemotion.py (151) :   category=4 [ Happiness ]
 INFO hsemotion.py (153) :   prob=0.15023145079612732
 INFO hsemotion.py (150) : + idx=2
 INFO hsemotion.py (151) :   category=7 [ Surprise ]
 INFO hsemotion.py (153) :   prob=0.0750264823436737
 INFO hsemotion.py (150) : + idx=3
 INFO hsemotion.py (151) :   category=6 [ Sadness ]
 INFO hsemotion.py (153) :   prob=0.05999182537198067
 INFO hsemotion.py (154) : 
 INFO hsemotion.py (155) : Script finished successfully.
kyakuno@mbakk hsemotion % 

kyakuno@mbakk hsemotion % python3 hsemotion.py -i surprise.png  
 INFO arg_utils.py (13) : Start!
 INFO arg_utils.py (163) : env_id: 2
 INFO arg_utils.py (166) : MPSDNN-Apple M2
 INFO model_utils.py (86) : ONNX file and Prototxt file are prepared!
 INFO hsemotion.py (121) : surprise.png
 INFO hsemotion.py (132) : Start inference...
 INFO hsemotion.py (146) : emotion_class_count=4
 INFO hsemotion.py (150) : + idx=0
 INFO hsemotion.py (151) :   category=7 [ Surprise ]
 INFO hsemotion.py (153) :   prob=0.9108676314353943
 INFO hsemotion.py (150) : + idx=1
 INFO hsemotion.py (151) :   category=3 [ Fear ]
 INFO hsemotion.py (153) :   prob=0.04306194931268692
 INFO hsemotion.py (150) : + idx=2
 INFO hsemotion.py (151) :   category=5 [ Neutral ]
 INFO hsemotion.py (153) :   prob=0.027052948251366615
 INFO hsemotion.py (150) : + idx=3
 INFO hsemotion.py (151) :   category=6 [ Sadness ]
 INFO hsemotion.py (153) :   prob=0.005641607567667961
 INFO hsemotion.py (154) : 
 INFO hsemotion.py (155) : Script finished successfully.

[archaludon]

Ailia Version is 1.2.16.0 And I believe I'm Using CUDA, since it names the GPU device in the pre-amble:

INFO arg_utils.py (13) : Start! INFO arg_utils.py (166) : VulkanDNN-NVIDIA T1200 Laptop GPU (FP16) INFO model_utils.py (86) : ONNX file and Prototxt file are prepared! INFO hsemotion.py (121) : emma.jpg INFO hsemotion.py (132) : Start inference... INFO hsemotion.py (146) : emotion_class_count=4 INFO hsemotion.py (150) : + idx=0 INFO hsemotion.py (151) : category=7 [ Surprise ] INFO hsemotion.py (153) : prob=0.36632421612739563 INFO hsemotion.py (151) : category=3 [ Fear ] INFO hsemotion.py (153) : prob=0.22524544596672058 INFO hsemotion.py (150) : + idx=2 INFO hsemotion.py (151) : category=5 [ Neutral ] INFO hsemotion.py (153) : prob=0.12056519836187363 INFO hsemotion.py (150) : + idx=3 INFO hsemotion.py (151) : category=4 [ Happiness ] INFO hsemotion.py (153) : prob=0.10977587103843689 INFO hsemotion.py (154) : INFO hsemotion.py (155) : Script finished successfully.

[kyakuno]

@archaludon In your log, you used Vulkan backend. Can you try -e 0 option for CPU inference?

python3 hsemotion.py -e 0

Also you can list environment using below command.

python3 hsemotion.py --env_list

[archaludon]

Ok - it works when using that -e 0 option for CPU inference. So something about the GPU is failing.

I get this output when running python hsemotion.py --env_list

INFO arg_utils.py (13) : Start! INFO arg_utils.py (153) : env[0]=Environment(id=0, type='CPU', name='CPU', backend='NONE', props=[]) INFO arg_utils.py (153) : env[1]=Environment(id=1, type='BLAS', name='CPU-IntelMKL', backend='NONE', props=[]) INFO arg_utils.py (153) : env[2]=Environment(id=2, type='GPU', name='VulkanDNN-Intel(R) UHD Graphics (FP32)', backend='VULKAN', props=[]) INFO arg_utils.py (153) : env[3]=Environment(id=3, type='GPU', name='VulkanDNN-Intel(R) UHD Graphics (FP16)', backend='VULKAN', props=['FP16']) INFO arg_utils.py (153) : env[4]=Environment(id=4, type='GPU', name='VulkanDNN-NVIDIA T1200 Laptop GPU (FP32)', backend='VULKAN', props=[]) INFO arg_utils.py (153) : env[5]=Environment(id=5, type='GPU', name='VulkanDNN-NVIDIA T1200 Laptop GPU (FP16)', backend='VULKAN', props=['FP16']) INFO arg_utils.py (163) : env_id: 5 INFO arg_utils.py (166) : VulkanDNN-NVIDIA T1200 Laptop GPU (FP16) INFO model_utils.py (86) : ONNX file and Prototxt file are prepared! INFO hsemotion.py (121) : lenna.png INFO hsemotion.py (132) : Start inference... INFO hsemotion.py (146) : emotion_class_count=4 INFO hsemotion.py (150) : + idx=0 INFO hsemotion.py (151) : category=7 [ Surprise ] INFO hsemotion.py (153) : prob=0.36632421612739563 INFO hsemotion.py (150) : + idx=1 INFO hsemotion.py (151) : category=3 [ Fear ] INFO hsemotion.py (153) : prob=0.22524544596672058 INFO hsemotion.py (150) : + idx=2 INFO hsemotion.py (151) : category=5 [ Neutral ] INFO hsemotion.py (153) : prob=0.12056519836187363 INFO hsemotion.py (150) : + idx=3 INFO hsemotion.py (151) : category=4 [ Happiness ] INFO hsemotion.py (153) : prob=0.10977587103843689 INFO hsemotion.py (154) : INFO hsemotion.py (155) : Script finished successfully.

[archaludon]

So it also works running it with 1, 2 or 4 after the -e option, but not with 3 or 5. My guess is it doesn't like FP16, only FP32 when using CPU.

[kyakuno]

I think it's probably because FP16 lacks precision.