Closed rajil closed 4 years ago
The object detection is using CUDA, but the face detection is not. This is probably the reason it gets booted out. Is CUDA not supported for face recognition?
Setting 'timeout=120' in gunicorn.conf.py fetched more time to finish query.
My config is like so,
[ml]
# Starting version 4.2 of OpenCV, the DNN models support CUDA
# If you have compiled OpenCV 4.2 with CUDA support correctly
# set this to yes. Note that if you have just installed a package
# chances are it is not properly set up with CUDA. It is much better
# you compile OpenCV from source (and uninstall any opencv packages you
# installed via pip or apt-get)
# Read https://www.pyimagesearch.com/2020/02/03/how-to-use-opencvs-dnn-module-with-nvidia-gpus-cuda-and-cudnn/ on how to do it right.
# Play special attention to putting in the right CUDA_ARCH_BIN value that
# matches your GPU or you'll face "invalid device errors in make_policy"
# while trying to actually run it (compile will work fine)
use_opencv_dnn_cuda=yes
[yolo]
yolo_type=full
config=./models/yolov3/yolov3.cfg
weights=./models/yolov3/yolov3.weights
labels=./models/yolov3/yolov3.labels
tiny_config=./models/tinyyolo/yolov3-tiny.cfg
tiny_weights=./models/tinyyolo/yolov3-tiny.weights
tiny_labels=./models/tinyyolo/yolov3-tiny.labels
[face]
face_num_jitters=0
face_upsample_times=1
face_model=cnn
face_train_model=hog
face_recog_dist_threshold=0.6
face_recog_knn_algo=ball_tree
known_faces_path=./known_faces
unknown_faces_path=./unknown_faces
unknown_face_name=unknown face
save_unknown_faces=yes
save_unknown_faces_leeway_pixels=50
A few things:
If you are installing the ES+hook using install.sh:
a) Face detection does use GPU, but make sure you have followed the hooks instructions on how to compile Dlib with GPU support
b) Looking at your logs, looks like you haven't configured face recognition properly as per the instructions. Specifically, you don't have any trained faces. You may want to first go through the training process
If you are using mlapi using @themoosman 's containerized fork (https://github.com/themoosman/mlapi), then you might want to post on his fork on questions specific to setting up face recognition in his container and/or Gunicorn settings. I haven't had a chance to look at what requirements he has on setting things up.
It looks like it can't find the face recognition files. Are they present?
DEBUG: Initializing face recognition with model:cnn upsample:1, jitters:0
DEBUG: trained file not found, reading from images and doing training...
ERROR: No known faces found to train, encoding file not created
ERROR: Error loading face recognition file: [Errno 2] No such file or directory: './known_faces/faces.dat'
That error means the OP hasn't trained any faces. I don't remember what mlapi does when there are no faces to recognize because none were trained. If it is doing nothing, that may also explain the timeout. I fixed this in the ES and don't remember if it was also fixed in mlapi. Another problem to look at and why I want both the local hook and mlapi to same the same libs.
I copied the known_faces directory to the mlapi server from ES server and the message went away. Also, I was able to get dlib going with CUDA.
I am trying to use an mlapi server for face detection. Objection detection works fine, but face detection fails.