imsoo/darknet_server - Githubissues

Darknet server

This is for the use of the Darknet (Open source neural networks) in cloud computing. using this project, You can send video or Webcam stream to server, and get result from Server in real time.

Examples of results

YOLO (Object Detection) OpenPose (Pose Estimation)

Overview

In this project, Server and client communicate based on ZeroMQ message library. Client read frame from video or Webcam using by OpenCV and send to server by json message format. Server receive message and do work something. (Object detection or Pose Estimation) and send result (processed frame and detection result) back to client by json message format.

Client - Server	Server Parallel Pipeline

Requirements

Windows or Linux
OpenCV : https://opencv.org/releases/
ZeroMQ (libzmq) : https://github.com/zeromq/libzmq
json-c : https://github.com/json-c/json-c
on Linux g++, on Windows MSVC 2015/2017/2019 https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community

Pre-trained models

yolov3.cfg (236 MB COCO Yolo v3) - requires 4 GB GPU-RAM: https://pjreddie.com/media/files/yolov3.weights
yolov3-tiny.cfg (34 MB COCO Yolo v3 tiny) - requires 1 GB GPU-RAM: https://pjreddie.com/media/files/yolov3-tiny.weights
openpose.cfg (200 MB OpenPose) - requires 4 GB GPU-RAM: https://github.com/lincolnhard/openpose-darknet
fight.cfg (235 MB Yolo v3 custom train) - requires 4 GB GPU-RAM: https://drive.google.com/open?id=1wqLMNwWGdkxPiFpeXJSLfnKZp8ZD99PS
Examples of results (fight)

How to Build

Server (Linux)


## get darknet
git clone https://github.com/AlexeyAB/darknet
cd darknet

add some code in yolo_v2_class.cpp

vi src/yolo_v2_class.cpp

add below Detector::get_net_height() member function definition

LIB_API int Detector::get_net_out_width() const { detector_gpu_t &detector_gpu = static_cast<detector_gpu_t >(detector_gpu_ptr.get()); return detector_gpu.net.layers[detector_gpu.net.n - 2].out_w; } LIB_API int Detector::get_net_out_height() const { detector_gpu_t &detector_gpu = static_cast<detector_gpu_t >(detector_gpu_ptr.get()); return detector_gpu.net.layers[detector_gpu.net.n - 2].out_h; } LIB_API float Detector::predict(float input) const { detector_gpu_t &detector_gpu = static_cast<detector_gpu_t >(detector_gpu_ptr.get()); return network_predict(detector_gpu.net, input); }

add some code in yolo_v2_class.hpp

vi include/yolo_v2_class.hpp

add below Detector::get_net_height() member function declaration

LIB_API int get_net_out_width() const; LIB_API int get_net_out_height() const; LIB_API float predict(float input) const;

Build a library darknet.so

vi Makefile ## set option LIBSO = 1 make ## build a library darknet.so sudo cp libdarknet.so /usr/local/lib/

Build darknet_server

git clone https://github.com/imsoo/darknet_server cd darknet_server/server make

Run Sink & Ventilator

./sink ./ventilator

Run worker (if GPU shows low utilization and has enough memory, run more worker)

./worker [-pose] [-gpu GPU_ID] [-thresh THRESH]


* #### Client (Linux)
``` sh
## Build darknet_client
git clone https://github.com/imsoo/darknet_server
cd darknet_server/client/darknet_client
make

## Run darknet client
./darknet_client <-addr ADDR> <-vid VIDEO_PATH | -cam CAM_NUM> [-out_vid] [-out_json] [-dont_show]

Client (Windows)

Visual Studio Setting Up and Build

How to use on the command line

Server (Linux)

YOLOv3 : ./worker cfg/yolov3.cfg weights/yolov3.weights names/cooc.names -gpu 0 -thresh 0.2
OpenPose : ./worker cfg/openpose.cfg weights/openpose.weights -gpu 0 -pose

Client (Windows | Linux)

Cam : ./darknet_client -addr x.x.x.x -cam 0
Video : ./darknet_client -addr x.x.x.x -vid test.mp4 
Save result video file : ./darknet_client -addr x.x.x.x -vid test.mp4 -out_vid  # save to test_output.mp4
Save result json file : ./darknet_client -addr x.x.x.x -vid test.mp4 -out_json  # save to test_output.json
Save result only (don't show window) : ./darknet_client -addr x.x.x.x -cam 0 -out_vid -out_json -dont_show

JSON Output

YOLO :

{
"det": [
{
  "frame_id": 1,
  "objects": [
    {
      "class_id": 27,
      "name": "giraffe",
      "absolute_coordinates": {
        "center_x": 275,
        "center_y": 194,
        "width": 8,
        "height": 28
      },
      "confidence": 0.20249
    }
  ]
}
]
}

OpenPose :

{
"det": [
{
  "frame_id": 1,
  "people": [
    {
      "0": [351.977112,175.938629],   // NOSE
      "1": [384.007935,207.964188],   // NECK
      "2": [339.164185,208.048325],   // RRShoulder
      "3": [336.007416,275.159973],   // RElbow
      "4": [0,0],                     // RWrist
      "5": [425.677979,204.863556],   // LShoulder
      "6": [454.353577,268.810059],   // LElbow
      "7": [444.86261,326.342651],    // LWrist
      "8": [361.499115,335.980316],   // RHip
      "9": [361.627502,428.830963],   // RKnee
      "10": [0,0],                    // RAnkle
      "11": [416.036926,335.951752],  // LHip
      "12": [419.235413,428.756653],  // LKnee
      "13": [0,0],                    // LAnkle
      "14": [345.621887,166.345505],  // REye
      "15": [364.737091,166.310699],  // LEye
      "16": [0,0],                    // REar
      "17": [387.237091,163.059067]   // LEar
    }
  ]
}
]
}

References

Darknet : https://github.com/AlexeyAB/darknet
OpenCV : https://github.com/opencv/opencv
ZeroMQ : https://github.com/zeromq/libzmq
json-c : https://github.com/json-c/json-c
openpose-darknet : https://github.com/lincolnhard/openpose-darknet
cpp-base64 : https://github.com/ReneNyffenegger/cpp-base64
mem_pool : https://www.codeproject.com/Articles/27487/Why-to-use-memory-pool-and-how-to-implement-it
share_queue : https://stackoverflow.com/questions/36762248/why-is-stdqueue-not-thread-safe