search

openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
https://docs.openvino.ai
Apache License 2.0
6.87k stars 2.19k forks source link

Where to get yolo v3 tiny yml example about accuracy_check? #417

Closed lihaofd closed 4 years ago

lihaofd commented 4 years ago

Hi, We are testing standard yolo v3 tiny model from darknet(https://github.com/AlexeyAB/darknet) with VOC 2012 on OpenVINO toolkit 2020.1 release

After converting the model to pb and then OpenVINO IR(https://github.com/opencv/dldt/blob/2020/model-optimizer/extensions/front/tf/yolo_v3_tiny.json), python mo_tf.py --input_model .\frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config .\yolo_v3_tiny.json --batch 1

it can detect target object successfully by using object_detection_demo_yolov3_async from openvino

But when running accuracy_check, accuracy_check -c /home/work/testmodel/yolo_v3_tiny/yolo.yml -td CPU It always shows 500 objects processed in 71.299 seconds map: 1.30%

Below is my script and yml configuration convert_annotation voc_detection --imageset_file /home/work/VOC/VOCdevkit/VOC2012/ImageSets/Main/train.txt --images_dir /home/work/VOC/VOCdevkit/VOC2012/JPEGImages --annotations_dir /home/work/VOC/VOCdevkit/VOC2012/Annotations/ -ss 500 -o /home/work/testmodel/yolo_v3_tiny/annotations -a yolo_v3_tiny.pickle -m yolo_v3_tiny.json

my yolo.yml models:

I guess there might have sth wrong with my yml configuration? Can anyone be helpful on it?

Thanks!

jgespino commented 4 years ago

Hi @lihaofd

Could you share a link the the exact model you are using? Looking at that repository, I'm only seeing Yolo V3 models that have been trained with COCO dataset.

Regards, Jesus

lihaofd commented 4 years ago

https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov3-tiny.cfg https://pjreddie.com/media/files/yolov3-tiny.weights

Seems it is based on COCO. I download COCO val and annotation from http://images.cocodataset.org/zips/val2014.zip and unzip to /home/work/COCO/val2014/ and from http://images.cocodataset.org/annotations/annotations_trainval2014.zip and unzip to

accuracy_check -c /home/work/testmodel/yolo_v3_tiny/yolo_coco.yml -td CPU

models:

22:53:55 accuracy_checker WARNING: /home/work/openvino_folder/openvino_2020.2.084/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/config/config_validator.py:123: UserWarning: YoloV3Adapter specifies unknown options: ['framework', 'tags', 'model', 'weights', 'adapter', '_vpu_log_level', 'mo_params', 'async_mode', 'device'] warnings.warn(message)

45%| | 18152/40504 [5:31:01<24696:18:01]Killed

@jgespino, could you help on that? Thanks!

lihaofd commented 4 years ago

@jgespino , any new update? could you help on this issue? Thanks!

jgespino commented 4 years ago

Hi @lihaofd

I tested and also had a fail at 65%. Not sure if the person_keypoints_val2014.json is the correct file as the YoloV3 has 80 classes. I am going to try going though the process with my custom trained Tiny YoloV3 model. I will let you know what I find out.

Regards, Jesus

jgespino commented 4 years ago

Hi @lihaofd

Apologies for the long delay. I'm not sure if you are still seeing this issue with the latest OpenVINO 2020.4 release. If you are, feel free to re-open this issue or start a new discussion.

Regards, Jesus