search

pytorch / glow

Compiler for Neural Network hardware accelerators
Apache License 2.0
3.22k stars 689 forks source link

[ModelZoo] support tiny_yolov2 model #2502

Open nickgg opened 5 years ago

nickgg commented 5 years ago

Add support for the Tiny YOLOv2 object detection model (https://github.com/onnx/models/tree/master/tiny_yolov2) and verify via a runnable example we can include in the ModelZoo regression test.

This uses a different image format to our existing ImageClassifier tests (1x3x416x416), but may be solvable via scaling. Attempting to load the tiny_yolov2 model into image classifier fails due to unsupported operator ImageScaler.

This issue blocks #2500.

tlepley-cadence commented 5 years ago

@nickgg We have also worked on enabling yolo v2 tiny. We didn't use the model of the ONNX zoo, but a ONNX model converted from Tensorflow. The one from the ONNX zoo looks a cleaner than ours (it uses higher level nodes, while ours uses low level nodes).

Note that in addition to ImageScaler, you'll need support for LeakyRelu. 

Total ops in the graph =  32
Op types = BatchNormalization:8 Conv:9 ImageScaler:1 LeakyRelu:8 MaxPool:6

A partial support of LeakyRelu is here: #2613 . You'll just need to add LeakyRelu to the ONNX importer, what we didn't have to do, but what should be pretty easy.

Note that the Yolo model needs a specific post-processing. It's an object detector, not an image classifier. Hopefully, we'll be able to share some code in the up-coming weeks.

FYI: @ayermolo

nadavrot commented 5 years ago

@tlepley Would you be willing to contribute your ONNX model to the glow model zoo? Do you mind sharing your pre-trained ONNX file?

@nickgg @mortzur

ayermolo commented 5 years ago

On side note if anyone is interested. They can use dark flow to convert Darkent YOLOv2/3etc to TF. https://github.com/thtrieu/darkflow.

CFG files: https://github.com/pjreddie/darknet/tree/master/cfg

Weithgs https://pjreddie.com/media/files/yolov2.weights https://pjreddie.com/media/files/yolov2-voc.weights https://pjreddie.com/media/files/yolov2-tiny.weights https://pjreddie.com/media/files/yolov2-tiny-voc.weights

As was mentioned these models are lower level so BatchNormalization/RELU are lowered. Also post processing needs to be implemented: IOU computation, nonmax supression. Still learning about object detection, but some/all of this post processing might be applicable to other networks.

nadavrot commented 5 years ago

@ayermolo do you have ONNX files for other networks that you would like to add to the Glow model zoo? We could add the new models to our nightly builds to make sure that they don't break.

nadavrot commented 5 years ago

@opti-mix @jackm321

I can see that we are only testing resnet in "resnet-runtime-test.sh" Can we test all of the files in our model zoo?

nickgg commented 5 years ago

@nadavrot I made #2500 for something like that.

ayermolo commented 5 years ago

@nadavrot I don't. Having issues converting TF flow model generated by Darkflow to ONNX. Put it on back burner for now.

ayermolo commented 5 years ago

Another useful link: Apparently there is a bug in original Darknet C implementation, and what the model has been trained on. https://github.com/pjreddie/darknet/issues/344

ayermolo commented 4 years ago

@nickgg would you guys be interested in the YoloV2 ONNX model, without post processing? It comes from darknet by way of darkflow.