Open zychen2016 opened 4 years ago
Hi @zychen2016, thank you for your interest in reproducing our experiments! Here's how you can do it with our Docker image for object detection
Run the SSD-MobileNet-v1-FPN model:
$ docker run --rm ctuning/object-detection-tf-py.tensorrt.ubuntu-18.04 \
"ck run program:object-detection-tf-py \
--dep_add_tags.lib-tensorflow=vprebuilt \
--dep_add_tags.weights=ssd,mobilenet-v1,fpn \
--env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=50 \
"
...
Summary:
-------------------------------
Graph loaded in 2.072438s
All images loaded in 0.840843s
All images detected in 40.240924s
Average detection time: 0.821243s
mAP: 0.4142710363523775
Recall: 0.46005777333020803
--------------------------------
vprebuilt
to vsrc
):
$ sudo apt install nvidia-docker2 nvidia-container-runtime
$ docker run --runtime=nvidia --rm ctuning/object-detection-tf-py.tensorrt.ubuntu-18.04 \
"ck run program:object-detection-tf-py \
--dep_add_tags.lib-tensorflow=vsrc \
--dep_add_tags.weights=ssd,mobilenet-v1,fpn \
--env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=50
"
...
Summary:
-------------------------------
Graph loaded in 1.909093s
All images loaded in 0.642674s
All images detected in 7.319242s
Average detection time: 0.149372s
mAP: 0.4142710363523775
Recall: 0.46005777333020803
--------------------------------
$ docker run --runtime=nvidia --rm ctuning/object-detection-tf-py.tensorrt.ubuntu-18.04 \
"ck run program:object-detection-tf-py \
--dep_add_tags.lib-tensorflow=vsrc \
--dep_add_tags.weights=ssd,mobilenet-v1,fpn \
--env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=50 \
--env.CK_ENABLE_TENSORRT=1 \
--env.CK_TENSORRT_DYNAMIC=1 \
"
...
Summary:
-------------------------------
Graph loaded in 13.394649s
All images loaded in 0.609364s
All images detected in 7.061026s
Average detection time: 0.144103s
mAP: 0.4142710363523775
Recall: 0.46005777333020803
--------------------------------
You can see a 5.5x speedup when going from the CPU to the GPU and 5.7x speedup to the GPU with TensorRT.
Thank you for your detailed answer!
It seems that you use the tensorflow-tensorrt not the pure tensorrt? so ,The node/layer that are not supported by tensorrt are supported by Tensorflow-tensorrt ?
That's right @zychen2016. More complex / less popular models may have more layers not supported by TensorRT, hence the improvement will be modest (like in the above case).
We are now trying to reproduce NVIDIA's submissions to MLPerf Inference v0.5. Stay tuned if you are interested.
Thans!
There is no run
when type ck --h
in terminal .
If I want to learn how to run with out ck
,Could you give some suggestions?
Thans! There is no
run
when typeck --h
in terminal . If I want to learn how to run with outck
,Could you give some suggestions?
Hi @zychen2016 .
There is no run because "run" belongs to "program" module. CK is a plugin based framework where "actions" belong to different "modules" shared by the community:
You can find "run program" there and you can see the API there ... While we take JSON file as input, you can take any key from there and change it from the command line, i.e.
ck run program:{some CK program} --flags="-O3"
You can also see this API from the command line as follows:
$ ck run program --help
Hope it's of any help ...
@zychen2016 CK makes it easy to run those models (with or without Docker). If you want to run without CK, you should try following Google researchers' instructions yourself. But be warned - we spent a lot of time simplifying the process as much as possible...
I see you have speed up the ssd mobilenet v1 fpn model. How to check your source code about speed up this model?