getting low fps & inference issue

[akashAD98]

1.i used this repo https://github.com/neuralmagic/deepsparse/tree/main/examples/ultralytics-yolo & this command !python annotate.py \ zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned_quant-aggressive_94\ --source "/content/loc1min.mp4" \ --quantized-inputs \ --image-shape 416 416 \ --save-dir '/content/ops/' \ --model-config '/content/coco128.yaml'\ --device 'cpu'

& im getting low fps on cpu, (yolov5s model) its normal fps or should we get 50-60 fps? bcz you have mentioned that model will be 10x faster. but its very less.

2. i trained model usiing sparsml repo on coco128 data for 40 epochs & converted .pth model into onnx & tried same inference script !python annotate.py \ /content/sparseml/integrations/ultralytics-yolov5/yolov5/runs/train/exp2/weights/best.onnx \ --source "/content/loc1min.mp4" \ --quantized-inputs \ --image-shape 416 416 \ --save-dir '/content/ops/' \ --model-config '/content/coco128.yaml' --device 'cpu' & getting this issue.

What's wrong here? my goal is to use a custom data train model with sparceml & do inference using deepspare.

[bfineran]

Hi @akashAD98 thank you for reporting this issue. To help us dive in, could you paste the results from running $ deepsparse.check_hardware to get more information about your CPU?

For the second issue, could you provide the training command used to create the model?

[akashAD98]

im doing setup on google collab

!python annotate.py /content/sparseml/integrations/ultralytics-yolov5/yolov5/runs/train/exp2/weights/best.onnx --source "/content/loc1min.mp4" --quantized-inputs --image-shape 416 416 --save-dir '/content/ops/' --model-config '/content/coco128.yaml' --device 'cpu'

this command for inference & yolov5 command for traing

traing !python train.py --data /content/sparseml/integrations/ultralytics-yolov5/yolov5/data/coco128.yaml --cfg ../models_v5.0/yolov5s.yaml --weights zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned-aggressive_96?recipe_type=transfer --hyp data/hyps/hyp.finetune.yaml --recipe ../recipes/yolov5.transfer_learn_pruned.md

conversion to onnx

!python export.py --weights /content/sparseml/integrations/ultralytics-yolov5/yolov5/runs/train/exp2/weights/best.pt --dynamic

@bfineran please have look

[bfineran]

thanks for the information @akashAD98. We are looking into it and will provide an update shortly. In the meantime, could you provide your pytorch, deepsparse, and sparseml versions as well as the commit hash of the neuralmagic/yolov5 repo you are using?

[akashAD98]

@bfineran im using the latest repo & there mentioned requirements.txt , I used https://github.com/neuralmagic/deepsparse/blob/main/examples/ultralytics-yolo/requirements.txt for doing setup.

[KSGulin]

@akashAD98 the low fps you're seeing may be a result of low computational resources offered by google colab. If you run the same annotation with a baseline model (with the command below), what fps do you see? I would recommend trying a run with the baseline and pruned model back-to-back, as I'm not certain if google colab provided resources vary over time.

python annotate.py
zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/base-none
--source "/content/loc1min.mp4"
--quantized-inputs
--image-shape 416 416
--save-dir '/content/ops/'
--model-config '/content/coco128.yaml'
--device 'cpu'

For the model that's giving you an error, could you share the command you used to train it?

[akashAD98]

@KSGulin ok ill try this

[akashAD98]

@KSGulin i followed this repo, & I'm able to do training of my custom moel https://github.com/dnth/yolov5-deepsparse-blogpost