A project that optimizes OWL-ViT for real-time inference with NVIDIA TensorRT.
Apache License 2.0
232
stars
42
forks
source link
Add Owlv2; Add option to use owlvit / owlv2's image preprocessing procedure (the roi_align in current implementation processes images differently and results in distribution shifts and subpar performance) #23
Add an option (no_roi_align) in the OwlPredictor class of nanoowl/owl_predictor.py to use the original owlvit / owlv2 implementation's image preprocessing procedure. From my debugging, I find that the current nanoowl repo preprocesses images differently from the original owlvit / owlv2 implementations due to the use of "roi_align". This causes the larger models (base-16, large-14) to perform subpar compared to the original owlvit / owlv2. Passing in no_roi_align=True fixes this issue.
Add an option (nms_threshold) to use non-maximal suppression bounding box filtering
Fix cv2 "image no write permission" error in drawing utils
no_roi_align) in theOwlPredictorclass ofnanoowl/owl_predictor.pyto use the original owlvit / owlv2 implementation's image preprocessing procedure. From my debugging, I find that the current nanoowl repo preprocesses images differently from the original owlvit / owlv2 implementations due to the use of "roi_align". This causes the larger models (base-16, large-14) to perform subpar compared to the original owlvit / owlv2. Passing inno_roi_align=Truefixes this issue.nms_threshold) to use non-maximal suppression bounding box filtering