pytorch / android-demo-app

PyTorch android examples of usage in applications
1.45k stars 596 forks source link

I write a letterbox function like detect.py for android object detection, it can make the output probablity higher and more robust. #259

Open delongwu opened 2 years ago

delongwu commented 2 years ago

Just enjoy it , this function almost same as python augmentations.py letterbox function. but has some diff at end of gray border, but it is ok.

you can use letterbox function to replace the

Bitmap resizedBitmap = Bitmap.createScaledBitmap(picBitMap, PrePostProcessor.mInputWidth, PrePostProcessor.mInputHeight, true);

/**
     * Image augmentations: first scale the image and later padding image,  increase the strength of the model.
     * Always this scale and padding will make the image object detection gave more high probability or more robust.
     * <p>
     * Reference:
     * https://github.com/ultralytics/yolov5/blob/db6ec66a602a0b64a7db1711acd064eda5daf2b3/utils/augmentations.py#L91-L122
     * def letterbox(im, new_shape=(640, 640), color=(114, 114, 114), auto=True, scaleFill=False, scaleup=True, stride=32):
     * method
     *
     * @param srcBitmap
     * @param newShape  (640*640)
     * @param color     always gray (114,114,114)
     * @param auto      default:false, no use
     * @param scaleFill default:false,  no use
     * @param scaleUp   default:false
     * @param stride    default:32 , no use
     * @return
     */
    public static Bitmap letterbox(Bitmap srcBitmap, Pair<Integer, Integer> newShape, Triple<Integer, Integer, Integer> color, Boolean auto,
                                   Boolean scaleFill, Boolean scaleUp, int stride) {
        // current shape
        int currentWidth = srcBitmap.getWidth();
        int currentHeight = srcBitmap.getHeight();

        // new shape eg: 640*640
        int newWidth = newShape.first;
        int newHeight = newShape.second;

        // only scale image,no padding,just return scale image
        // I modify this logic something difference with the python code clean & speed.
        if (scaleFill) {
            // filter =  bilinear filtering
            return Bitmap.createScaledBitmap(srcBitmap, newWidth, newHeight, true);
        }

        // Scale ratio (new / old)
        float r = Math.min(newWidth * 1.0f / currentWidth, newHeight * 1.0f / currentHeight);

        //  Only scale down, do not scale up (for better val mAP)
        if (!scaleUp) {
            r = Math.min(r, 1.0f);
        }

        int newUnpadWidth = Math.round(currentWidth * r);
        int newUnpadHeight = Math.round(currentHeight * r);

        //  wh padding
        int dw = newWidth - newUnpadWidth;
        int dh = newHeight - newUnpadHeight;

        // auto always false, no use for android demo
        if (auto) { // # wh padding
            dw = dw % stride;
            dh = dh % stride;
        }

        // resize
        if (!(currentWidth == newUnpadWidth && currentHeight == newUnpadHeight)) {
            srcBitmap = Bitmap.createScaledBitmap(srcBitmap, newUnpadWidth, newUnpadHeight, true);
        }

        // padding with gray color
        Bitmap outBitmap = Bitmap.createBitmap(srcBitmap.getWidth() + dw, srcBitmap.getHeight() + dh, Bitmap.Config.ARGB_8888);
        Canvas can = new Canvas(outBitmap);
        can.drawRGB(color.getFirst(), color.getSecond(), color.getThird()); // gray color
        can.drawBitmap(srcBitmap, dw, dh, null);

        return outBitmap;
    }
zhiqwang commented 2 years ago

This result is great, have you compared the lost in speed?