Hello!

[ghost]

Hello Thank you so much for your code, it works really well to detect rectangle!

But, i have 1 issue when i want to get the photo of rectangle that i detect. I already try to do it, but what i get is always returning the false point of the photo, so it returned different photo with what i detect on the cameraPreview. Can you help me ?

What i did : Get the points from your getPath method, and i just simply using method Transform (getPerspectiveTransform) from MatORI(Original Image) with MatSRC(Mat with 4 points inside).

Thank you so much! Hope you can help me! :)

[iyotetsuya]

Thank you for contacting me. Can you capture screen and post your code about that? I need to know what the problem is.

[ghost]

Sorry for late reply. This is my code for getting the points from your getPath function:

              if(edgeMat != null) {                               // through this when it detects rectangle
                    pa = new Point((float) cameraPoints.get(0).x,
                            (float) cameraPoints.get(0).y);

                    pb = new Point((float) cameraPoints.get(2).x,
                            (float) cameraPoints.get(2).y);

                    pc = new Point((float) cameraPoints.get(3).x,
                            (float) cameraPoints.get(3).y);

                    pd = new Point((float) cameraPoints.get(1).x,
                            (float) cameraPoints.get(1).y);

                    List<Point> Temp1 = new ArrayList<Point>();
                    Temp1.add(pa);
                    Temp1.add(pb);
                    Temp1.add(pc);
                    Temp1.add(pd);

                    List<Point> Top1 = new ArrayList<Point>();
                    List<Point> Bot1 = new ArrayList<Point>();
                    Point center = ProcImg.GetCenter(pa, pb, pc, pd);

                    try {
                        for (int i = 0; i < Temp1.size(); i++) {
                            if (Temp1.get(i).y < center.y) {
                                Top1.add(Temp1.get(i));
                            } else {
                                Bot1.add(Temp1.get(i));
                            }
                        }

                        if (Top1.get(0).x < Top1.get(1).x) {
                            TL = Top1.get(0);
                            TR = Top1.get(1);
                        } else {
                            TL = Top1.get(1);
                            TR = Top1.get(0);
                        }

                        if (Bot1.get(0).x < Bot1.get(1).x) {
                            BL = Bot1.get(0);
                            BR = Bot1.get(1);
                        } else {
                            BL = Bot1.get(1);
                            BR = Bot1.get(0);
                        }

                        if(!temp) {                             // my idea is to only get the points of rectangle one time and at the first

                            matData.finalPoints = new ArrayList<Point>();
                            matData.finalPoints.add(TL);
                            matData.finalPoints.add(BL);
                            matData.finalPoints.add(BR);
                            matData.finalPoints.add(TR);

                            temp = true;
                        }

and this is my capture function : i called with this cameraData.camera.autoFocus(AutoFocus);

Camera.AutoFocusCallback AutoFocus = new Camera.AutoFocusCallback(){
        @Override
        public void onAutoFocus(boolean data, Camera camera) {
            camera.takePicture(get_img, view_img, save_img);
        }
    };
    Camera.ShutterCallback get_img = new Camera.ShutterCallback(){
        @Override
        public void onShutter() {}
    };

    Camera.PictureCallback view_img = new Camera.PictureCallback(){
        @Override
        public void onPictureTaken(byte[] image, Camera camera) {
            camera.addCallbackBuffer(image);
        }
    };

    Camera.PictureCallback save_img = new Camera.PictureCallback(){
        @SuppressLint("SimpleDateFormat")
        @Override
        public void onPictureTaken(byte[] image, Camera camera1) {
            if (image != null){
                Camera.Parameters parameters                = cameraData.camera.getParameters();
                //Get Picture Size METHODE I
                Camera.Size PicSize                         = parameters.getPictureSize();
                android.hardware.Camera.Size SupportSize    = null;

                //Get Picture Size METHODE II
                List<android.hardware.Camera.Size> SupSize = parameters.getSupportedPictureSizes();
                for(int i=0; i < SupSize.size();i++){
                    int h   = SupSize.get(i).height;
                    int w   = SupSize.get(i).width;

                    if (i != 0){
                        int h1  = SupSize.get(i-1).height;
                        int w1  = SupSize.get(i-1).width;

                        if (h * w > h1 * w1){
                            SupportSize = SupSize.get(i);
                        }else {
                            SupportSize = SupSize.get(i-1);
                        }
                    }
                }

                //Get Picture Size METHODE III
                for (android.hardware.Camera.Size size1 : SupSize) {
                    SizeSupport = size1;
                    break;
                }

                //SET PICTURE SIZE WITH COMPARATION
                if (SupportSize.width * SupportSize.height >=  PicSize.width * PicSize.height ){
                    if (SupportSize.width * SupportSize.height >= SizeSupport.width * SizeSupport.height){
                        heightcam   = SupportSize.height;
                        widthcam    = SupportSize.width;
                    }else {
                        heightcam   = SizeSupport.height;
                        widthcam    = SizeSupport.width;
                    }
                }else {
                    if (PicSize.width * PicSize.height >= SizeSupport.width * SizeSupport.height){
                        heightcam   = PicSize.height;
                        widthcam    = PicSize.width;
                    }else {
                        heightcam   = SizeSupport.height;
                        widthcam    = SizeSupport.width;
                    }
                }
                resolusi                            = heightcam * widthcam ;
                //*/

                Intent intentresultcam      = new Intent(MainActivity.this, ResultActivity.class);
                ByteArrayOutputStream bs    = new ByteArrayOutputStream();

                //*
                //FOR PASSING IMAGE WITH INTENT
                if (resolusi <= 4000000){
                    bitmapPicture               = BitmapFactory.decodeByteArray(image, 0, image.length);
                }else{
                    BitmapFactory.Options options   = new BitmapFactory.Options();
                    int scale;
                    if ( (resolusi > 4000000) && (resolusi < 7800000) ){
                        scale   = 2;
                    }else if ( (resolusi >= 7800000) && (resolusi < 12000000) ){
                        scale   = 2;
                    }else {
                        scale   = 2;
                    }
                    options.inPreferredConfig   = Bitmap.Config.RGB_565;
                    options.inSampleSize        = scale;
                    options.inPurgeable         = true;
                    options.inInputShareable    = true;
                    bitmapPicture               = BitmapFactory.decodeByteArray(image, 0, image.length);
                }
                Matrix matrix = new Matrix();
                matrix.postRotate(90);
                bitmapPicture                   = Bitmap.createBitmap(bitmapPicture, 0, 0, bitmapPicture.getWidth(), bitmapPicture.getHeight(),matrix, true);
                bitmapPicture.compress(Bitmap.CompressFormat.JPEG, 100, bs);
                //*/

                /*
                //BEGIN FOR SAVE CAMERA IMAGE
                Matrix matrix = new Matrix();
                matrix.postRotate(90);
                if (resolusi <= 8000000){
                    bitmapPicture                       = BitmapFactory.decodeByteArray(image, 0, image.length);
                    bitmapPicture                       = Bitmap.createBitmap(bitmapPicture, 0, 0, bitmapPicture.getWidth(), bitmapPicture.getHeight(),matrix, true);
                    bitmapPicture.compress(Bitmap.CompressFormat.JPEG,90, bs);
                }else {
                    BitmapFactory.Options options       = new BitmapFactory.Options();
                    options.inSampleSize                = 2;
                    bitmapPicture                       = BitmapFactory.decodeByteArray(image, 0, image.length,options);
                    bitmapPicture                       = Bitmap.createBitmap(bitmapPicture, 0, 0, bitmapPicture.getWidth(), bitmapPicture.getHeight(),matrix, true);
                    bitmapPicture.compress(Bitmap.CompressFormat.JPEG,90, bs);
                }

                File FolderCam                      = new File(AppDirectory, "o2ocam");
                if (!FolderCam.exists()){
                    FolderCam.mkdir();
                }
                byte[] byteArray = bs.toByteArray();
                ClassImage ImgSave                          = new ClassImage();
                ImgSave.SaveBufferImage(byteArray, book_id + ".png", FolderCam);
                //END FOR SAVE CAMERA IMAGE
                */
                Mat MatSRC = Converters.vector_Point2f_to_Mat(pointsOnMain); // value of pointsOnMain is from matData.finalPoints which is i get it from the getPath function

                Bitmap bitmapimage  = BitmapFactory.decodeByteArray(bs.toByteArray(), 0, bs.toByteArray().length);
                Mat matimage = new Mat();
                Utils.bitmapToMat(bitmapimage, matimage);

                Mat MatTransform    = ProcImg.Transform(matimage, MatSRC, bitmapimage.getWidth(), bitmapimage.getHeight());
                Bitmap bitmapCrop   = bitmapimage.copy(Bitmap.Config.RGB_565, true);
                Utils.matToBitmap(MatTransform, bitmapCrop);

                ByteArrayOutputStream bos = new ByteArrayOutputStream();
                bitmapCrop.compress(Bitmap.CompressFormat.PNG, 0 /*ignored for PNG*/, bos);
                byte[] bitmapdata = bos.toByteArray();
//                ByteArrayInputStream croppedBs = new ByteArrayInputStream(bitmapdata);

                //FOR PASSING IMAGE WITH INTENT
                intentresultcam.putExtra("RESULTCAM", bs.toByteArray());
                intentresultcam.putExtra("Resolution", resolusi);
                startActivity(intentresultcam);

                if ( image != null){
                    image   = null;
                }
                if(!bitmapPicture.isRecycled()){
                    bitmapPicture.recycle();
                }
                try {
                    bs.close();
                } catch (IOException e) {}
                System.gc();
            }else{
                Toast.makeText(getApplicationContext(), "No Image", Toast.LENGTH_SHORT).show();
            }
        }
    };

And then in ResultActivity i get the bitmap with this : Bitmap bitmapResult = BitmapFactory.decodeByteArray(i.getByteArrayExtra("RESULTCAM"), 0, i.getByteArrayExtra("RESULTCAM").length);

Oh ya and this is my Transform function at the ProcImage which i called this function at the top for cropping the matPoints to the OriginalMat :

public Mat Transform(Mat MatOri, Mat MatSourceCrop,int width, int height){
        List<Point> DSTPoint = new ArrayList<Point>();
        DSTPoint.add(new Point(0, 0));
        DSTPoint.add(new Point(0, height));
        DSTPoint.add(new Point(width, height));
        DSTPoint.add(new Point(width, 0));
        Mat MatDST = Converters.vector_Point2f_to_Mat(DSTPoint);

        Mat MatPerspective  = Imgproc.getPerspectiveTransform(MatSourceCrop, MatDST);
        Core.perspectiveTransform(MatSourceCrop, MatDST, MatPerspective);

        Mat MatResult           = new Mat(width, height, CvType.CV_8UC1);
        Imgproc.warpPerspective(MatOri, MatResult, MatPerspective, new Size(width, height));

        return MatResult;
    }

So sorry if i give you a hardcode, because i'm afraid there is something wrong at my take picture function. Thank you so much for your help!

[iyotetsuya]

I think that possibly, maybe it doesn't scale correctly. The points from getPath() are calculate for DrawView (scale to screen size). If you want to calculate for original image, please see matData.points in public static Observable<MatData> getPath(MatData matData) or scale back from your image size.

Thank you for your reply.

[ghost]

Thank you for reply!

I'm aware for that problem, i see that you were resize your camera ratio so it will return the different points from original image. So i picked the Original points from where u did not resize your ratio, but it returned the same thing.

[iyotetsuya]

Umm...., if you crop image by the points directly, abort the method Core.perspectiveTransform, Is it correct result? (a skew square you want to detect)

[ghost]

If i abort that method, it returned original photo without cropped. do you have other simpler method ?

Thank you!

[iyotetsuya]

I want to make sure that you get correct points :) maybe you can try step by step, draw the points on picture first.

[ghost]

I solved it somehow! But can you explain me about the matData.cameraRatio and matData.resizeRatio ? if you don't mind.

Thank you!

[iyotetsuya]

Nice going! sure, I should write code that could be read and understood.

.concatMap(matData -> OpenCVHelper.resize(matData, 400, 400))
.map(matData -> {
    matData.resizeRatio = (float) matData.oriMat.height() / matData.resizeMat.height();
    matData.cameraRatio = (float) cameraPreview.getHeight() / matData.oriMat.height();
    return matData;
})

Point cameraPoint = new Point(
    point.x * matData.resizeRatio * matData.cameraRatio,
    point.y * matData.resizeRatio * matData.cameraRatio);

I resize image to improve running speed, so if I want to draw the points on view, it should be resize back to original size first, then get the position for the CameraPreview.

Please feel free to ask if there are any parts that are unclear.

[ghost]

Yes it is !

What i'm confusing now is when i take a picture, it returned bad quality photo. then i tho i should change the resize method to my screen width and height. so it will be like this :

.concatMap(matData -> OpenCVHelper.resize(matData, screenWidth, screenHeight))
.map(matData -> {
    matData.resizeRatio = (float) matData.oriMat.height() / matData.resizeMat.height();
    matData.cameraRatio = (float) cameraPreview.getHeight() / matData.oriMat.height();
    return matData;
})

it takes a long time to take a photo, but it's still returning bad quality photo, can you tell me why ? Thank you!

[iyotetsuya]

What source you use to display the photo? You should get photo from oriMat.

[ghost]

i use this to take a pic

Camera.AutoFocusCallback AutoFocus = new Camera.AutoFocusCallback(){
        @Override
        public void onAutoFocus(boolean data, Camera camera) {
            camera.takePicture(get_img, view_img, save_img);
        }
    };
    Camera.ShutterCallback get_img = new Camera.ShutterCallback(){
        @Override
        public void onShutter() {}
    };

    Camera.PictureCallback view_img = new Camera.PictureCallback(){
        @Override
        public void onPictureTaken(byte[] image, Camera camera) {
            camera.addCallbackBuffer(image);
        }
    };

    Camera.PictureCallback save_img = new Camera.PictureCallback(){
        @SuppressLint("SimpleDateFormat")
        @Override
        public void onPictureTaken(byte[] image, Camera camera1) {
            if (image != null){
                Camera.Parameters parameters                = cameraData.camera.getParameters();
                //Get Picture Size METHODE I
                Camera.Size PicSize                         = parameters.getPictureSize();
                android.hardware.Camera.Size SupportSize    = null;

                //Get Picture Size METHODE II
                List<android.hardware.Camera.Size> SupSize = parameters.getSupportedPictureSizes();
                for(int i=0; i < SupSize.size();i++){
                    int h   = SupSize.get(i).height;
                    int w   = SupSize.get(i).width;

                    if (i != 0){
                        int h1  = SupSize.get(i-1).height;
                        int w1  = SupSize.get(i-1).width;

                        if (h * w > h1 * w1){
                            SupportSize = SupSize.get(i);
                        }else {
                            SupportSize = SupSize.get(i-1);
                        }
                    }
                }

                //Get Picture Size METHODE III
                for (android.hardware.Camera.Size size1 : SupSize) {
                    SizeSupport = size1;
                    break;
                }

                //SET PICTURE SIZE WITH COMPARATION
                if (SupportSize.width * SupportSize.height >=  PicSize.width * PicSize.height ){
                    if (SupportSize.width * SupportSize.height >= SizeSupport.width * SizeSupport.height){
                        heightcam   = SupportSize.height;
                        widthcam    = SupportSize.width;
                    }else {
                        heightcam   = SizeSupport.height;
                        widthcam    = SizeSupport.width;
                    }
                }else {
                    if (PicSize.width * PicSize.height >= SizeSupport.width * SizeSupport.height){
                        heightcam   = PicSize.height;
                        widthcam    = PicSize.width;
                    }else {
                        heightcam   = SizeSupport.height;
                        widthcam    = SizeSupport.width;
                    }
                }
                resolusi                            = heightcam * widthcam ;
                //*/

                Intent intentresultcam      = new Intent(MainActivity.this, ResultActivity.class);
                ByteArrayOutputStream bs    = new ByteArrayOutputStream();

                //*
                //FOR PASSING IMAGE WITH INTENT
                if (resolusi <= 4000000){
                    bitmapPicture               = BitmapFactory.decodeByteArray(image, 0, image.length);
                }else{
                    BitmapFactory.Options options   = new BitmapFactory.Options();
                    int scale;
                    if ( (resolusi > 4000000) && (resolusi < 7800000) ){
                        scale   = 2;
                    }else if ( (resolusi >= 7800000) && (resolusi < 12000000) ){
                        scale   = 2;
                    }else {
                        scale   = 2;
                    }
                    options.inPreferredConfig   = Bitmap.Config.RGB_565;
                    options.inSampleSize        = scale;
                    options.inPurgeable         = true;
                    options.inInputShareable    = true;
                    bitmapPicture               = BitmapFactory.decodeByteArray(image, 0, image.length);
                }
                Matrix matrix = new Matrix();
                matrix.postRotate(90);
                bitmapPicture                   = Bitmap.createBitmap(bitmapPicture, 0, 0, bitmapPicture.getWidth(), bitmapPicture.getHeight(),matrix, true);
                bitmapPicture.compress(Bitmap.CompressFormat.JPEG, 100, bs);
                //*/

                /*
                //BEGIN FOR SAVE CAMERA IMAGE
                Matrix matrix = new Matrix();
                matrix.postRotate(90);
                if (resolusi <= 8000000){
                    bitmapPicture                       = BitmapFactory.decodeByteArray(image, 0, image.length);
                    bitmapPicture                       = Bitmap.createBitmap(bitmapPicture, 0, 0, bitmapPicture.getWidth(), bitmapPicture.getHeight(),matrix, true);
                    bitmapPicture.compress(Bitmap.CompressFormat.JPEG,90, bs);
                }else {
                    BitmapFactory.Options options       = new BitmapFactory.Options();
                    options.inSampleSize                = 2;
                    bitmapPicture                       = BitmapFactory.decodeByteArray(image, 0, image.length,options);
                    bitmapPicture                       = Bitmap.createBitmap(bitmapPicture, 0, 0, bitmapPicture.getWidth(), bitmapPicture.getHeight(),matrix, true);
                    bitmapPicture.compress(Bitmap.CompressFormat.JPEG,90, bs);
                }

                File FolderCam                      = new File(AppDirectory, "o2ocam");
                if (!FolderCam.exists()){
                    FolderCam.mkdir();
                }
                byte[] byteArray = bs.toByteArray();
                ClassImage ImgSave                          = new ClassImage();
                ImgSave.SaveBufferImage(byteArray, book_id + ".png", FolderCam);
                //END FOR SAVE CAMERA IMAGE
                */
                Mat MatSRC = Converters.vector_Point2f_to_Mat(pointsOnMain); // value of pointsOnMain is from matData.finalPoints which is i get it from the getPath function

                Bitmap bitmapimage  = BitmapFactory.decodeByteArray(bs.toByteArray(), 0, bs.toByteArray().length);
                Mat matimage = new Mat();
                Utils.bitmapToMat(bitmapimage, matimage);

                Mat MatTransform    = ProcImg.Transform(matimage, MatSRC, bitmapimage.getWidth(), bitmapimage.getHeight());
                Bitmap bitmapCrop   = bitmapimage.copy(Bitmap.Config.RGB_565, true);
                Utils.matToBitmap(MatTransform, bitmapCrop);

                ByteArrayOutputStream bos = new ByteArrayOutputStream();
                bitmapCrop.compress(Bitmap.CompressFormat.PNG, 0 /*ignored for PNG*/, bos);
                byte[] bitmapdata = bos.toByteArray();
//                ByteArrayInputStream croppedBs = new ByteArrayInputStream(bitmapdata);

                //FOR PASSING IMAGE WITH INTENT
                intentresultcam.putExtra("RESULTCAM", bs.toByteArray());
                intentresultcam.putExtra("Resolution", resolusi);
                startActivity(intentresultcam);

                if ( image != null){
                    image   = null;
                }
                if(!bitmapPicture.isRecycled()){
                    bitmapPicture.recycle();
                }
                try {
                    bs.close();
                } catch (IOException e) {}
                System.gc();
            }else{
                Toast.makeText(getApplicationContext(), "No Image", Toast.LENGTH_SHORT).show();
            }
        }
    };

i take the bs.toByteArray() as a original image.

[iyotetsuya]

You can try to use Bitmap.Config.ARGB_8888 or decrease options.inSampleSize.

[ghost]

i don't use options, u can see i don't put it on my parameter, i think it's because i didn't resize it back to the original size, can you tell me how to do it ? thank you!

[iyotetsuya]

Resize it back can't improve the quality of an image,

Do you use this way for saving bad quality photo?

byte[] byteArray = bs.toByteArray();
ClassImage ImgSave                          = new ClassImage();
ImgSave.SaveBufferImage(byteArray, book_id + ".png", FolderCam);

What actually happens when you call ImgSave.SaveBufferImage?

[ghost]

No, i don't use that. i found that i have to send bitmap to another intent, so i have to convert bitmap to stream, and because of that i have to compress the bitmap using this bm.compress(Bitmap.CompressFormat.PNG, 100, bs); but i already change this method to passing the Mat of the bitmap, but it still returning bad quality photo, it's better than before, but it's not the original image that i cropped :(

[iyotetsuya]

Hmmm...It's a little difficult for trace what's wrong with it. Can you push whole code?

[ghost]

Yes, i pushed the whole code here: https://github.com/stevian12/RectangleDetection-master

So sorry if i pushed it wrong, i'll remove it after u see it!

Thank you so much!

[iyotetsuya]

I'm busy these few days, but will check it on weekend. Thanks for your patience.

[ghost]

Thank you for your reply! yes i'll wait, thanks :)

[ghost]

oh ya sorry to interrupt you, i found out that camera quality is not good(it's different quality with the main camera), so that's why it returned bad quality photo, any chance to get the quality of original camera?

Thank you!

[iyotetsuya]

if you mean front camera(often bad quality than rear), I think there is no other way to do that. but you can check parameters.getSupportedPictureSizes() make sure you get the biggest resolution about it.

[ghost]

not the front camera, how to check it ? isn't it the function from Camera android ?

[iyotetsuya]

You can check it by this way.

int numberOfCameras = Camera.getNumberOfCameras();
for (int i = 0; i < numberOfCameras; i++) {
    Camera.CameraInfo caminfo = new Camera.CameraInfo();
    Camera.getCameraInfo(i, caminfo);
    int facing = caminfo.facing;
    if (facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
        Camera.open(i);
    }
}

[ghost]

pardon, i mean how to check parameters.getSupportedPictureSizes()

Thank you!

[iyotetsuya]

Sorry for the late reply. Maybe this link can solve your problem. http://stackoverflow.com/questions/25909899/poor-picture-quality-from-custom-camera

[iyotetsuya]

Please reopen issue if you still have problems.

[ghost]

Hello!

Thank you for being bear with me haha!

I have 1 question again if you don't mind, i'm trying to interact with Rxjava. I want to call oncompleted when i detect the rectangle, but the problem is the application keep detecting rectangle and oncompleted never called. This is my code :

Subscriber<MatData> mySubscriber = new Subscriber<MatData>() {
            @Override
            public void onNext(MatData matData) {
                if (drawView != null) {
                    if (matData.cameraPath != null) {
                        drawView.setPath(matData.cameraPath);
                        cameraData.camera.autoFocus(AutoFocus);
                    } else {
                        drawView.setPath(null);
                    }
                    drawView.invalidate();
                }
            }
            @Override
            public void onCompleted() {
                    cameraData.camera.autoFocus(AutoFocus);
        }

Thank you for your help!

[iyotetsuya]

Call Subscriber.onCompleted() instead of Subscriber. onNext(), when you detect the rectangle.

[ghost]

Hello, it took a photo. But, i have to wait like 15 seconds or even more for like stabilize then it can took a photo of it, what i'm going to do is i want in the first detection of rectangle, it directly take a photo. can be like that ?

Thank you so much!

[nikhil4567]

Hello mates , is there any efficient way where we can improve quality of bitmap we get when rectangle is detected. And any suggestions of how to make camera preview focus when rectangle is detected, so that bitmap is not clear not blurred. As in some of mobile like MI everything is working but bitmap we get is blurred, rectangle is detected without keeping focus. Stevian are you able to get good quality image?