Portrait orientation on Android

[opencv-pushbot]

Transferred from http://code.opencv.org/issues/3565

|| Sergey Abdula on 2014-02-20 04:06
|| Priority: Normal
|| Affected: 2.4.0 - 2.4.7
|| Category: android
|| Tracker: Bug
|| Difficulty: Easy
|| PR: 
|| Platform: ARM / Android

Portrait orientation on Android

Hi,
If to change  android:screenOrientation from landscape to portrait mode in "OpenCVTutorial 1 - Camera Preview", an image is rotated to 270 degree clockwise.

It's pretty basic feature for mobile applications (easy change orientation) and if I understand correctly, it was discussed last year (see http://stackoverflow.com/questions/14816166/rotate-camera-preview-to-portrait-android-opencv-camera). However, I could not find any opened or closed issue related to this problem.

There is kind of way to fix it (see http://littlecheesecake.me/blog/13804736/opencv-android-orientation) but it's required to rewrite the basic class (which is not good idea) and it does not work properly anyway.

Thanks in advance.

Sergey

History

Alexander Karsakov on 2014-02-20 06:53

-   Status changed from New to Open

Alexander Smorkalov on 2014-02-25 07:24

-   Target version set to 2.4.9
-   Difficulty set to Easy

Sergey Abdula on 2014-03-14 02:43

Hi Alex,

It will be nice to have a taken picture in correct orientation in portrait mode as well.

Alexander Smorkalov on 2014-04-01 23:12

We cannot change API behavior in 2.4 branch. I reasign the task on 3.0 release (master branch). It will not be compatible with 2.4.x, so we can change API behavour and fix such issues.
-   Target version changed from 2.4.9 to 3.0

Stephen G on 2015-05-06 02:09

Found the error in calculateCameraFrameSize in CameraBridgeViewBase.

A. mMaxWidth/Height are never set and should be (easily done by overriding onSizeChanged in the CameraView and calling setMaxFrameSize)

B. the supportedSizes list only list the max hieght/widths in one dimensions. the comparisons done in the weird double if statements need to be completed for width <= maxAllowedWidth ... *&& height <= maxAllowedWidth ...*. Then, the mScale calculate also needs to take this change into account with something like this: 
<pre>
mScale = Math.max(
    Math.min(((float) height) / mFrameHeight, ((float) width) / mFrameWidth),
    Math.min(((float) width) / mFrameHeight, ((float) height) / mFrameWidth)
);
</pre>

This solution doesn't fill the whole screen, but it does keep the proper ratio.

Does it work? could it be updated for 3 since it's still in an RC state?

Andrea Tomassetti on 2015-05-19 11:37

Alexander Smorkalov wrote:
> We cannot change API behavior in 2.4 branch. I reasign the task on 3.0 release (master branch). It will not be compatible with 2.4.x, so we can change API behavour and fix such issues.

Hi, I've just downloaded OpenCV 3.0RC1 for Android but there is the same issue. I'm wrong? It was fixed?

Thanks in advance

Fabian Hoyos on 2015-07-03 16:25

Hi, would I like to know if the bug Portrait orientation on Android is fixed in the version 3.0 of opencv Android SDK. I have the same problem when I use opencv Android sdk in mode portrait in the app to open the camera and face detection. 

Thanks for information. I hope soon know the solution, please.

[hugolm84]

@hugolm84 I tried you latest code but it seems the line mCamera.setPreviewDisplay(getHolder()); prevents any image processing from onCameraFrame() from showing on the preview just like in the first suggestion by @OctavioCega . Is that not the case for you?

Did you mention me by mistake? Please refresh my memory otherwise.

[jslok]

Yep sorry, my mistake. Looks like the person I meant to tag is a deleted account.

[Ankur0506]

@OctavioCega

Great job, bro!! It works for me! i just cancelled in CameraBridgeViewVase.java from line 415 how you said

if (canvas != null) {
                canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
                if (BuildConfig.DEBUG)
                    Log.d(TAG, "mStretch value: " + mScale);
                if(getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT) {
                    Matrix matrix = new Matrix(); // I rotate it with minimal process
                    matrix.preTranslate((canvas.getWidth() - mCacheBitmap.getWidth()) / 2, (canvas.getHeight() - mCacheBitmap.getHeight()) / 2);
                    matrix.postRotate(90f,(canvas.getWidth()) / 2,(canvas.getHeight()) / 2);
                    float scale = (float) canvas.getWidth() / (float) mCacheBitmap.getHeight();
                    matrix.postScale(scale, scale, canvas.getWidth()/2 , canvas.getHeight()/2 );
                    canvas.drawBitmap(mCacheBitmap, matrix, new Paint());

                }
                else {

                    if (mScale != 0) {
                        canvas.drawBitmap(mCacheBitmap, new Rect(0, 0, mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
                                new Rect((int) ((canvas.getWidth() - mScale * mCacheBitmap.getWidth()) / 2),
                                        (int) ((canvas.getHeight() - mScale * mCacheBitmap.getHeight()) / 2),
                                        (int) ((canvas.getWidth() - mScale * mCacheBitmap.getWidth()) / 2 + mScale * mCacheBitmap.getWidth()),
                                        (int) ((canvas.getHeight() - mScale * mCacheBitmap.getHeight()) / 2 + mScale * mCacheBitmap.getHeight())), null);
                    } else {
                        canvas.drawBitmap(mCacheBitmap, new Rect(0, 0, mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
                                new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
                                        (canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
                                        (canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
                                        (canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
                    }
                } 

Thanks !!!

[master0v]

So, as a user of the OpenCV 4.3, how can I rotate the preview to portrait now, that it has been merged into the main branch?

[sencagri]

@master0v It seems it will take a very long time to fix. All of these are workaround methods. Currently, there is no official fix for this.

[master0v]

Hi sencagri. Thanks for your reply.

It looks like all the solutions above are based on transforming the captured image. The correct solution should probably focus on storing the data recorded from sensor in a correct manner staraightaway.

What are the rules for contributing to OpenCV? I would like to see if I can help solve it.

Thanks, A.

[sencagri]

@master0v https://github.com/opencv/opencv/wiki/How_to_contribute

[master0v]

Awesome, thanks!

[KlemenAndrasic]

Any solution yet on this?

[YairHanimov]

@robertomanfreda your solution looks nice however, now i can't use onCameraFrame.. for example if i want to change my camera view to grayscale, it won't change. any ideas on how to fix this?

use the old method that he posted. in the new way he said there no option of proccing i think. also change the
// mScale = Math.max(((float)height)/mFrameHeight, ((float)width)/mFrameWidth); in javacam to 1.5 or something try play with the number also this issues from 2013 i am shocked that there no really good version for that.

[AlessandroGambaro]

Okay, I think I've managed to solve every problem!

There are 2 steps to follow.

I have tested the implementations on elephones M2, Galaxy S4 and ZUK.

This seems to solve definitively any problems related to frame size and rotation of the matrix.

Repeat: I ran the bug fix on opencv 3_3_0 (currently the latest version) so I do not know if you can make these changes even to earlier versions of opencv but I suppose it is possible.

STEP 1: JavaCameraView.java

In order to make the frame fullscreen change this line of code FROM:

if ((getLayoutParams().width == LayoutParams.MATCH_PARENT) && (getLayoutParams().height == LayoutParams.MATCH_PARENT))
                        mScale = Math.min(((float) height) / mFrameHeight, ((float) width) / mFrameWidth);
                    else
                        mScale = 0;

TO:

if ((getLayoutParams().width == LayoutParams.MATCH_PARENT) && (getLayoutParams().height == LayoutParams.MATCH_PARENT))
                        mScale = Math.max(((float) height) / mFrameHeight, ((float) width) / mFrameWidth);
                    else
                        mScale = 0;

STEP 2: CameraBridgeViewBase.java In order to rotate correctly the matrix you can replace deliverAndDrawFrame method with this code:

protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
        Mat modified;

        if (mListener != null) {
            modified = mListener.onCameraFrame(frame);
        } else {
            modified = frame.rgba();
        }

        boolean bmpValid = true;
        if (modified != null) {
            try {
                Utils.matToBitmap(modified, mCacheBitmap);
            } catch(Exception e) {
                Log.e(TAG, "Mat type: " + modified);
                Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
                Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
                bmpValid = false;
            }
        }

        if (bmpValid && mCacheBitmap != null) {
            Canvas canvas = getHolder().lockCanvas();
            if (canvas != null) {
                //this is the rotation part

                if (getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT && mCameraIndex == -1) {
                    Log.d(TAG, "INDEX IS: " + mCameraIndex);
                    canvas.save();
                    canvas.rotate(90,  (canvas.getWidth()/ 2),(canvas.getHeight()/ 2));
                }

                if (getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT && mCameraIndex == 1) {
                    canvas.save();
                    canvas.rotate(270,  (canvas.getWidth()/ 2),(canvas.getHeight()/ 2));
                }

                if (mScale != 0) {
                    canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
                            new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
                                    (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
                                    (int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
                                    (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
                } else {
                    canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
                            new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
                                    (canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
                                    (canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
                                    (canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
                }

                if (mFpsMeter != null) {
                    mFpsMeter.measure();
                    mFpsMeter.draw(canvas, 20, 30);
                }
                if (getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT) {
                    canvas.restore();
                }
                getHolder().unlockCanvasAndPost(canvas);
            }
        }
    }

i'm not sure that mCameraIndex is always -1 for default camera and 1 for front camera.... i'm testing it but i think that's correct! 🗡

Let me know if my solution works for you! 🍡

Hi I've used your idea and also added this code to support reverse landscape:

if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.JELLY_BEAN_MR1) {
                    int rotation = this.getDisplay().getRotation();
                    if (getResources().getConfiguration().orientation == Configuration.ORIENTATION_LANDSCAPE &&
                            rotation == Surface.ROTATION_270) {
                        canvas.save();
                        canvas.rotate(180, (canvas.getWidth() / 2), (canvas.getHeight() / 2));
                    }
                }

Thanks, Alessandro

[KaihangHe]

The root of the problem is JavaCameraView LINE 144

List<android.hardware.Camera.Size> sizes = params.getSupportedPreviewSizes();

params.getSupportedPreviewSizes() return error size

the size[0] should be 1080w x 1920h but 1080h x 1920w

refer： https://stackoverflow.com/questions/40477253/android-how-to-use-camera-getsupportedpreviewsizes-for-portrait-orientation

[linklist2]

It's 2022, is there a detailed and perfect solution to this problem now?

[JPBotelho]

7.5 years! and still here we are

[Thullner]

Is there finally a solution?

[antonov256]

I was working on a student project when I came across this bug. It was 2014.

It's 2023 now (almost 2024), 9 years have passed. I'm senior software developer, and this bug stays unfixed 😀

That's ridiculous. Let's see if we can see some progress after Elon Musk mission to Mars will be accomplished.

[antonov256]

To be fair, no one of us, guys, pays for the development. That's why we can't demand a bug fix.

[MuhammadFurqan786]

i got a solution by changing in deliverAndDrawFrame in CameraBridgeViewBase.java convert this

                 if (mScale != 0) {
                     canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
                     new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
                     (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
                     (int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
                     (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 +mScale*mCacheBitmap.getHeight())),null);
            }

to this

            if (mScale != 0) {

                canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
                        new Rect(0,0,canvas.getWidth(),canvas.getHeight()) ,null);
                }