Support HW scaler in Android

[SDLBugzilla]

This bug report was migrated from our old Bugzilla tracker.

These attachments are available in the static archive:

• Proof of concept patch (310c0fc6.diff, text/plain, 2014-10-29 21:05:19 +0000, 4483 bytes)
• Attempt to use the HW scaler in SDL (SDL_patch_HW_scaler_android.patch, text/plain, 2014-11-08 10:58:56 +0000, 2750 bytes)

Reported in version: 2.0.3 Reported for operating system, platform: Android (All), Other

Comments on the original bug report:

On 2014-10-29 00:58:33 +0000, Hak Matsuda wrote:

In Android, there is an API to utilize HW scaler to expand egl surface with pretty low cost. This feature is useful for high resolution devices but with relatively weak GPU fill rate. (for more details, please refer http://android-developers.blogspot.com/2013_09_01_archive.html)

Please consider enabling HW scaler in SDL Android. Proposed change is:

• In Android_CreateWindow(), add ANativeWindow_setBuffersGeometry() right before creating EGL surface
• Same in Java_org_libsdl_app_SDLActivity_onNativeSurfaceChanged()
• Also it's handy if an app can specify max windows size for each portrait and landscape screen separately.

On 2014-10-29 21:05:19 +0000, Hak Matsuda wrote:

Created attachment 1919 Proof of concept patch

Attached a 'proof of concept' patch for the API usage.

The patch defines 2 internal function, Android_SetScalerResolutionPortrait() and Android_SetScalerResolutionLandscape(). Each API sets values for a HW scaler sizes of portrait/landscape.

On 2014-11-07 09:50:15 +0000, Sylvain wrote:

Just a question, but I may be wrong !

Could your patch be redundant with the rendering scaling method that already exists in SDL ?

That is calling "RenderSetLogicalSize" with smaller dimensions :

SDL_RenderSetLogicalSize(renderer, smaller_width, smaller_height);

On 2014-11-07 19:09:19 +0000, Hak Matsuda wrote:

Hi Sylvain, Thanks for your comments!

In current implementation, SDL_RenderSetLogicalSize() changes logical scaling but still renders to full resolution render target.

In terms of API design perspective, the HW scaler changes 'physical' resolution from EGL view point (e.g. changing scaler setting (320, 240) -> Full screen resolution becomes 320x240). So for me, changing window size sounds feasible. (The patch itself is just a proof of concept and does not define any new behavior to existing API)

On 2014-11-08 10:46:23 +0000, Sylvain wrote:

Hi Hak,

Yes, your are totally right. And I didn't formulate my question correctly.

I use the "RenderLogicalSize" for a little different purpose : to "fake", from the application point of view, the size to actually render in the end.

My applications virtually see the render logical size, which can be smaller than the actually device size. So the application prepares a set of images that are smaller. Which is a little gain of performance.

So my question, in fact is:

let's say you have a device in full-hd: 1920x1080. there are two stages in the applications: software, then HW rendering.

Stage-1)

• When you prepare your graphics SDL_Surface for full-hd
• You create the SDL_Texture materials

Stage-2)

• You render with HW, (where the setBuffersGeometry is meaningfull)

Let's say we want to optimise by rendering in a smaller resolution. 960*540. This is possible to optimise Stage-1 and Stage-2 independently.

Let's say, Stage-1 has been optimised. The question is: Then, compared to Stage-1, is optimising Stage-2 adding a significant gain?

I think the web page is talking on optimising both Stage-1 and Stage-2 together. Whereas, in SDL, optimising Stage-1 is already possible. Anyway, optimising Stage-2 is of course a gain.

On 2014-11-08 10:58:56 +0000, Sylvain wrote:

Created attachment 1927 Attempt to use the HW scaler in SDL

I also tried you patch and it worked. I haven't been able to really notice the gain on a full-hd smart-phone. But this is because my phone is a high-end device.

I wish I could have tested is with my Nexus10 which is quite slow, but I use it for another purpose.

(I have tried to down-scale to buffer geometry a lot and that worked)

From a SDL point of view, I think the size of HW buffer could be mapped under the concept of SDL_Window size, when creating the Window.

• we could allow any size of Window, it would be fullscreen anyway, and SDL would scale the geometry buffer in consequence.

But the problem was mainly the java "surfaceChanged" that occurs some time to send new size of window, which is not the size of the resized geometry buffer. (so there is some ugly code there). Here's a path as an attempt to use the HW scaler in SDL.

On 2014-11-11 00:55:35 +0000, Hak Matsuda wrote:

Hi Sylvain, Thanks again for comments and updated patch.

Answering your question regarding Stage-1 (rendering to smaller RT and blit it) optimization vs Stage-2 (rendering to smaller RT and scale it) optimization, I would say there is a pros and cons.

Stage-1 optimization: Pros:

• Can render HUD etc. in full resolution after blit Cons:
• Need redundant blit operation to full resolution RT

Stage-2 optimization: Pros:

• Scaler operation is (usually) free of performance impact.
• (Depending on scaler implementation) can have better quality than blit in shader. (Some good HW scaler has multi tap filtering) Cons:
• Can not render HUD in full resolution

If you don't draw HUD in full resolution, then stage-2 optimization is just a plus. In high-end device, you wouldn't notice a performance difference, but still can have some gain of thermal&battery reduction.

But the problem was mainly the java "surfaceChanged" that occurs some time to send new size of window, which is not the size of the resized geometry buffer. (so there is some ugly code there). I agree. An app needs to remember and handle the scaler settings, which is ugly.

On 2017-08-14 04:42:40 +0000, Sam Lantinga wrote:

Hey Sylvain, is there a downside to accepting your patch? How does this change the current behavior on Android devices?

On 2017-08-14 11:50:37 +0000, Sylvain wrote:

This is a 3 year old patch, I will look at it again !

On 2017-08-14 11:57:57 +0000, Alex Szpakowski wrote:

I wonder if SDL should also use the hardware scaler on Android with the highdpi window flag, to make it more consistent with iOS/macOS?

i.e., if highdpi is not enabled, the surface would be scaled down so that its resolution matches the size in dp units (DPI-scaled units) of the screen, and if highdpi is enabled its size is in actual pixels.

The Android code for window size, mouse position, etc. would have to be updated to match that coordinate space as well I guess (which would be consistent with SDL's apis on other platforms).

On 2017-08-14 12:42:50 +0000, Sylvain wrote:

The more the API is consistent among platforms, the better!

I have some trouble to understand what the right behavior is.

• Should we always have fullscreen for Android ? I would say no. with newer platform, it could almost behave as a desktop. Eg creating a window at x/y (not 0/0) and w/h not fullscreen. On the other hand, I (quickly) tried to have a non-fullscreen app, and the background is always black. (even with ARGG_8888 and alpha layer transparent).

  I would use the scaler when fullscreen (SDL_WINDOW_FULLSCREEN) is requested and the width/height is not the native screen dimension.

• on android the screen size width/height comes from JAVA surfaceChanged() and it informs the SDL lib though a call to .onNativeResize().

• Currently, on android, the w/h and fullscreen are ignored. In fact, it always goes fullscreen, and the w/h is fullscreen.

I don't know what deeply HIGHDPI is. but on my smartphone S7, the resolution of the screen (surfaceChanged) reported is 1080x1920. But the screen is capable of WQHD (1440 x 2560). To have QHD enabled, I need to setup the phone (Parameter > Display > Change Screen Resolution) and even reboot the phone so that surfaceChanged reports 1440 x 2560.

[icculus]

I'm putting this in the 3.2 milestone because it was mentioned in #3519, but this isn't an endorsement that we should do this. But this bug is ancient, so if not, let's close it.