SDL & iOS: When using 'SDL_WINDOW_ALLOW_HIGHDPI' everything is rendered to 3x smaller window and touch doesn't align what's being touched on screen

[pzoltowski]

---

Dear ImGui 1.89 WIP (18808)
--------------------------------

sizeof(size_t): 8, sizeof(ImDrawIdx): 2, sizeof(ImDrawVert): 20
define: __cplusplus=201703
define: __APPLE__
define: __GNUC__=4
define: __clang_version__=13.0.0 (clang-1300.0.29.30)
--------------------------------

io.BackendPlatformName: imgui_impl_sdl
io.BackendRendererName: imgui_impl_metal
io.ConfigFlags: 0x00000000
io.ConfigMacOSXBehaviors
io.ConfigInputTextCursorBlink
io.ConfigWindowsResizeFromEdges
io.ConfigMemoryCompactTimer = 60.0
io.BackendFlags: 0x0000000E
 HasMouseCursors
 HasSetMousePos
 RendererHasVtxOffset
--------------------------------

io.Fonts: 1 fonts, Flags: 0x00000000, TexSize: 512,1024
io.DisplaySize: 375.00,812.00
io.DisplayFramebufferScale: 1.00,1.00
--------------------------------

style.WindowPadding: 24.00,24.00
style.WindowBorderSize: 1.00
style.FramePadding: 12.00,9.00
style.FrameRounding: 0.00
style.FrameBorderSize: 0.00
style.ItemSpacing: 24.00,12.00
style.ItemInnerSpacing: 12.00,12.00

Version/Branch of Dear ImGui:

Version: latest from master (but v1.88 also the same issue) Branch: master

Back-end/Renderer/Compiler/OS

Back-ends: imgui_impl_sdl.cpp + imgui_impl_metal.mm Compiler: buildin from Xcode 13.2.1 (MacOS 11.6 Big Sur) Operating System: iOS 15.5 (iPhone XS)

My Issue/Question:

I'm trying to make some good looking example for iOS using SDL + metal. When using 'SDL_WINDOW_ALLOW_HIGHDPI' and following FAQ re DPI, text is crispy seems properly rendered 3x but it's only rendered to smaller viewport (3x smaller), also touch input is not aligned with what is rendered (touch behaves like everything is rendered properly to fullscreen but e.g. need to slide on right edge of my iphone to scroll small rendered window).

I used slightly modified example from examples/example_sdl_metal.mm (just loading custom font, showing only demo window, and tried different SDL_CreateWindow). Used the latest SDL (2.32.2). Small modification for ShowDemoWindow below (just to render fullscreen and some offset for notch):

    // We specify a default position/size in case there's no data in the .ini file.
    // We only do it to make the demo applications a little more welcoming, but typically this isn't required.
//    const ImGuiViewport* main_viewport = ImGui::GetMainViewport();
//    ImGui::SetNextWindowPos(ImVec2(main_viewport->WorkPos.x + 650, main_viewport->WorkPos.y + 20), ImGuiCond_FirstUseEver);
//    ImGui::SetNextWindowSize(ImVec2(550, 680), ImGuiCond_FirstUseEver);

    static bool use_work_area = true;
    window_flags = 0;
    window_flags = ImGuiWindowFlags_NoMove | ImGuiWindowFlags_NoResize | ImGuiWindowFlags_NoSavedSettings;

    auto frame_height = ImGui::GetFrameHeight();
    auto font_size = ImGui::GetFontSize();

    const ImGuiViewport* viewport = ImGui::GetMainViewport();
    float notch_offset = 40.0f;
    float edge_offset = 10.0f;
    ImVec2 canvas_position = viewport->WorkPos;
    ImVec2 canvas_size = viewport->WorkSize;

// this below didn't help either
//    canvas_size.x = 1125;
//    canvas_size.y = 2436;

    canvas_position.y += notch_offset;
    canvas_size.y -= notch_offset;
    canvas_size.y -= edge_offset;

    canvas_position.x += edge_offset;
    canvas_size.x -= 2*edge_offset;

Some investigation:

  ImGui::StyleColorsDark();

  float _size = 13*3;
  NSString* bundlePath = [NSBundle mainBundle].bundlePath;
  NSString* fontPath = [NSString stringWithFormat: @"%@/%@", bundlePath, @"/telegrama_render.otf"];    
  io.Fonts->AddFontFromFileTTF([fontPath UTF8String], _size);

  ImGuiStyle *style = &ImGui::GetStyle();
  style->ScaleAllSizes(3.0f);

  //... any of window below have the same issue and same results

  SDL_Window* window = SDL_CreateWindow("Dear ImGui SDL+Metal example", SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED, 1125, 2436, SDL_WINDOW_ALLOW_HIGHDPI | SDL_WINDOW_FULLSCREEN | SDL_WINDOW_METAL);

  //...or 

  SDL_Window* window = SDL_CreateWindow("Dear ImGui SDL+Metal example", SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED, 375, 812, SDL_WINDOW_RESIZABLE | SDL_WINDOW_ALLOW_HIGHDPI);

  //... or 

  SDL_Window* window = SDL_CreateWindow("Dear ImGui SDL+Metal example", SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED, 1125, 2436, SDL_WINDOW_RESIZABLE | SDL_WINDOW_ALLOW_HIGHDPI);

Result (screenshot 1):

• font are sharp,
• everything seems rendered proper size and sharp
• viewport(main screen) is 3x smaller
• touch inputs doesn't match what's being rendered (seems like touch uses all screen coordinates but need to touch something outside of visible to screen to luckly open some accordion)

Printed debug values:

  SDL_GetRendererOutputSize: 1125 2436
  SDL_GetWindowSize: 375 812
  SDL_GL_GetDrawableSize: 375 812
  ImGui::GetFrameHeight(): 57
  ImGui::GetFontSize(): 39
  ImGui::GetMainViewport()->WorkSize: 375 812

if I hardcode void ImGui_ImplSDL2_NewFrame() to give bigger display_w and display_h:

  SDL_GL_GetDrawableSize(bd->Window, &display_w, &display_h);
  display_w *= 3;
  display_h *= 3;

Result (screenshot 2):

• everything is rendered full screen size
• touch events matches what's being rendered
• everyting seems again just scaled and not sharp

if I turn off SDL_WINDOW_ALLOW_HIGHDPI everything is rendered fullscreen but not sharp anymore (upscaled). (screenshot 3)

**Screenshot

[pzoltowski]

Also not sure if this is because of some mismatch between SDL on iOS and on desktop. They have some notes here: Notes -- Retina / High-DPI and window sizes

[pzoltowski]

I tried also the following combo ios+sdl+opengl3 (had to force via #define IMGUI_IMPL_OPENGL_ES2 1 in imconfig.h and add #include <OpenGLES/ES2/gl.h> in 'example_sdl_opengl3/main.mmto make it compile), but the result it's the same - couldn't find a way to make it with high dpi enabled. I noticed officialexample_apple_metal` example also has the same issue.

[olinorwell]

Has anyone found a suitable workaround for this? The same issues still occur. IMGUI+SDL2 on iOS can't cope with the high DPI setting. The issue of course being that it's a setting you need if you want a sensible looking app on iOS. (i.e. not blurred).

If anyone could give some hints I could maybe delve in deeper.

To confirm, the rendering is fine, it's just the input that's the problem.

The issue appears to be that the input XY range is perhaps connected to the rendering resolution, but with this setting it needs to be connected to the actual window size as reported by SDL.

[olinorwell]

I have been able to get it to work. In ImGui_ImplSDL2_ProcessEvent I disabled SDL_MOUSEBUTTONDOWN and SDL_MOUSEBUTTONUP cases when running on iOS, I then wrote cases for SDL_FINGERDOWN and SDL_FINGERUP which effectively do io.AddMouseButtonEvent(0, true); and io.AddMouseButtonEvent(0, false); respectively.

This does appear to work and now I can have SDL using HIGHDPI mode on iOS (which is effectively the sensible default) and have IMGUI work correctly. No idea why it wasn't working. Hope that helped someone out there.