API confusion: the normalization range of the SDL_TouchFingerEvent coordinates should possibly be documented better

[ell1e]

Okay so I did not realize the 0...1 coordinates given by SDL_TouchFingerEvent already map to the window indicated by windowID rather than the whole screen/touch device extents, since the wiki doesn't really specify it. So this is mostly a documentation problem.

It appears to be impossible to reliably obtain the pixel coordinates of SDL_TouchFingerEvent in all scenarios like complex multi window multi screen settings, especially relating to a window when windowID is set.

These are the solutions I can think of, and I recommend implementing all of them:

- The SDL_FINGERMOTION/SDL_FINGERDOWN/SDL_FINGERUP should have members with local pixel coordinates relating to windowID if set. For this purpose, I recommend adding int32_t mx and int32_t my to the end of the SDL_TouchFingerEvent struct. They can be set to -1 if windowID == 0, otherwise should be the coordinates relative to the top-left of the window. For High-DPI they should be points like the mouse event coordinates are.

- There should be a way to get the point dimensions of a device type and it's high dpi scale factor in the public API, e.g. int SDL_GetTouchDeviceSize(SDL_TouchID touchID, uint32_t *w, unt32_t *h); and int SDL_GetTouchDeviceDPIScaler(SDL_TouchID touchID, double *scale); (the latter could also be replaced by a way to get the corresponding display which then needs a new int SDL_GetDisplayDPIScaler(int displayIndex, double *scaler);)

Please note the second suggestion in itself is likely useless when the user is trying to manually translate with this info to a window that is partially on two displays that have different DPI, since then I assume it is impossible to know how exactly it translates to window coordinates. Therefore, I think both recommendations should be implemented.

Currently, without any of these, it seems to be impossible for any multi touch application to e.g. have a sprite follow each finger if it is windowed since there appears to be no way to translate to the window coordinates (since x and y members of SDL_TouchFingerEvent are over the entire surface of the touch device which usually corresponds to an unknown(!) one of the displays, so in windowed mode this is very disjointed from the windows). So it would be great if this could be added some time soon, for the apparent lack of any alternatives. I guess one can try to hack back to the display via windowID, but this breaks down again when a window is partially on multiple displays, and it's pretty complicated on top of things.

[cgutman]

since x and y members of SDL_TouchFingerEvent are over the entire surface of the touch device which usually corresponds to an unknown(!) one of the displays, so in windowed mode this is very disjointed from the windows

I'm not sure this is true (at least for SDL_GetTouchDeviceType() == SDL_TOUCH_DEVICE_DIRECT).

Windows backend normalizes to window coordinates: https://github.com/libsdl-org/SDL/blob/a4ddb175f1f1d832960c830191daaab7eb25638f/src/video/windows/SDL_windowsevents.c#L1108-L1120

X11 does too: https://github.com/libsdl-org/SDL/blob/bd06538778102f72bad8393ef07da5a1ec444217/src/video/x11/SDL_x11xinput2.c#L231-L233

I assume you're seeing events that aren't using window coordinates. What backend are you using?

[ell1e]

@cgutman I just assumed they aren't normalized to the window coordinates because that's not what the wiki says here: https://wiki.libsdl.org/SDL_TouchFingerEvent so if that is the case then that probably needs updating. So if I understood you right, given event->tfinger.windowID != 0, then dx and dy will be normalized to the point size of the window's inner (non-decorated) area? Is that what is happening? (And I assume then with windowID == 0 it's normalized to the touch device/corresponding screen as before?)

[cgutman]

Yeah, the docs on the wiki for this leave much to be desired, but window-relative is what I see.

What I'm doing in my program with SDL_TouchFingerEvent basically boils down to:

SDL_GetWindowSize(window, &windowWidth, &windowHeight);
x = event->tfinger.x * windowWidth;
y = event->tfinger.y * windowHeight;

I'm not reading windowID because I manage only a single window, but that should work for multi-window applications like yours. I'm not sure under what practical circumstances you could get an ID of 0, but you certainly can't do anything sensible in a multi-window application with that kind of event.

One other thing to note (also not clear in the documentation) is that it is possible for x and y to exceed 0..1 range. They are normalized to 0..1 but not clamped to it. This can happen because some OSes track touch events outside the window area if they start within the window area.

FWIW, I wrote myself the following comment in my code:

    // Observations on Windows 10: x and y appear to be relative to 0,0 of the window client area.
    // Although SDL documentation states they are 0.0 - 1.0 float values, they can actually be higher
    // or lower than those values as touch events continue for touches started within the client area that
    // leave the client area during a drag motion.
    // dx and dy are deltas from the last touch event, not the first touch down.

[ell1e]

I'm not sure under what practical circumstances you could get an ID of 0, but you certainly can't do anything sensible in a multi-window application with that kind of event.

I assume that happens in windowed mode when a tap starts outside any window, haven't tested though. And yes, in a multi window application outside of screen gestures (which don't need window coordinates) it's useless, so I do drop those.

How does one get a wiki account again? I should fix this, since apparently this is mostly a documentation issue

[icculus]

(Also: the 2D render API catches finger touch events and scales them to match the logical render size, adjusting for letterboxing and such, and I'm not sure that's documented anywhere yet.)