Open Nanokoli opened 9 months ago
ocornut
commented
9 months ago No, it doesn’t support multi-touch, and will not, in the sense that it won’t be possible to interact with multiple widgets simultaneously.
However, if the aim is to translate multi-touch events to specific high levels events (eg: scrolling, zooming) then that would be a possibility to develop on a backend case by case basis.
Nanokoli
commented
9 months ago all right,maybe i should focus on the last finger put on the screen,the widgets which was touched has the id of touch data after touched
ocornut
commented
9 months ago Yes I believe that's a reasonable workaround. Most of our backend don't explicitly support touch and rely on underlying layer (e.g. GLFW, SDL) to translate touch to mouse so the details of that handling would be backend specific, but could be reworked.
basharast
commented
8 months ago I had few challenges regarding touch in general, including multi-touch so I'm posting this and I hope it's related and helpful for those looking for multi-touch solution, I do believe it cannot easily solved from imgui side since each device and system has it's own way to translate touch events the main points I faced were:
the solution was not very complicated, my project was C++/UWP so the system will report each point with specific id/index which is what you only want to track the points. the way I used to check whether the touch is on the target element was to calculate that manually: it will help to opt-out imgui native way for events and simulate some other events from your side.
bool IsPointInButton(const ImVec2& point, const ImVec2& buttonPos, const ImVec2& buttonSize)
{
return (point.x >= buttonPos.x && point.x <= buttonPos.x + buttonSize.x && point.y >= buttonPos.y &&
point.y <= buttonPos.y + buttonSize.y);
}bool IsPointInCircle(const ImVec2& point, const ImVec2& center, float radius)
{
float distance = sqrt((point.x - center.x) * (point.x - center.x) + (point.y - center.y) * (point.y - center.y));
return distance <= radius;
}and there is a list to keep pointers status updated so I can check each of them while render
struct input_pointer
{
int id;
float x;
float y;
bool isInContact;
};
std::list<input_pointer> pointers{};
input_pointer getTouch(unsigned index)
{
input_pointer pointer{};
if (index < pointers.size())
{
auto l_front = pointers.begin();
std::advance(l_front, index);
}
return pointer;
}
void addOrUpdatePointer(input_pointer pointer)
{
for (auto pItem = pointers.begin(), end = pointers.end(); pItem != end; ++pItem)
{
if (pItem->id == pointer.id)
{
pItem->x = pointer.x;
pItem->y = pointer.y;
pItem->isInContact = pointer.isInContact;
return;
}
}
pointers.push_back(pointer);
}
void removePointer(int id)
{
pointers.erase(std::remove_if(pointers.begin(), pointers.end(), [&](const input_pointer& x) { return x.id == id; }),
pointers.end());
}Now my app events will always report to addOrUpdatePointer and removePointer
and while render I keep track points status:
for (auto sItem = pointers.begin(); sItem != pointers.end(); ++sItem)
{
// Get the touch point's coordinates
float touchX = sItem->x;
float touchY = sItem->y;
// Translate touch point's coordinates to ImGui window space
ImVec2 touchPos = ImVec2(touchX, touchY);
// Do calculation here
}This simple example that I worked on, and keep in mind I don't have good experience in C++ so some lines can be re-written in better way. there is also a way to improve the accuracy so you don't lose the touch if it wasn't exactly on the target by small margin. based on the project requirements you could just opt-out any point that shouldn't be reported to imgui. or simulate the position of the target movement.
digitalsignalperson
commented
6 months ago In a given backend implementation, could multitouch (e.g. two fingers controlling two sliders independently) be approximated by just sequencing the platform's touch events with interleaving calls to ImGuiIO_AddMousePosEvent and ImGuiIO_AddMouseButtonEvent, with saving/restoring initial touch positions and restoring drag position? Or do things break if there is so many up/down/move events within like 1 frame?
digitalsignalperson
commented
6 months ago I have a janky POC using sokol's imgui-sapp.cc demo. I have two sliders (boosted size with ImGui::GetStyle().ScaleAllSizes(10.0f);) and with one finger on each I can change both values simultaneously. But the animation has some lag to it if you move quickly (maybe that can be sorted out somehow like letting each touch point having exclusive control for 1-2 frames).
The process is roughly:
Aside from laggy animations, a caveat is that when dragging a slider, any slightly dragging away from the slider bounding box results in our saving/restoring touch positions restoring the touch point outside the slider, and resulting in dragging the window. A few solutions to this:
For now I did a temporary hack only for horizontal sliders: let the new touch down point be the initial touch down y position, and the new event's x position
Code:
static struct {
sg_pass_action pass_action;
sapp_touchpoint touches_init[SAPP_MAX_TOUCHPOINTS];
sapp_touchpoint touches_drag[SAPP_MAX_TOUCHPOINTS];
bool touch_active[SAPP_MAX_TOUCHPOINTS];
int id_active_down = -1;
} state;
...
static bool is_touch_event(sapp_event_type t) {
switch (t) {
case SAPP_EVENTTYPE_TOUCHES_BEGAN:
case SAPP_EVENTTYPE_TOUCHES_MOVED:
case SAPP_EVENTTYPE_TOUCHES_ENDED:
case SAPP_EVENTTYPE_TOUCHES_CANCELLED:
return true;
default:
return false;
}
}
void mylog(const char * str, uint32_t lineOrNumber) {
slog_func("mylog", 2, 0, str, lineOrNumber, nullptr, nullptr);
}
void input(const sapp_event* event) {
const sapp_event& ev = *event;
if (is_touch_event(event->type)) {
if (event->type == SAPP_EVENTTYPE_TOUCHES_BEGAN) {
for (int i = 0; i < ev.num_touches; i++) {
int id = ev.touches[i].identifier;
if (!state.touch_active[id]) {
state.touches_init[id] = ev.touches[i];
state.touches_drag[id] = ev.touches[i];
state.touch_active[id] = true;
// raise the last active touch
if (state.id_active_down != -1) {
mylog("BEGIN: Raising last id", state.id_active_down);
sapp_event ev_touch_up = ev;
ev_touch_up.type = SAPP_EVENTTYPE_TOUCHES_ENDED;
// NOTE: sokol's simgui_handle_event only passes the touch data in event.touches[0], so always set the data there
ev_touch_up.touches[0] = state.touches_drag[state.id_active_down];
simgui_handle_event(&ev_touch_up);
}
// lower the new active touch
{
sapp_event ev_touch_dn = ev;
ev_touch_dn.touches[0] = ev.touches[i];
simgui_handle_event(&ev_touch_dn);
mylog("BEGIN: Lowering new active touch id", id);
}
state.id_active_down = id;
}
}
}
if (event->type == SAPP_EVENTTYPE_TOUCHES_MOVED) {
// First do the move event for the touch that is already down
for (int i = 0; i < ev.num_touches; i++) {
int id = ev.touches[i].identifier;
if (id == state.id_active_down) {
// mylog("MOVED: Moving last id", state.id_active_down);
sapp_event ev_touch_move = ev;
ev_touch_move.touches[0] = ev.touches[i];
state.touches_drag[id] = ev.touches[i];
simgui_handle_event(&ev_touch_move);
}
}
// Now for any other events, raise and lower
int orig_id_active_down = state.id_active_down;
for (int i = 0; i < ev.num_touches; i++) {
int id = ev.touches[i].identifier;
if (id != orig_id_active_down) {
// raise the active down at the new position
if (state.id_active_down != -1) {
mylog("MOVED: Raising last id", state.id_active_down);
sapp_event ev_touch_up = ev;
ev_touch_up.type = SAPP_EVENTTYPE_TOUCHES_ENDED;
ev_touch_up.touches[0] = state.touches_drag[state.id_active_down];
simgui_handle_event(&ev_touch_up);
}
// restore the other touch active down to either original position (trick to maintain drag distance?), or use last position if that's janky
{
sapp_event ev_touch_dn = ev;
ev_touch_dn.type = SAPP_EVENTTYPE_TOUCHES_BEGAN;
// ev_touch_dn.touches[0] = state.touches_init[id]; // original - it's necessary or else we'll probably drag the window around
// ev_touch_dn.touches[0] = ev.touches[i]; // new position
// ev_touch_dn.touches[0] = state.touches_drag[id]; // last position
// hack for POC: Assume all controls are horizontal
ev_touch_dn.touches[0] = {
.pos_x = state.touches_drag[id].pos_x, // last x position
.pos_y = state.touches_init[id].pos_y // original y position
};
simgui_handle_event(&ev_touch_dn);
}
// now move it to the new position
{
sapp_event ev_touch_move = ev;
ev_touch_move.touches[0] = ev.touches[i];
simgui_handle_event(&ev_touch_move);
state.touches_drag[id] = ev.touches[i];
}
mylog("MOVED: Lowered and moved id", id);
// and mark this as the active down touch for the next one
state.id_active_down = id;
}
}
}
if (event->type == SAPP_EVENTTYPE_TOUCHES_ENDED || event->type == SAPP_EVENTTYPE_TOUCHES_CANCELLED) {
for (int i = 0; i < ev.num_touches; i++) {
int id = ev.touches[i].identifier;
if (id == state.id_active_down) {
mylog("ENDED/CANCELED: raising id", id);
sapp_event ev_touch_up = ev;
ev_touch_up.touches[0] = ev.touches[i];
simgui_handle_event(&ev_touch_up);
state.id_active_down = -1;
}
state.touch_active[id] = false;
}
}
} else {
simgui_handle_event(event);
}
}
}```? Or do things break if there is so many up/down/move events within like 1 frame?
it’s going to grow the inputs queue, create delays in processing. Not mentioning the constant activation back and forth are going to be a problem.
If you don’t need full fledged dear imgui features, it may be more reasonable to create custom basic UI logic buttons sliders supporting it.
Nanokoli
commented
6 months ago multitouch may be not multiple
digitalsignalperson
commented
6 months ago Hehehe it works pretty smoothly! I added a MAX_CONSECUTIVE_EVENTS to stagger events a bit and looks good to me :)
Demo:
https://github.com/ocornut/imgui/assets/45611618/81c32813-502c-4a72-8013-0699106d1d84
If you don’t need full fledged dear imgui features, it may be more reasonable to create custom basic UI logic buttons sliders supporting it.
Yeah I agree it's probably easier to make something custom that lives alongside imgui, though this is kind of fun to try and make work for the sake of it.
Patch for my previous snippet:
@@ -89,8 +89,11 @@ static struct {
sapp_touchpoint touches_drag[SAPP_MAX_TOUCHPOINTS];
bool touch_active[SAPP_MAX_TOUCHPOINTS];
int id_active_down = -1;
+ int consecutive_events;
} state;
+#define MAX_CONSECUTIVE_EVENTS 5
+
static void reset_minmax_frametimes(void) {
state.max_raw_frame_time = 0;
state.min_raw_frame_time = 1000.0;
@@ -276,10 +279,12 @@ void input(const sapp_event* event) {
mylog("BEGIN: Lowering new active touch id", id);
}
state.id_active_down = id;
+ state.consecutive_events = 0;
}
}
}
if (event->type == SAPP_EVENTTYPE_TOUCHES_MOVED) {
+ if (ev.num_touches == 1 || state.consecutive_events < MAX_CONSECUTIVE_EVENTS) {
// First do the move event for the touch that is already down
for (int i = 0; i < ev.num_touches; i++) {
int id = ev.touches[i].identifier;
@@ -289,9 +294,12 @@ void input(const sapp_event* event) {
ev_touch_move.touches[0] = ev.touches[i];
state.touches_drag[id] = ev.touches[i];
simgui_handle_event(&ev_touch_move);
+ state.consecutive_events++;
+ }
}
}
// Now for any other events, raise and lower
+ if (state.id_active_down == -1 || state.consecutive_events >= MAX_CONSECUTIVE_EVENTS) {
int orig_id_active_down = state.id_active_down;
for (int i = 0; i < ev.num_touches; i++) {
int id = ev.touches[i].identifier;
@@ -330,6 +338,8 @@ void input(const sapp_event* event) {
mylog("MOVED: Lowered and moved id", id);
// and mark this as the active down touch for the next one
state.id_active_down = id;
+ state.consecutive_events = 0;
+ }
}
}
}Hehehe it works pretty smoothly! I added a
MAX_CONSECUTIVE_EVENTSto stagger events a bit and looks good to me :)Demo:
https://github.com/ocornut/imgui/assets/45611618/81c32813-502c-4a72-8013-0699106d1d84
If you don’t need full fledged dear imgui features, it may be more reasonable to create custom basic UI logic buttons sliders supporting it.
Yeah I agree it's probably easier to make something custom that lives alongside imgui, though this is kind of fun to try and make work for the sake of it.
Patch for my previous snippet:
@@ -89,8 +89,11 @@ static struct { sapp_touchpoint touches_drag[SAPP_MAX_TOUCHPOINTS]; bool touch_active[SAPP_MAX_TOUCHPOINTS]; int id_active_down = -1; + int consecutive_events; } state; +#define MAX_CONSECUTIVE_EVENTS 5 + static void reset_minmax_frametimes(void) { state.max_raw_frame_time = 0; state.min_raw_frame_time = 1000.0; @@ -276,10 +279,12 @@ void input(const sapp_event* event) { mylog("BEGIN: Lowering new active touch id", id); } state.id_active_down = id; + state.consecutive_events = 0; } } } if (event->type == SAPP_EVENTTYPE_TOUCHES_MOVED) { + if (ev.num_touches == 1 || state.consecutive_events < MAX_CONSECUTIVE_EVENTS) { // First do the move event for the touch that is already down for (int i = 0; i < ev.num_touches; i++) { int id = ev.touches[i].identifier; @@ -289,9 +294,12 @@ void input(const sapp_event* event) { ev_touch_move.touches[0] = ev.touches[i]; state.touches_drag[id] = ev.touches[i]; simgui_handle_event(&ev_touch_move); + state.consecutive_events++; + } } } // Now for any other events, raise and lower + if (state.id_active_down == -1 || state.consecutive_events >= MAX_CONSECUTIVE_EVENTS) { int orig_id_active_down = state.id_active_down; for (int i = 0; i < ev.num_touches; i++) { int id = ev.touches[i].identifier; @@ -330,6 +338,8 @@ void input(const sapp_event* event) { mylog("MOVED: Lowered and moved id", id); // and mark this as the active down touch for the next one state.id_active_down = id; + state.consecutive_events = 0; + } } } }
It look like to switch the mouse place quickly and do call IMGUI's Input to simulate multiple interaction for widgets.May rapid changes in position cause anomalies in the value of Δ.and cause some iiuse?
digitalsignalperson
commented
6 months ago It look like to switch the mouse place quickly and do call IMGUI's Input to simulate multiple interaction for widgets.May rapid changes in position cause anomalies in the value of Δ.and cause some iiuse?
yes, it is kind alternating between controlling the different touch points very quickly
Here's the demo if you want to try: https://github.com/floooh/sokol-samples/compare/master...digitalsignalperson:sokol-samples:master
This line is the part that needs improvement: https://github.com/digitalsignalperson/sokol-samples/blob/81d2964bfe28bd6b71b163336b2fca48a68ca344/sapp/imgui-sapp.cc#L240
Based on looking at the imgui demo, I see in the metrics/debugger window NavLastIds[0] can kind of be used to get a bounding box on the widget the first touch hits, but for a slider it includes the label too. A map of the widget ids could always be created to register them with the multitouch code.
Playing around with it as-is on some horizontal sliders, occasionally moving my fingers around still somehow results in me moving the window and weirdness.
But again, this is probably just for fun, but idk if it works good enough I might just use it.
Nanokoli
commented
4 months ago https://github.com/user-attachments/assets/8c8c69e9-0916-42ef-bddb-82a3a4b1853d
It works nice，i use a struct to control the figer's capture status，fun OnTouch only translate the last fighter's info to input imgui
and then every widget just need to maintenance their TouchCapturelist to read their needs，jsut like
It look like to switch the mouse place quickly and do call IMGUI's Input to simulate multiple interaction for widgets.May rapid changes in position cause anomalies in the value of Δ.and cause some iiuse?
yes, it is kind alternating between controlling the different touch points very quickly
Here's the demo if you want to try: floooh/sokol-samples@master...digitalsignalperson:sokol-samples:master
This line is the part that needs improvement: https://github.com/digitalsignalperson/sokol-samples/blob/81d2964bfe28bd6b71b163336b2fca48a68ca344/sapp/imgui-sapp.cc#L240
Based on looking at the imgui demo, I see in the metrics/debugger window NavLastIds[0] can kind of be used to get a bounding box on the widget the first touch hits, but for a slider it includes the label too. A map of the widget ids could always be created to register them with the multitouch code.
Playing around with it as-is on some horizontal sliders, occasionally moving my fingers around still somehow results in me moving the window and weirdness.
But again, this is probably just for fun, but idk if it works good enough I might just use it.
Version/Branch of Dear ImGui:
Version 1.XX, Branch: XXX (master/docking/etc.)
Back-ends:
imgui_impl_XXX.cpp + imgui_impl_XXX.cpp
Compiler, OS:
Android
Full config/build information:
No response
Details:
I try to use it in Android As UI module,It Works Nicely,but i need two finger to control The applicant,I can get the info of every fingers,but i dont know how to input them to the imgui system,IMGUI Have one only mouse and nothing about Multi-Touch,I try to fix ImGuiMouseButton_COUNT to enable my ideas,but It seem unsafe,
Screenshots/Video:
No response
Minimal, Complete and Verifiable Example code: