Phaengris / fusuma-plugin-touchscreen

The Fusuma plugin for touchscreens
MIT License
5 stars 0 forks source link

Rotation support #1

Open m4z-0xf00 opened 1 year ago

m4z-0xf00 commented 1 year ago

Im looking a way to read the rotation gesture for a touchscreen radial menu and you said features requests are welcome so there we go.

Thanks for the work you're doing, can't help you for the code because I dont know how to write in ruby. I can provide some testing if you need, I have a touchscreen on my laptop so I can try stuff for you. I've seen the spec folder in the draft, I'm not sure what it does, but if you need mine you just got to ask.

Phaengris commented 1 year ago

Rotate support is a planned feature, yes. Now the draft supports tap / swipe, then I plan to implement hold, then pinch / rotate.

The spec folder contains auto tests (or should contain, at least :see_no_evil: when I add them).

@m4z-0xf00 what do you mean "if you need mine"? Is about specs?

Testing is appreciated! But unfortunately it can't be installed just with "gem install fusuma-plugin-touchscreen" as Fusuma itself, because of it's not released as a gem yet... There's a way to run it locally, but it requires a bit of Ruby skills.

m4z-0xf00 commented 1 year ago

Yes I meant about specs, Im not a native speaker so there are chances my sentences sound weird ^^ Anyways can you quickly explain to me the plugin stuff for fusuma, do you just have to explain where to get the data or do you need to implement the math and detection stuff ? About the testing I figured it wasn't ready, but it's ok good stuff needs time to build, don't hesitate to tell when you want me to test.

Phaengris commented 1 year ago

Yes, I have to implement the math. It's already done for swipes. Need to think how to do the math for pinches / rotations. Have some ideas, need to experiment :)

m4z-0xf00 commented 1 year ago

Assuming you have a list a all touch points, you would need to find a way to reliably track the fingers without them switching (when too close for example) then with that you can calculate the distance/angle between the fingers using vector math. The difficult part to me is the tracking as the touch points may not be tracked well by the system and some touch points can disappear. Also an user may be touching the screen with the palm or with more fingers than expected. I suppose some kind of delay could be used or you can have a data structure representing fingers that you can update when you receive tracking data. Maybe comparing the deltas between the new data and the old can help too.

Phaengris commented 1 year ago

If you still need it, this is the implementation I came to in the end. It's very simple and may not handle corner cases well, but for now it works. https://github.com/Phaengris/fusuma-plugin-touchscreen/blob/draft/lib/fusuma/plugin/detectors/touch_detectors/rotate_detector.rb

m4z-0xf00 commented 1 year ago

Hello, sorry I've been away for some time, I manged to install the plugin, though I'm not even sure fusuma sees it. This is the list I get on load:

Enabled Plugins: Fusuma::Plugin::Buffers::GestureBuffer Fusuma::Plugin::Buffers::KeypressBuffer Fusuma::Plugin::Buffers::TapBuffer Fusuma::Plugin::Buffers::TimerBuffer Fusuma::Plugin::Detectors::HoldDetector Fusuma::Plugin::Detectors::KeypressDetector Fusuma::Plugin::Detectors::PinchDetector Fusuma::Plugin::Detectors::RotateDetector Fusuma::Plugin::Detectors::SwipeDetector Fusuma::Plugin::Detectors::TapDetector Fusuma::Plugin::Events::Records::ContextRecord Fusuma::Plugin::Events::Records::GestureRecord Fusuma::Plugin::Events::Records::IndexRecord Fusuma::Plugin::Events::Records::KeypressRecord Fusuma::Plugin::Events::Records::TextRecord Fusuma::Plugin::Executors::CommandExecutor Fusuma::Plugin::Executors::SendkeyExecutor Fusuma::Plugin::Filters::KeypressFilter Fusuma::Plugin::Filters::LibinputDeviceFilter Fusuma::Plugin::Inputs::LibinputCommandInput Fusuma::Plugin::Inputs::TimerInput Fusuma::Plugin::Parsers::KeypressParser Fusuma::Plugin::Parsers::LibinputGestureParser Fusuma::Plugin::Parsers::TapParser

The thing is I'm not really sure if I know how to load the plugin, is there something to add in the config ? I have swipe and rotate sections in it but I don't know if it's enough. Could you perhaps share an example config ?

m4z-0xf00 commented 1 year ago

Update:

D, [2023-08-03T10:59:09.224088 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_FRAME +218.810s } D, [2023-08-03T10:59:09.228623 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_MOTION +218.816s 0 (0) 79.79/29.96 (251.38/51.12mm)} D, [2023-08-03T10:59:09.229239 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_MOTION +218.816s 1 (1) 74.19/44.24 (233.77/75.50mm)} D, [2023-08-03T10:59:09.229756 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_MOTION +218.816s 2 (2) 74.12/35.11 (233.54/59.92mm)} D, [2023-08-03T10:59:09.230371 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_FRAME +218.816s }

m4z-0xf00 commented 1 year ago

Update:


D, [2023-08-03T11:19:11.973557 #2258283] DEBUG -- : {:input_event=>time: 2023-08-03 11:19:11 +0200, tag: libinput_command_input, record:  event4   TOUCH_FRAME             +0.142s  }
D, [2023-08-03T11:19:11.973805 #2258283] DEBUG -- : Fusuma::Plugin::Parsers::TouchParser#parse_record
D, [2023-08-03T11:19:11.973879 #2258283] DEBUG -- :   record = #<Fusuma::Plugin::Events::Records::TextRecord:0x00005647f8efd198 @text=" event4   TOUCH_FRAME             +0.142s\t">
D, [2023-08-03T11:19:11.975017 #2258283] DEBUG -- : {:input_event=>time: 2023-08-03 11:19:11 +0200, tag: libinput_command_input, record:  event4   TOUCH_MOTION            +0.146s  0 (0) 53.61/34.45 (168.92/58.79mm)}
D, [2023-08-03T11:19:11.975274 #2258283] DEBUG -- : Fusuma::Plugin::Parsers::TouchParser#parse_record
D, [2023-08-03T11:19:11.975352 #2258283] DEBUG -- :   record = #<Fusuma::Plugin::Events::Records::TextRecord:0x00005647f8ecb738 @text=" event4   TOUCH_MOTION            +0.146s\t0 (0) 53.61/34.45 (168.92/58.79mm)">
D, [2023-08-03T11:19:11.975494 #2258283] DEBUG -- : > tap detector
D, [2023-08-03T11:19:11.975546 #2258283] DEBUG -- :   no movement?
D, [2023-08-03T11:19:11.975656 #2258283] DEBUG -- : > hold detector
D, [2023-08-03T11:19:11.975708 #2258283] DEBUG -- :   no movement?
D, [2023-08-03T11:19:11.975777 #2258283] DEBUG -- :   finger 0 moved too much: 9.50084206794322 > 5.0
D, [2023-08-03T11:19:11.975831 #2258283] DEBUG -- : > swipe detector
D, [2023-08-03T11:19:11.975877 #2258283] DEBUG -- :   movement?
D, [2023-08-03T11:19:11.975922 #2258283] DEBUG -- :   angles?
D, [2023-08-03T11:19:11.976014 #2258283] DEBUG -- :   !too much difference between 354.6854543300552 and 129.00259942608486, not a swipe
D, [2023-08-03T11:19:11.976069 #2258283] DEBUG -- : > pinch detector
D, [2023-08-03T11:19:11.976125 #2258283] DEBUG -- :   movement?
D, [2023-08-03T11:19:11.976175 #2258283] DEBUG -- :   at least 2 fingers?
D, [2023-08-03T11:19:11.976222 #2258283] DEBUG -- :   distance change between first 2 fingers?
D, [2023-08-03T11:19:11.976295 #2258283] DEBUG -- :   assuming direction is -1, testing all pairs of fingers
D, [2023-08-03T11:19:11.976417 #2258283] DEBUG -- :   fingers 0 and 1 moved from 99.73262755989136 to 94.47556562413375
D, [2023-08-03T11:19:11.976482 #2258283] DEBUG -- :   pinch in detected!
/usr/lib/ruby/gems/3.0.0/gems/fusuma-plugin-touchscreen-0.1.0/lib/fusuma/plugin/detectors/touch_detectors/pinch_detector.rb:58:in `detect': uninitialized constant Fusuma::Plugin::Events::Records::TouchRecords::PinchRecord (NameError)
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-plugin-touchscreen-0.1.0/lib/fusuma/plugin/detectors/touch_detector.rb:65:in `block in detect'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-plugin-touchscreen-0.1.0/lib/fusuma/plugin/detectors/touch_detector.rb:64:in `each'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-plugin-touchscreen-0.1.0/lib/fusuma/plugin/detectors/touch_detector.rb:64:in `detect'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:127:in `block in detect'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:126:in `each'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:126:in `each_with_object'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:126:in `detect'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:77:in `pipeline'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:68:in `block in run'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:68:in `loop'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:68:in `run'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:21:in `run'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/exe/fusuma:46:in `<top (required)>'
    from /usr/bin/fusuma:25:in `load'
    from /usr/bin/fusuma:25:in `<main>'
m4z-0xf00 commented 1 year ago

Update:

Phaengris commented 1 year ago

Wow, someone really tried my plugin :) thanks. I'll take a look on that. Can't guarantee quick answer, but as soon as I have time.

Phaengris commented 1 year ago

Well, it seems the first version was very buggy. Working on fixes. Sorry for being a bit slow on that.

Fixed bugs:

TODO:

So version 0.1.1 with many bugfixes is upcoming :)

And thank you for trying my plugin, it served as a great inspiration for me to work further on it :)

Phaengris commented 1 year ago

Crap. This "hold" problem goes into Fusuma itself, related to events timing. Need time.