Rotation support

[m4z-0xf00]

Im looking a way to read the rotation gesture for a touchscreen radial menu and you said features requests are welcome so there we go.

Thanks for the work you're doing, can't help you for the code because I dont know how to write in ruby. I can provide some testing if you need, I have a touchscreen on my laptop so I can try stuff for you. I've seen the spec folder in the draft, I'm not sure what it does, but if you need mine you just got to ask.

[Phaengris]

Rotate support is a planned feature, yes. Now the draft supports tap / swipe, then I plan to implement hold, then pinch / rotate.

The spec folder contains auto tests (or should contain, at least :see_no_evil: when I add them).

@m4z-0xf00 what do you mean "if you need mine"? Is about specs?

Testing is appreciated! But unfortunately it can't be installed just with "gem install fusuma-plugin-touchscreen" as Fusuma itself, because of it's not released as a gem yet... There's a way to run it locally, but it requires a bit of Ruby skills.

[m4z-0xf00]

Yes I meant about specs, Im not a native speaker so there are chances my sentences sound weird ^^ Anyways can you quickly explain to me the plugin stuff for fusuma, do you just have to explain where to get the data or do you need to implement the math and detection stuff ? About the testing I figured it wasn't ready, but it's ok good stuff needs time to build, don't hesitate to tell when you want me to test.

[Phaengris]

Yes, I have to implement the math. It's already done for swipes. Need to think how to do the math for pinches / rotations. Have some ideas, need to experiment :)

[m4z-0xf00]

Assuming you have a list a all touch points, you would need to find a way to reliably track the fingers without them switching (when too close for example) then with that you can calculate the distance/angle between the fingers using vector math. The difficult part to me is the tracking as the touch points may not be tracked well by the system and some touch points can disappear. Also an user may be touching the screen with the palm or with more fingers than expected. I suppose some kind of delay could be used or you can have a data structure representing fingers that you can update when you receive tracking data. Maybe comparing the deltas between the new data and the old can help too.

[Phaengris]

If you still need it, this is the implementation I came to in the end. It's very simple and may not handle corner cases well, but for now it works. https://github.com/Phaengris/fusuma-plugin-touchscreen/blob/draft/lib/fusuma/plugin/detectors/touch_detectors/rotate_detector.rb

[m4z-0xf00]

Hello, sorry I've been away for some time, I manged to install the plugin, though I'm not even sure fusuma sees it. This is the list I get on load:

Enabled Plugins: Fusuma::Plugin::Buffers::GestureBuffer Fusuma::Plugin::Buffers::KeypressBuffer Fusuma::Plugin::Buffers::TapBuffer Fusuma::Plugin::Buffers::TimerBuffer Fusuma::Plugin::Detectors::HoldDetector Fusuma::Plugin::Detectors::KeypressDetector Fusuma::Plugin::Detectors::PinchDetector Fusuma::Plugin::Detectors::RotateDetector Fusuma::Plugin::Detectors::SwipeDetector Fusuma::Plugin::Detectors::TapDetector Fusuma::Plugin::Events::Records::ContextRecord Fusuma::Plugin::Events::Records::GestureRecord Fusuma::Plugin::Events::Records::IndexRecord Fusuma::Plugin::Events::Records::KeypressRecord Fusuma::Plugin::Events::Records::TextRecord Fusuma::Plugin::Executors::CommandExecutor Fusuma::Plugin::Executors::SendkeyExecutor Fusuma::Plugin::Filters::KeypressFilter Fusuma::Plugin::Filters::LibinputDeviceFilter Fusuma::Plugin::Inputs::LibinputCommandInput Fusuma::Plugin::Inputs::TimerInput Fusuma::Plugin::Parsers::KeypressParser Fusuma::Plugin::Parsers::LibinputGestureParser Fusuma::Plugin::Parsers::TapParser

The thing is I'm not really sure if I know how to load the plugin, is there something to add in the config ? I have swipe and rotate sections in it but I don't know if it's enough. Could you perhaps share an example config ?

[m4z-0xf00]

Update:

• uninstalled the gem and used the gem2arch package generator, turns out the direct gem install puts file into the home folder and fusuma looks in /usr/share.
• now fusuma sees my touch events in verbose modes but it doesn't seem to generate gestures from it, this is probably coming from my config I'll investigate that
• here's some of the output I get

D, [2023-08-03T10:59:09.224088 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_FRAME +218.810s } D, [2023-08-03T10:59:09.228623 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_MOTION +218.816s 0 (0) 79.79/29.96 (251.38/51.12mm)} D, [2023-08-03T10:59:09.229239 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_MOTION +218.816s 1 (1) 74.19/44.24 (233.77/75.50mm)} D, [2023-08-03T10:59:09.229756 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_MOTION +218.816s 2 (2) 74.12/35.11 (233.54/59.92mm)} D, [2023-08-03T10:59:09.230371 #2232286] DEBUG -- : {:input_event=>time: 2023-08-03 10:59:09 +0200, tag: libinput_command_input, record: event4 TOUCH_FRAME +218.816s }

[m4z-0xf00]

Update:

• filtered fusuma to use only the touchscreen devices (idk why, but there are 2 on my device even if I have on touch screen)
• the plugin sees the events and passes it to the detectors
• swipe detected, crash witnessed
• it's not happy with line 58 (last line) in pinch_detector.rb
• the line in question:

  Plugin::Events::Records::TouchRecords::PinchRecord.new(finger: touch_buffer.finger, direction: direction)

• Here's a sample of the log

D, [2023-08-03T11:19:11.973557 #2258283] DEBUG -- : {:input_event=>time: 2023-08-03 11:19:11 +0200, tag: libinput_command_input, record:  event4   TOUCH_FRAME             +0.142s  }
D, [2023-08-03T11:19:11.973805 #2258283] DEBUG -- : Fusuma::Plugin::Parsers::TouchParser#parse_record
D, [2023-08-03T11:19:11.973879 #2258283] DEBUG -- :   record = #<Fusuma::Plugin::Events::Records::TextRecord:0x00005647f8efd198 @text=" event4   TOUCH_FRAME             +0.142s\t">
D, [2023-08-03T11:19:11.975017 #2258283] DEBUG -- : {:input_event=>time: 2023-08-03 11:19:11 +0200, tag: libinput_command_input, record:  event4   TOUCH_MOTION            +0.146s  0 (0) 53.61/34.45 (168.92/58.79mm)}
D, [2023-08-03T11:19:11.975274 #2258283] DEBUG -- : Fusuma::Plugin::Parsers::TouchParser#parse_record
D, [2023-08-03T11:19:11.975352 #2258283] DEBUG -- :   record = #<Fusuma::Plugin::Events::Records::TextRecord:0x00005647f8ecb738 @text=" event4   TOUCH_MOTION            +0.146s\t0 (0) 53.61/34.45 (168.92/58.79mm)">
D, [2023-08-03T11:19:11.975494 #2258283] DEBUG -- : > tap detector
D, [2023-08-03T11:19:11.975546 #2258283] DEBUG -- :   no movement?
D, [2023-08-03T11:19:11.975656 #2258283] DEBUG -- : > hold detector
D, [2023-08-03T11:19:11.975708 #2258283] DEBUG -- :   no movement?
D, [2023-08-03T11:19:11.975777 #2258283] DEBUG -- :   finger 0 moved too much: 9.50084206794322 > 5.0
D, [2023-08-03T11:19:11.975831 #2258283] DEBUG -- : > swipe detector
D, [2023-08-03T11:19:11.975877 #2258283] DEBUG -- :   movement?
D, [2023-08-03T11:19:11.975922 #2258283] DEBUG -- :   angles?
D, [2023-08-03T11:19:11.976014 #2258283] DEBUG -- :   !too much difference between 354.6854543300552 and 129.00259942608486, not a swipe
D, [2023-08-03T11:19:11.976069 #2258283] DEBUG -- : > pinch detector
D, [2023-08-03T11:19:11.976125 #2258283] DEBUG -- :   movement?
D, [2023-08-03T11:19:11.976175 #2258283] DEBUG -- :   at least 2 fingers?
D, [2023-08-03T11:19:11.976222 #2258283] DEBUG -- :   distance change between first 2 fingers?
D, [2023-08-03T11:19:11.976295 #2258283] DEBUG -- :   assuming direction is -1, testing all pairs of fingers
D, [2023-08-03T11:19:11.976417 #2258283] DEBUG -- :   fingers 0 and 1 moved from 99.73262755989136 to 94.47556562413375
D, [2023-08-03T11:19:11.976482 #2258283] DEBUG -- :   pinch in detected!
/usr/lib/ruby/gems/3.0.0/gems/fusuma-plugin-touchscreen-0.1.0/lib/fusuma/plugin/detectors/touch_detectors/pinch_detector.rb:58:in `detect': uninitialized constant Fusuma::Plugin::Events::Records::TouchRecords::PinchRecord (NameError)
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-plugin-touchscreen-0.1.0/lib/fusuma/plugin/detectors/touch_detector.rb:65:in `block in detect'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-plugin-touchscreen-0.1.0/lib/fusuma/plugin/detectors/touch_detector.rb:64:in `each'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-plugin-touchscreen-0.1.0/lib/fusuma/plugin/detectors/touch_detector.rb:64:in `detect'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:127:in `block in detect'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:126:in `each'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:126:in `each_with_object'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:126:in `detect'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:77:in `pipeline'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:68:in `block in run'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:68:in `loop'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:68:in `run'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/lib/fusuma.rb:21:in `run'
    from /usr/lib/ruby/gems/3.0.0/gems/fusuma-2.4.1/exe/fusuma:46:in `<top (required)>'
    from /usr/bin/fusuma:25:in `load'
    from /usr/bin/fusuma:25:in `<main>'

[m4z-0xf00]

Update:

• added a require, looks bad but it werks

  require_relative '../../events/records/touch_records/pinch_record'

• I know kung fu
• I wonder what happens when I uncomment

  # Fusuma::Plugin::Detectors::TouchDetectors::RotateDetector.new

[Phaengris]

Wow, someone really tried my plugin :) thanks. I'll take a look on that. Can't guarantee quick answer, but as soon as I have time.

[Phaengris]

Well, it seems the first version was very buggy. Working on fixes. Sorry for being a bit slow on that.

Fixed bugs:

• Touch devices detecting.
• Enabled rotate detector (was disabled by mistake).
• Fixed bug when first tap was detected as hold.
• Fixed bug when config file entries were searched incorrectly for gestures with directions.

TODO:

• the hold gesture tends to "swallow" pinches and rotates. I have to deal with that.

So version 0.1.1 with many bugfixes is upcoming :)

And thank you for trying my plugin, it served as a great inspiration for me to work further on it :)

[Phaengris]

Crap. This "hold" problem goes into Fusuma itself, related to events timing. Need time.