Tremor Filter

[ClaudeJGreengrass]

Hi, Congratulations and thanks for the work you have done on the RP2040 USB Host device.

I've been researching Essential Tremor for a couple of years now and have built an electrical neurostimulator using COTS and it can give relief of hand/forearm tremors for a couple of hours. Back in August 2022 I started looking for a way to build a tremor filter for a computer mouse. This lead to AtMakers and Bill Binko (ATMakersBill) and their work on USB Filters. Now that Adafruit has build the RM2040 with USB Host the hardware problem of a Tremor Filter for a computer mouse is solved. What is need is some software to handle the problems that people with hand/forearm tremors have with using computer mice.

I would like to use your code base but I don't want to step on anyone toes so I'm asking you how I can contribute code and ideas to solve the problem I have with using a computer mouse and to make it possible to help others.

all the best James Kissel

[dglaude]

Hi James,

I was in the ATmakers meeting when @ATMakersBill presented his filter and what he did with hid-remapper.

And since that meeting I have been thinking about how tremor affect the mouse movement, and what mathematical "filter" could be applied.

Here I am reverse engineering what Bill said and showed to us (I watched the video multiple time).

Bill added the "log" (for logarithm) "operation" in the "expression" language. So this is a custom change he made to the code and recompiled. That new mathematical function can help a lot "smooth" the input value (X or Y movement) into more soft movement. I am not sure if "log" is the way, but it sure can help and could have other use in hid-remapper. My suggestion would be for Bill to submit a Pull Request (git terminology for a suggested change) to jfedor2.

The next thing that Bill does is to use two expression to mathematically modify the X (and Y) mouse value read from the hardware mouse and send the modified value to the computer. The beauty of this way of doing is that he use the existing flexibility build in hid-remapper to experiment/build the filter. And the formula he use use the new "log" function he added. (By trying to understand the formula showed by Bill, it seems that "log" does not consume the value on the stack, but just push on top of the stack the log value) I could be wrong there and my RPL (Reverse Polish Lisp) is a bit rusty, but I had and HP48 calculator...

I don't know if it can help with tremor (but it could simplify Bill formula) would be to have additional operation in the expression language. In particular, I would love to have:

• abs(x) to have the absolute value of the value on the stack
• sign(x) to have -1, 0 or 1 depending on the sign of the value on the stack
• swap(x,y) to swap the top of the stack with the value below (swap is for me, not really needed for tremor, but it help for complex stack management)

Finally, one known documented limitation of expression and hid_remapper is that you can only get the current X and Y delta of the mouse, and compute the value you want to send to the computer. Maybe to fight tremor, you need to know the previous mouse mouvement. Typically if you want to make an average of the last few moves. This does not seems possible with the current capabilities of hid_remapper. I don't know yet if it is needed, or could have other usage in hid_remapper (like click debounce or double click detection).

Another limitation is that we don't have a "memory" or variable that we could modify or use in expression. I would love to have a parameter that could be tuned let's say by the middle button roller. That would permit an easy way for you to tune one value, without using the hid_remapper user interface. One typical usage could be DPI scalling that you control with the middle button. Today in hid_remapper, the example use layer to switch between fixed value of DPI scalling.

I don't know the magic mathematical formula for tremor compensation. And it might require experimentation. Actually, it would require recording of mouse activity for know task (intent) to see how the shaking behave. And maybe with such value, we could simulate various function and find one that seems to work in theory (and hopefully in practice).

I think one improvement could be to not consider X and Y separatly, but as a whole, as a mouse vector movement. It will make complicated formula, but hid_remapper has an example that rotate the mouse mouvement from 30° using sin and cos. When you described the tremor movement you said it was on some angle, different for left-handed and right-handed person.

Assuming the shaking is oriented in +30° or -30°. Then maybe we could take X and Y value, check if this is 30° angle and filter that out completely. Basically you would be able to go up/down, left right. You could do 45° or 135° but anything in the +/-30° area would be impossible or strongly reduced. I don't exactly know the right math formula, especially because I hate trigonometry, but it involve multiplying X and Y by a factor. That factor would be most of the time 1, but would be 0 for an angle of +/-30° and maybe 0.5 for 28° and 32°. Of course we need an easy way for you to tune this angle (maybe the scaling value from the hid_remapper mapping UI and accessible from the expression language).

The nice thing about that filtering I am proposing (without providing the formula) is that it does not need the history of move or memory. I'll try to dig further and see if I can figure out some math to implement that, with or without the need for "log". I see I need to compute the angle, then using that angle in a formula to make it near 0 when near 30°, then use that as a multiplying factor for X and for Y (so it need to be done in the X and Y expression). Maybe Bill, jfedor2, or you will catch a formula that match what I try to describe, and give that a try. And that would be great too.

I don't know if this is the best place, but let's keep brainstorming this topic, hoping it help.

[ClaudeJGreengrass]

Hi David,

It's nice to make your acquaintance and it is really great to have someone to discuss technical details on the Tremor filter and thank you for your considered response on the Tremor Filter.

I've got other commitments this evening, but will get back to you tomorrow morning.

all the best jlk

On 2023-05-03 19:52, David Glaude wrote:

Hi James,

I was in the ATmakers meeting when @ATMakersBill [1] presented his filter and what he did with hid-remapper.

And since that meeting I have been thinking about how tremor affect the mouse movement, and what mathematical "filter" could be applied.

Here I am reverse engineering what Bill said and showed to us (I watched the video multiple time).

Bill added the "log" (for logarithm) "operation" in the "expression" language. So this is a custom change he made to the code and recompiled. That new mathematical function can help a lot "smooth" the input value (X or Y movement) into more soft movement. I am not sure if "log" is the way, but it sure can help and could have other use in hid-remapper. My suggestion would be for Bill to submit a Pull Request (git terminology for a suggested change) to jfedor2.

The next thing that Bill does is to use two expression to mathematically modify the X (and Y) mouse value read from the hardware mouse and send the modified value to the computer. The beauty of this way of doing is that he use the existing flexibility build in hid-remapper to experiment/build the filter. And the formula he use use the new "log" function he added. (By trying to understand the formula showed by Bill, it seems that "log" does not consume the value on the stack, but just push on top of the stack the log value) I could be wrong there and my RPL (Reverse Polish Lisp) is a bit rusty, but I had and HP48 calculator...

I don't know if it can help with tremor (but it could simplify Bill formula) would be to have additional operation in the expression language. In particular, I would love to have:

• abs(x) to have the absolute value of the value on the stack
• sign(x) to have -1, 0 or 1 depending on the sign of the value on the stack
• swap(x,y) to swap the top of the stack with the value below (swap is for me, not really needed for tremor, but it help for complex stack management)

Finally, one known documented limitation of expression and hid_remapper is that you can only get the current X and Y delta of the mouse, and compute the value you want to send to the computer. Maybe to fight tremor, you need to know the previous mouse mouvement. Typically if you want to make an average of the last few moves. This does not seems possible with the current capabilities of hid_remapper. I don't know yet if it is needed, or could have other usage in hid_remapper (like click debounce or double click detection).

Another limitation is that we don't have a "memory" or variable that we could modify or use in expression. I would love to have a parameter that could be tuned let's say by the middle button roller. That would permit an easy way for you to tune one value, without using the hid_remapper user interface. One typical usage could be DPI scalling that you control with the middle button. Today in hid_remapper, the example use layer to switch between fixed value of DPI scalling.

I don't know the magic mathematical formula for tremor compensation. And it might require experimentation. Actually, it would require recording of mouse activity for know task (intent) to see how the shaking behave. And maybe with such value, we could simulate various function and find one that seems to work in theory (and hopefully in practice).

I think one improvement could be to not consider X and Y separatly, but as a whole, as a mouse vector movement. It will make complicated formula, but hid_remapper has an example that rotate the mouse mouvement from 30° using sin and cos. When you described the tremor movement you said it was on some angle, different for left-handed and right-handed person.

Assuming the shaking is oriented in +30° or -30°. Then maybe we could take X and Y value, check if this is 30° angle and filter that out completely. Basically you would be able to go up/down, left right. You could do 45° or 135° but anything in the +/-30° area would be impossible or strongly reduced. I don't exactly know the right math formula, especially because I hate trigonometry, but it involve multiplying X and Y by a factor. That factor would be most of the time 1, but would be 0 for an angle of +/-30° and maybe 0.5 for 28° and 32°. Of course we need an easy way for you to tune this angle (maybe the scaling value from the hid_remapper mapping UI and accessible from the expression language).

The nice thing about that filtering I am proposing (without providing the formula) is that it does not need the history of move or memory. I'll try to dig further and see if I can figure out some math to implement that, with or without the need for "log". I see I need to compute the angle, then using that angle in a formula to make it near 0 when near 30°, then use that as a multiplying factor for X and for Y (so it need to be done in the X and Y expression). Maybe Bill, jfedor2, or you will catch a formula that match what I try to describe, and give that a try. And that would be great too.

I don't know if this is the best place, but let's keep brainstorming this topic, hoping it help.

-- Reply to this email directly, view it on GitHub [2], or unsubscribe [3]. You are receiving this because you authored the thread.Message ID: @.> [ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "https://github.com/jfedor2/hid-remapper/issues/66#issuecomment-1533891076", "url": "https://github.com/jfedor2/hid-remapper/issues/66#issuecomment-1533891076", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

Links:

[1] https://github.com/ATMakersBill [2] https://github.com/jfedor2/hid-remapper/issues/66#issuecomment-1533891076 [3] https://github.com/notifications/unsubscribe-auth/ACPL55NVCZCL3K537R77JYDXELVSXANCNFSM6AAAAAAXU2J3Q4

--

EssentialTremorLab

essential-tremors-101

[ClaudeJGreengrass]

Hi David,

I've kind of got an answer from Bill Binko. He's been incommunicado on the Tremor Filter subject since the video was made. He says he has made a fork but whether or not we can play in his fork is not clear.

Bill is a very busy man with many responsibilities so I am going to assume that he won't be available most of the time for either development nor for technical discussions. This suggests to me, that we create a fork and carry on our discussions regarding a Tremor Filter there and just get on with it.

Your thoughts?

[dglaude]

Dear James,

I did not find Bill fork, but I am sure he has a copy of the code and even compiled an UF2 for me to enable GPIO 18 for the Adafruit board before it was fully supported here. And maybe the version he gave me has support for log.

Right now, I don't plan to fork or at least compile my own variant of hid-remapper, just because I hope to use existing feature and the flexibility of expressions to implement a filter. But sure if there are limitation that cannot be overcome, someone will have to put his hand in the code...

I have enough things to try without having to add feature to hid-remapper. So I have a fork and I will document my experiment in that form. I'll publish the expressions I use and share the json file if you want to give it a try: https://github.com/dglaude/hid-remapper/tree/experiment/expressions

Feel free to discuss there if we want to avoid spamming this place.

Regards

[ClaudeJGreengrass]

David, Sounds like a plan. I looked at your fork. I could not see the "Issues" there.

[dglaude]

Sorry, apparently "Issues" was not enabled by default. I also started "Discussions" but I never used that feature, maybe it is more user friendly?

[itsnoteasy]

I think the easiest way to add a tremor filter would be to create a square deadzone around the mouse cursor of customizable size. the deadzone would be dragged around the screen, only moved when the cursor hits the edge of the square box. the computer would interpret the center of the deadzone as the intended cursor point. the old mlt04 mouse sensors used a 3x3 deadzone, and it worked quite well.

[ClaudeJGreengrass]

Hi, @itsnoteasy,

We now have our own build tree for tremor filters at https://github.com/ATMakersOrg/hid-remapper/discussions

Feel free to post comments, suggestions, and report bugs there. BTY, what s/w implemented the deadzone?

thank yoiu

[itsnoteasy]

the deadzone was implemented in the mouse firmware. I think it's a good approach because there's no need to detect changes over time. consider a 3x3 pixel grid arranged like a numpad 1 to 9. you plug in your peripheral and you're on pixel 5, then move diagonally down right, you're in box 9, cursor hasn't moved, move diagonally another box down right, cursor is still on pixel 9, but has moved one unit. the tremor makes the muscle naturally orbit around a centre position, if the deadzone is wider than the orbit, then you've fixed the issue. 3x3 is very small though, consider a deadzone 10px by 10px, or any size really.

[itsnoteasy]

I recalled an mlt04 emulation driver someone once wrote as a curiosity. if you wanted to, you could install it, to see if it is usable, and if so then you could work on modifying the code for a larger deadzone

https://github.com/narsn/MLT04-Emulation

[ClaudeJGreengrass]

@itsnoteasy Thank you for the reference. I'll keep it in mind if at some time in the future we want to go down that technical route.

The device we are trying to build will be a 'plug-and-play' with no additional software needed.

[jfedor2]

Out of curiosity I implemented the "sliding window" approach described above, using the features introduced in the latest release.

I don't think it works very well by itself, but maybe it could be useful as a building block together with some other processing.

Registers 1 and 2 are used to store the current position of the cursor within the sliding window. Expressions 1 and 2 fetch the mouse deltas and add them to the registers.

Expressions 3 and 4 check if the current values in registers 1 and 2 are outside the sliding window and if they are, the overflow values are sent as the current deltas to the PC.

Expressions 5 and 6 clamp the register contents to the sliding window.

hid-remapper-config-sliding-window-tremor-filter.zip

[itsnoteasy]

I tried out the tremor filter, it's not bad, but it's tendency to prefer to move either horizontally or vertically is frustrating, seems to be a bug there. on the other hand, i wanted that effect to make scrolling easier, so every cloud has a silver lining.

edit: perhaps if two hid remappers were used in series and one was dedicated to x axis, and the other to y axis the issue would be gone? I'd try if i could find an extra cable.

[jfedor2]

Perhaps the window should be a circle instead of a square? Or something like that, I don't know, there's room for experimentation for sure.

[itsnoteasy]

I found a second cable, and managed to load the x axis expressions on one remapper and the y axis on another remapper, and it behaves exactly the same as before, so that would imply your coding is not at fault. I find this to be incredibly irritating to use due to its unpredictability, perhaps it would feel like an improvement to someone with a medical condition. I am unsure of what a circular deadzone would involve. certainly the coding would be difficult. Further comments by testers are surely invited.

[dglaude]

(Maybe on the day of the release), I did try r2023-07-19 and I tried to use and understand the auto-clicker example... but it failed. At that time I did not knew if it was me not understanding the expression, or my testing strategy failing. So I just gave up as reporting the problem would have require much more investigation and I was about to leave for vacation. So I am happy you made sense of it and could use it for tremor.

I agree that the computation should be circular and not independent for x and y. There is now square root for that and that is also used by auto-clicker. If it was not for my failure above and the summer break, I would have try to do something like what you did @jfedor2 .

So I guess that now I need to try your attempt.

UPDATE: My testing procedure for the auto-clicker was wrong... I need to move longer, and it only click once (I guess). So there was nothing wrong with it.

[dglaude]

Ok, tested and I see the point of @itsnoteasy ... while it is possible to move at 45°, everything seem so horizontal or vertical.

[ClaudeJGreengrass]

the deadzone was implemented in the mouse firmware. I think it's a good approach because there's no need to detect changes over time. consider a 3x3 pixel grid arranged like a numpad 1 to 9. you plug in your peripheral and you're on pixel 5, then move diagonally down right, you're in box 9, cursor hasn't moved, move diagonally another box down right, cursor is still on pixel 9, but has moved one unit. the tremor makes the muscle naturally orbit around a centre position, if the deadzone is wider than the orbit, then you've fixed the issue. 3x3 is very small though, consider a deadzone 10px by 10px, or any size really.

Here are my thoughts on the use of a deadzone. Mouse navigation movements come in at least two different levels. Gross/fast movements to travers large areas of the screen and slower/more precise movement to navigate to a relatively small target relative to the screen size.

Little, if any attenuation is required for the former. During the latter the amplitude of the tremor becomes significant relative to the intention movement implying a need for attenuation. As the target is approached, Intentional Tremors can be triggered in some individuals adding to the existing tremor signal.

Because of these factors, I chose to attenuate the mouse movement based on the instantaneous speed of the mouse. There are two advantages to this choice. Firstly no calculation is required to determine the speed as the data is limited to ±127 so the abs() function will suffice. Secondly there is no delay or lag in the display of the mouse movement.

I choose to divide the mouse movement into three sections. Fast, between 127 and 32, medium between 31 and 8, and slow for any movement below 8. The first range has no attenuation. The middle range as an attenuation of 2. It requires two units of mouse movement to move the screen cursor 1 unit. The slowest speed has an attenuation of 4.

These values work well for myself with my mild to moderate tremors: ≈ 5.2Hz and ≈ 2 cm amplitude. YMMV