joyosc 1.0 roadmap

[danomatika]

Some things I am thinking about for joyosc 1.0:

• [x] sensor event support
• [x] touchpad event support
• [x] codebase refactor / modernization -> break up singleton Config and lower amount of redundant code
• [x] no longer print to stdout by default, add -v/--verbose option to enable this
• [x] device GUID matching
• [x] device name and GUID exclusion
• [ ] joystick event remapping via name, ie. map axis 1 to "leftx" (similar to SDL's game controller name mappings)
• [ ] axis value scaling: hid (default, int -32767 to 32767), normalize (float -1 to 1), alt normalize? (float 0 to 1), MIDI (float 0-127)?
• [ ] rumble support
• [ ] raw sensor to motion processing, ie. gyro -> pitch, roll, yaw
• [ ] transition master branch to main?

joystick event remapping via name

This allows for handling non-standard controllers in a similar way to gamepads. I mainly am wanting to access the button pressure values on some controllers which are exposed as axes only if opened as a joystick. In this case, I would make the mapping manually and be able to send button "x" both as button x bool and axis x int.

Reference:

• SDL: Pressure sensitive buttons
• SDL_GameControllerMapping

An additional option would be to parse the SDL mapping string to set up the manual mapping for the Joystick and you only need to specify names for those mappings which are not found. Could also change remapping layer to use SDL structs

axis value scaling

This would take a step out when working with incoming numbers in Pd etc. I have the conversions already tested just not applied since there was a lot of churn in develop last week.

My use case is to match the axis scaling between hid values joyosc to iOS GCController events in PdParty which are normalized -1 to 1. I have done this within the patches, but would be nice to remove a possible failure point in the future if/when I transition or add additional systems.

rumble support

Add support for handling incoming OSC messages to invoke the controller "rumble" ie. vibration motors. Something like:

/joyosc/device/rumble gc0 0.7 200 -> rumble controller "gc0" at 70% for 200 ms

References:

• Lazy Foo Force Feedback tutorial
• SDL_HapticRumblePlay

raw sensor to motion processing

Integrate raw sensor value to motion processing to remove a step needed in receiving applications. A new "-m/--motion" and related config options would be added to turn this on. It would be off like the sensors option. It should also work without sensors to where internally, sensor events are on and processed but only motion events are sent out. The OSC address scheme would mirror sensor events:

/joyosc/devices/gc0/motion accel x y z?
maybe better as...
/joyosc/devices/gc0/motion accel pitch roll yaw

Reference: GamepadMotion.hpp

[danomatika]

Ping @agraef

[danomatika]

Most of these features would be optional. I would keep the default behavior largely as it is now, except for verbose printing and possibly making default axis values normalized. The HID int -32767 to 32767 range is more precise to a degree but I think less helpful for beginners. I would try to allow for globe las well as per-device config overrides for flexibility.

[agraef]

Looks good to me. I have some thoughts about processed motion sensor data, though -- see below.

Axis scaling/normalization would certainly be useful. This might also make sense for the motion data (e.g., normalizing acceleration data to units of 1 g = 9.81 m/s^2, and angular velocities so that 1 unit means 2π radians). IMHO, all these should be options, though, so that you get the raw data as SDL delivers it by default (this would also be important for backward compatibility).

Event remapping sounds useful as well. Having rumble would be awesome. :)

Now for the sensor data.

Most of these features would be optional.

That's very important especially for the sensor data, as it's a lot of data, and with the calculated gravity, local/world/player space, and maybe some other data (quaternions? accel minus gravity?), that could be easily 5-7 times the amount of sensor messages that we already have. And there's more: In the gpmotion external, I also calculate accumulated rotation. This is what you probably want to use in most applications, and you need precise delta times in order to calculate these values (accumulated rotation += angular velocity * delta t), which are readily available in Pd, but joyosc might use SDL's ticks or event timestamps for that instead.

But do you really want to do all this processing in joyosc itself? The beauty of joyosc IMHO is that it's a fairly straightforward mapping of gamepad inputs to OSC, nothing more, nothing less. It's hard to tell what kinds of post-processing different applications may need, there's no one size fits all here. And sensor fusion is just one approach developed for an entirely different purpose, we don't even know yet how useful it will be for our purposes.

Also, you'd use a lot of network bandwidth for transmitting all these processed values when they can easily be calculated from just the raw sensor data on the receiving end.

Therefore I'm not entirely convinced that incorporating all these calculations into joyosc itself is the right way to go. I think that a more modular approach would be needed. Like having different externals/plugins for different environments which process the raw data received from joyosc as they see fit. In time, with other plugins available, say, for Max, SC3, Csound, etc. one might see some usage patterns emerge that would allow you to factor out some common utility functions into a joyosc helper library to better support all the applications.