Feature request: Support for FTMS bike

[chriselsen]

It would be great if Gymnasticon could support generic FTMS (Fitness Training Machine Service) enabled bike trainer as a source.

While these bike trainer can connect directly to Zwift via these BLE services, there are two reasons why you might still want to put Gymnasticon in between:

• Certain bikes provide inflated power values and could benefit from Gymnasticon's existing feature of a power scale and/or power offset. One such bike is the Bowflex C6 / Schwinn IC4/IC8 (same bike, different names). This bike is known to provide incorrect power values.
• Certain bikes (e.g. the Bowflex C6 from above) can only connect to a single receiver over BLE. Therefore it is not possible to connect Zwift (e.g. via PC/Mac/Apple TV) while at the same time connecting a bike computer or smart watch. Gymnasticon could provide an alternative here and allow multiple receivers to connect.

[ptx2]

It should be easy enough to add support for BLE CPS as a bike/source.

I haven't read that thread yet but if it's possible to identify these bikes by their BLE manufacturer info or advertising name, and if they are typically always off by the same amount (?) Gymnasticon could use suitable default power adjustment values for these bikes automatically to save the user from manual configuration.

[chriselsen]

Here is a BLE scan log file for the Bowflex C6: LBX_LOGS_2021-01-31_11-04.txt

And here are some screenshots to go along with it:

Any other way I can help make this work?

[ptx2]

Ah so is there no 1818 service (CPS) but there is a FTMS service. I guess it reports power and cadence via the Fitness Machine Service Indoor Bike Data characteristic.

It would be helpful to see a log of Indoor Bike Characteristic notification values for a few seconds of riding if possible.

Also do you have a ballpark idea what kind of correction needs to be applied (e.g. watts is always 20% too high) and does it vary (much) from one bike to the next?

[chriselsen]

Yes, you're right. Just noticed that as well. Maybe the type of bike should be called "ftms" because of that. ;-) Unfortunately each bike's power value is off. I'm trying to figure out a way to see if one can determine how much their own bike is off. But until then I would just pass through the values.

Here are two log files for the FTMS notify on power, cadence and for the 2nd file also with heart rate.

Without heart rate: LBX_LOGS_2021-01-31_11-59.txt

With heart rate: LBX_LOGS_2021-01-31_12-02.txt

The following devices can successfully pick up that data (only one at a time):

• Apple TV 4K
• Samsung Tab S4 & S6
• Pixel 3

A Garmin Fenix 5+ watch can pick up the cadence, but not the power values. But that issue is on Garmin.

Let me know if there is anything else I can provide.

[chriselsen]

Here are two additional interesting resources that might help:

• Blog post on FTMS from someone at the parent company of Schwinn/Bowflex: https://jjmtaylor.com/post/fitness-machine-service-ftms/
• FTMS Specification: https://www.bluetooth.org/DocMan/handlers/DownloadDoc.ashx%3Fdoc_id%3D423422

[chriselsen]

I'm a bit stuck here: Looking at the logs output above I expect the power output (and most likely also the cadence) inside the FTMS service at serviceUUID 1826 and characteristicUUID 2ad2, available via notify.

Now I wrote a basic test script to get a dump of the data and see what's actually included:

var noble = require('@abandonware/noble');

noble.on('stateChange', function(state) {
  if (state === 'poweredOn') {
    noble.startScanning(["1826"]);
  } else {
    noble.stopScanning();
  }
});

noble.on('discover', function(peripheral) {
  // Once peripheral is discovered, stop scanning
  noble.stopScanning();

  peripheral.connect(function(error){
    var serviceUUID = ["1826"];
    var characteristicUUID = ["2ad2"];

    // use noble's discoverSomeServicesAndCharacteristics
    // scoped to the heart rate service and measurement characteristic
    peripheral.discoverSomeServicesAndCharacteristics(serviceUUID, characteristicUUID, function(error, services, characteristics){
      characteristics[0].notify(true, function(error){
        characteristics[0].on('data', function(data, isNotification){
          console.log('data is: ' + data);
        });
      });
    });
  });
});

But that script just hangs and doesn't output anything. If I change the serviceUUID to 180d and characteristicUUID to 2a37 for Heartrate monitor, I do see my heartrate with a corresponding sensor.

Also tried serviceUUID to 1816 and characteristicUUID to 2a5b for cadence, but that didn't output anything either.

For power and cadence I can see on the bike that noble is connecting via bluetooth and if I add a console.log(characteristics[0]); I can see that I'm connected and the characteristics exist.

[chriselsen]

I'm not 100% sure, but it looks like I'm hitting this noble bug.

I installed the mentioned node-ble on the same device and ran the following simple test script:

require('dotenv').config()

const { createBluetooth } = require('.')
const { TEST_DEVICE, TEST_NOTIFY_SERVICE, TEST_NOTIFY_CHARACTERISTIC } = process.env

async function main () {
  const { bluetooth, destroy } = createBluetooth()

  // get bluetooth adapter
  const adapter = await bluetooth.defaultAdapter()
  await adapter.startDiscovery()
  console.log('discovering')

  // get device and connect
  const device = await adapter.waitDevice(TEST_DEVICE)
  console.log('got device', await device.getAddress(), await device.getName())
  await device.connect()
  console.log('connected')

  const gattServer = await device.gatt()

  // subscribe characteristic
  const service2 = await gattServer.getPrimaryService(TEST_NOTIFY_SERVICE)
  const characteristic2 = await service2.getCharacteristic(TEST_NOTIFY_CHARACTERISTIC)
  await characteristic2.startNotifications()
   while (true) {
    await new Promise(done => {
      characteristic2.once('valuechanged', buffer => {
        console.log('subscription', buffer)
        done()
      })
    })
   }

  await characteristic2.stopNotifications()
  destroy()
}

main()
  .then(console.log)
  .catch(console.error)

The output was then as expected:

discovering
got device EF:4F:1E:1F:49:F6 IC Bike
connected
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 00 00 00 00 00>
subscription <Buffer 44 02 00 00 14 00 08 00 00>
subscription <Buffer 44 02 00 00 14 00 08 00 00>
subscription <Buffer 44 02 00 00 14 00 08 00 00>
subscription <Buffer 44 02 da 02 18 00 08 00 00>
subscription <Buffer 44 02 da 02 18 00 08 00 00>
subscription <Buffer 44 02 da 02 18 00 08 00 00>
subscription <Buffer 44 02 da 02 1c 00 08 00 00>
subscription <Buffer 44 02 da 02 1c 00 08 00 00>
subscription <Buffer 44 02 da 02 20 00 08 00 00>
subscription <Buffer 44 02 da 02 20 00 0b 00 00>

[ptx2]

Interesting! Are you able to grab btsnoop logs for the entire duration of each script so we can compare what's going on?

e.g.

btmon -w noble.log
# run noble script

btmon -w node-ble.log
# run node-ble script

[chriselsen]

Here are the two files: noble.log node-ble.log

It seems that noble isn't even subscribing to the notify. At least I couldn't see any notify messages in the btmon log.

[chriselsen]

I also managed to decipher the values that are provided via serviceUUID 1826 and characteristicUUID 2ad2:

  console.log('Power: ', buffer.readInt16LE(6))
  console.log('Cadence: ', buffer.readUInt16LE(4)/2)
  console.log('Speed: ', buffer.readUInt16LE(2)/100)
  console.log('Heart Rate: ', buffer.readInt8(8))

Output (without the HR monitor connected): Power: 13 Cadence: 18 Speed: 9.1 Heart Rate: 0 The speed value is shown as a numerical value on the console and matches. The cadence is only shown as a rough bar, but still matches more or less. Power goes up, when I increase resistance (while maintaining cadence). So that makes sense as well.

But it doesn't see to match the GATT spec, which is very confusing.

[ptx2]

That is confusing! It looks like some of the flags are the wrong way around in that XML. This one looks right though? Flags bit 2 is instantaneous cadence there which seems to line up with the data.

I just tried looking at the btmon logs but doesn't look like the entire connection is captured. Do you think you could grab both logs again but restart bluetoothd before running each script to clear out any internal state? It might also be useful to capture the noble debug logs at the same time by running with DEBUG=*

e.g. for each script

# run script.js twice for max 15 seconds each
# save btmon to btmon.log and node debug logs to debug1.txt and debug2.txt

btmon -w btmon.log &
systemctl stop bluetooth
systemctl start bluetooth
DEBUG=* timeout 15 node script.js 2>&1 |tee debug1.txt
DEBUG=* timeout 15 node script.js 2>&1 |tee debug2.txt
kill %1

If all goes well, btmon -r btmon.log will list the LE Create Connection HCI Command/Events and ACL Data RX/TX showing the service/characteristic discovery, the attempt at enabling notifications (maybe), etc.

BTW the snippet runs the script twice b/c I was getting pretty consistent connection failures the very first time I ran the noble script after restarting bluetoothd.

[chriselsen]

Here are the two debug files for the noble script: debug1.txt debug2.txt

Did you also want them for the node-ble script?

I tried to write an "icbike" implementation using node-ble, but I have a hard time figuring out how to use node-ble with async in the OO model of node js.

[chriselsen]

For the format: The most confusing part is that when I convert the "capabilities" part 0x4402 of the data string to binary, there are only 3 bits set. But I'm getting 4 different values out of the data. That's the part that I can't wrap my head around. At least the description of how long each value is was correct. Therefore the trial and error search wasn't too bad.

[ptx2]

Ah I see what you mean. I think it's because C1 (Instantaneous Speed) is required when bit 0 is 0. Whereas the others are required when their respective bits are 1. Which is indeed confusing :-)

[chriselsen]

I think it's because C1 (Instantaneous Speed) is required when bit 0 is 0. Ha! That explains it. Nice.

[ptx2]

All good on the node debug logs, thanks! Do you have the updated btmon.log also with the bluetoothd restart? It'd be great to get that one for both noble and node-ble to compare.

[chriselsen]

Sure. Here is the btmon.log for noble that I forgot: btmon.log

And here are the 3 logs for the Node-BLE version: debug-node-ble1.txt debug-node-ble2.txt node-ble-btmon.log

[ptx2]

btmon.log looks good! Can you do another node-ble-btmon.log? It looks like it's still missing some stuff. If it captures everything including the bluetoothd restart there should be a line like = bluetoothd: Terminating in the output:

$ btmon -r btmon.log |grep 'Terminating'
= bluetoothd: Terminating

$ btmon -r node-ble-btmon.log |grep 'Terminating'
$

[chriselsen]

Just killed my dev install, trying this: https://github.com/abandonware/noble/issues/99#issuecomment-710125718 Will have to build a new image. Give me 30 minutes or so.

[ptx2]

Ok I have to run. Curious to know how that firmware downgrade goes, good luck!

[chriselsen]

OK, here is a new run for the Node-BLE version: node-ble-btmon.log node-ble-debug1.txt node-ble-debug2.txt

The Firmware upgrade didn't bring the expected result. Couldn't get the Bluetooth stack back working. So I had to re-image the Pi.

[chriselsen]

FYI: Also ran the above mentioned noble based node script on an older Raspberry, one with external Bluetooth USB dongle. Same result. Therefore I don't think this is a HW / Firmware issue.

[ptx2]

Hey, I just had a look over those logs.

The btmon.log shows noble sending a Read By Type Request for the CCCD descriptor. The next step if it received a response would be to write to that descriptor to enable notify on the indoor bike data characteristic. But the response never comes and it just sits there waiting until the script timeout is reached. I was expecting to see this same exchange happen successfully in the node-ble.log but I couldn't see it in there.

One difference in these logs is the connection parameters. I doubt it's the issue but I pushed up a branch that attempts to update the connection parameters after connecting: https://github.com/ptx2/gymnasticon/pull/41

So some things to try:

1) Try https://github.com/ptx2/gymnasticon/pull/41 2) Try https://github.com/ptx2/gymnasticon/pull/41 with this line https://github.com/ptx2/gymnasticon/pull/41/files#diff-c5a6e318d7ad93c779858b2ff16a1c604cb49d03b8c834ecb7399e7793085675R64 commented out (npm run build after edit) 3) Run gymnasticon in bot mode and modify your node-ble script to send power/cadence to gymnasticon over UDP e.g.

# run gymnasticon in bot mode, listen for power/cadence on udp/3000, scale power by 0.8 before forwarding it on
gymnasticon --bike bot --bot-power 0 --bot-cadence 0 --bot-host 127.0.01 --bot-port 3000 --power-scale 0.8

# update the cadence/power by sending JSON over UDP
echo '{"cadence": 100, "power": 300}' |nc -uw0 127.0.0.1 3000

We could just use node-ble of course but would like to see if the noble issue is fixable first before adding another dependency (still need noble for non-Linux platforms, and still need bleno on all platforms for peripheral role support). To that end, I've ordered some BLE chips, based on reports in that noble ticket, that will hopefully let me reproduce the issue here.

But who knows, maybe this long shot works? :-)

[chriselsen]

Thanks so much for looking into this.

I tried #41, but unfortunately without success: Gymnasticon basically hangs the same way it does in the test script. Also I ported your hcitool changes into the small test script that I was using:

var noble = require('@abandonware/noble');
var child = require('child_process');

noble.on('stateChange', function(state) {
  if (state === 'poweredOn') {
    noble.startScanning(["1826"]);
    console.log('Scanning...');
  } else {
    noble.stopScanning();
  }
});

noble.on('discover', function(peripheral) {
  console.log('Device discovered...');
  noble.stopScanning();

  peripheral.connect(function(error){
    console.log('Connected...');
    var serviceUUID = ["1826"];
    var characteristicUUID = ["2ad2"];

    const handle = peripheral._noble._bindings._handles[peripheral.uuid];
    const minInterval = 24*1.25;
    const maxInterval = 40*1.25;
    const latency = 0;
    const supervisionTimeout = 420;
    const cmd = '/usr/bin/hcitool'
    const args = [
      'lecup',
      '--handle', `${handle}`,
      '--min', `${Math.floor(minInterval/1.25)}`,
      '--max', `${Math.floor(maxInterval/1.25)}`,
      '--latency', '0',
      '--timeout', `${Math.floor(supervisionTimeout/10)}`,
    ]
    child.execFileSync(cmd, args);

    peripheral.discoverSomeServicesAndCharacteristics(serviceUUID, characteristicUUID, function(error, services, characteristics){
      console.log('Discovering services...');
      characteristics[0].notify(true, function(error){
        console.log('Subscribing to services...');
        characteristics[0].on('data', function(data, isNotification){
          console.log('data is: ' + data);
          var power = buffer.readInt16LE(6);
          var cadence = buffer.readUInt16LE(4)/2;
          var speed = buffer.readUInt16LE(2)/100;
          var heartrate = buffer.readInt8(8);
          var datajson = '{ power: ' + power + ', cadence: ' + cadence + ' }';
          console.log(datajson);
        });
      });
    });
  });
});

Same issues still persists. Commenting out the call to the hcitool in #41 then also didn't work.

You said that noble hangs after sending a Read By Type Request for the CCCD descriptor, because it's not receiving a response. Does node-ble just write to that descriptor without reading it first? Would it make sense to change noble, so that it doesn't attempt to read (validate) first, but write right away.

Yesterday I implemented your 3. proposal above of using the "bot mode". But along the way I learned that it has some caveats:

• Noble and node-ble don't get along in parallel. So I commented out these lines of code. I'm wondering if it would make sense to add a flag to each bike which indicates whether noble is needed and only execute these lines if it is. The Peleton and Bot bike e.g. don't need it.
• Turns out that node-ble and bleno on the same device also don't get along. While no error messages are thrown, whichever app is started last, just isn't able to communicate over BLE. So I moved the IC-Bike client to a different Raspberry device.
• Running Gymnasticon on one Raspberry and the IC-Bike client on another Raspberry, I was able to ride for about 40-45 minutes via Zwift. The node-ble based client received the data from the IC-bike, send it via UDP to Gymnasticon. Gymnasticon then send it to Zwift via BLE and my Garmin watch via ANT+. So far, so good. But then node-ble based script died and I had to restart the bluetooth service to get it running again. I've seen a bunch of mentions that BlueZ - which is what node-ble uses - likes to hang after a while.

Next I'll either try to "harden" the node-ble based client, or port it to Python or alike. But I agree that getting this running inside of Gymnasticon with noble would be best.

Let me know if there is anything I can provide / help from my side.

[ptx2]

Would it make sense to change noble, so that it doesn't attempt to read (validate) first, but write right away.

Good idea! I pushed up a change to that same branch with two possible workarounds to try.

[chriselsen]

Success!

That worked! I only tried option 1, but with it I was able to connect to the IC bike and get data out of it. Did a short 2 minute test run and data flowed IC-Bike -> Gymnasticon -> Zwift | Garmin. Also waited for the IC Bike to back to sleep and after waking up the bike again, Gymnasticon reconnected without a problem. From my side everything is working.

Thanks so much for the help and great work on this!

Now I'll play around with the offset function in Gymnasticon. But my initial research shows that my bike is only about 5% off, while I've see reports from others where the reported power values were double of what they should be.

[ptx2]

Great news! Glad to hear it survived a sleep/wake. Thanks for the help finding and fixing this :-)

[ptx2]

Added in #41.