ttu / ruuvitag-sensor

Python package for communicating with RuuviTag BLE Sensor and for decoding sensor data from broadcasted data
https://ttu.github.io/ruuvitag-sensor/
MIT License
196 stars 78 forks source link

Any plans for OSX? #18

Closed dcbb closed 4 years ago

dcbb commented 7 years ago

This is awesome and I'd love to get started right away as my tags arrived today. But I'm on a mac.

Are there plans for supporting readout on OSX any time soon?

There seems to be an open source project around Eddystone on mac in JavaScript / node, but I'd sure love to stay with Python.

ttu commented 7 years ago

I was checking this last year, but couldn't find any implementation that would have worked with Linux, OSX and Windows. Couldn't even find Bluetooth command line tools for Windows, so didn't check OSX option either.

Code base already has handling for different BLE communication implementation for each OS, but haven't had time to figure out actual implementations and as most people seem to be using RaspberryPI with RuuviTags this hasn't had high priority.

So for now with OSX I would use https://github.com/sandeepmistry/noble or some RuuviTag node package e.g. https://github.com/Espesen/node-ruuvitag.

dcbb commented 7 years ago

Thanks for the comment and the links. I'll have a look. I'm actually targeting Raspi in the end as well, but my dev system is a mac. Btw, it works on Mac in a Ubuntu VM.

ttu commented 7 years ago

Most of the time I do my development on Windows and then execute the code on Raspberry. Too lazy to start Ubuntu dev machine :) Maybe should also consider running Ubuntu VM.

What software you use to run VM on mac? So other people who have do development on mac also know what works.

samuelphy commented 7 years ago

I'm also keen to know which dev setup that works on a Mac!

ttu commented 7 years ago

Virtual Machines

Docker doesn't support using host's Bluetooth devices. Neither does Windows 10's Hyper-V. One option is apparently to use VirtualBox (or maybe some VMWare applications).

Cross Platform Bluetooth Library

Have been trying to find some cross platform bluetooth communication package, with support for BLE, like node's Noble, but haven't found anything so far. Python has socket's that can be used to communicate with Bluetooth, but haven't figured out how I could use that with Windows or macOS.

If someone has a solution that would work with Linux/macOS/Windows, I can gladly implement it. It can be Python3+ and Python 2.7 can have support only for Linux.

ukBaz commented 7 years ago

@WayneKeenan and @IanHarvey had some ideas about how to make a Python Bluetooth library cross platform. Anything you can share?

havnjero commented 6 years ago

has this come any closer to fruition? also very interested in a native mac osx app or cron

ttu commented 6 years ago

@havnjero I have been checking Bleson. It is still in alpha stage and last time I tried it, scanning BLE devices didn't return any device data, but I will have to inspect that more. Pretty likely will eventually create alternative communication with that, but still have to wait for a while.

ttu commented 6 years ago

Tried Bleson with Linux and it could find HCI data. Will start to implement optional communication with Bleson.

@havnjero Could you try if Bleson's debug output shows some RuuviTag data?

$ pip install git+https://github.com/TheCellule/python-bleson
$ python -m bleson --observer --debug
ttu commented 6 years ago

Added first implementation with Bleson to own branch: https://github.com/ttu/ruuvitag-sensor/tree/bleson-ble-communication

Works with Linux. On Windows finds ble devices, but doesn't return any data. Haven't tried with macOS.

Must be installed with dependency-links as Bleson is installed from GitHub

$ pip install -e . --process-dependency-links
rwfnf commented 6 years ago

Just tried the bleson-ble-communication branch on Debian and it seems to work, but I get some errors and the callback only seems to get called once rather than for each broadcast. See below, any ideas?

In [1]: from ruuvitag_sensor.ruuvi import RuuviTagSensor
   ...: def handle_data(found_data):
   ...:     print('MAC ' + found_data[0])
   ...:     print(found_data[1])
   ...: 
   ...: RuuviTagSensor.get_datas(handle_data)
   ...: 

2018-05-02 14:39:01,896   INFO -                 ruuvi.py:124 -                get_datas(): Get latest data for sensors. Stop with Ctrl+C.
2018-05-02 14:39:01,897   INFO -                 ruuvi.py:125 -                get_datas(): MACs: []
2018-05-02 14:39:01,897   INFO -     ble_communication.py:169 -                    start(): Start receiving broadcasts (device 0)
MAC EF:13:DA:80:A2:EE
{'measurement_sequence_number': 12145, 'acceleration_y': -400, 'acceleration_x': -308, 'tx_power': 4, 'mac': 'eea280da13ef', 'humidity': 53.13, 'acceleration': 1021.473445567725, 'temperature': 13.71, 'pressure': 1000.92, 'movement_counter': 127, 'acceleration_z': 888, 'battery': 3.157}
2018-05-02 14:39:08,612 WARNING -       type_converters.py:193 -          from_hcipayload(): TODO: Unhandled GAP type, pos=10 type=0x19 len=3
2018-05-02 14:39:08,613 WARNING -       type_converters.py:194 -          from_hcipayload(): Data: 01 04 01 ef 13 da 80 a2 ee 12 03 19 00 00 02 0a 04 0a 09 52 75 75 76 69 31 33 65 66 
2018-05-02 14:39:08,613 WARNING -       type_converters.py:195 -          from_hcipayload(): POS : 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 
2018-05-02 14:39:08,613 WARNING -       type_converters.py:193 -          from_hcipayload(): TODO: Unhandled GAP type, pos=14 type=0x0a len=2
2018-05-02 14:39:08,613 WARNING -       type_converters.py:194 -          from_hcipayload(): Data: 01 04 01 ef 13 da 80 a2 ee 12 03 19 00 00 02 0a 04 0a 09 52 75 75 76 69 31 33 65 66 
2018-05-02 14:39:08,613 WARNING -       type_converters.py:195 -          from_hcipayload(): POS : 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

After this nothing else gets listed.

holitics commented 6 years ago

Just FYI, I tried quickly on macOS (10.13.4) and could not get it to work.

WayneKeenan commented 6 years ago

"Could not get it to work"

So, did you get an error or did you get no real useful info?

If it's the later I'm not surprised.

Thanks, Wayne

On 2 May 2018 at 16:05, holitics notifications@github.com wrote:

Just FYI, I tried quickly on OSX and could not get it to work.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ttu/ruuvitag-sensor/issues/18#issuecomment-386010328, or mute the thread https://github.com/notifications/unsubscribe-auth/AH7ZNjWphj29G17Gj7r_x3K8tiCX282Aks5tucs2gaJpZM4O60wR .

WayneKeenan commented 6 years ago

A known issue, but fixed I think, but the possible fix is not yet in PyPi.

Thanks, Wayne

On 2 May 2018 at 14:40, rwfnf notifications@github.com wrote:

Just tried the bleson-ble-communication branch on Debian and it seems to work, but I get some errors and the callback only seems to get called once rather than for each broadcast. See below, any ideas?

In [1]: from ruuvitag_sensor.ruuvi import RuuviTagSensor ...: ...: def handle_data(found_data): ...: print('MAC ' + found_data[0]) ...: print(found_data[1]) ...: ...: RuuviTagSensor.get_datas(handle_data) ...: 2018-05-02 14:39:01,896 INFO - ruuvi.py:124 - get_datas(): Get latest data for sensors. Stop with Ctrl+C. 2018-05-02 14:39:01,897 INFO - ruuvi.py:125

  • get_datas(): MACs: [] 2018-05-02 14:39:01,897 INFO - ble_communication.py:169 - start(): Start receiving broadcasts (device 0) MAC EF:13:DA:80:A2:EE {'measurement_sequence_number': 12145, 'acceleration_y': -400, 'acceleration_x': -308, 'tx_power': 4, 'mac': 'eea280da13ef', 'humidity': 53.13, 'acceleration': 1021.473445567725, 'temperature': 13.71, 'pressure': 1000.92, 'movement_counter': 127, 'acceleration_z': 888, 'battery': 3.157} 2018-05-02 14:39:08,612 WARNING - type_converters.py:193 - from_hcipayload(): TODO: Unhandled GAP type, pos=10 type=0x19 len=3 2018-05-02 14:39:08,613 WARNING - type_converters.py:194 - from_hcipayload(): Data: 01 04 01 ef 13 da 80 a2 ee 12 03 19 00 00 02 0a 04 0a 09 52 75 75 76 69 31 33 65 66 2018-05-02 14:39:08,613 WARNING - type_converters.py:195 - from_hcipayload(): POS : 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 2018-05-02 14:39:08,613 WARNING - type_converters.py:193 - from_hcipayload(): TODO: Unhandled GAP type, pos=14 type=0x0a len=2 2018-05-02 14:39:08,613 WARNING - type_converters.py:194 - from_hcipayload(): Data: 01 04 01 ef 13 da 80 a2 ee 12 03 19 00 00 02 0a 04 0a 09 52 75 75 76 69 31 33 65 66 2018-05-02 14:39:08,613 WARNING - type_converters.py:195 - from_hcipayload(): POS : 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

After this nothing else gets listed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ttu/ruuvitag-sensor/issues/18#issuecomment-385981083, or mute the thread https://github.com/notifications/unsubscribe-auth/AH7ZNhS-TpxP5Iw_FkY9i2BZl1micv4bks5tubdKgaJpZM4O60wR .

holitics commented 6 years ago

Sorry, I only spent a couple minutes on it. This works great on Raspian. I was trying to get it working on MacOS native (as that is my development environment). It still gets the dummy data by default. If I go into ruuvi.py and modify line 11 to force it to not use the dummy BLE adapter (BleCommunicationDummy) by adding "Darwin" in there, the ble.get_data() function on line 148 returns no data. Digging in further shows that an external process sudo command calling to hcidump is failing.

If someone would like me to dig further (with some hints on how to proceed, what to install, etc.), please let me know.

Thanks!

ttu commented 6 years ago

@holitics Bleson implementation is not in the main branch yet, so you need to use code from bleson-ble-communication branch

$ git clone https://github.com/ttu/ruuvitag-sensor.git
$ cd ruuvitag-sensor
$ git checkout bleson-ble-communication
$ pip install -e . --process-dependency-links
ttu commented 6 years ago

@rwfnf try to re-install packages. Setup.py has a bleson dependency from GitHub, so callback should now be called for each bradcast

WayneKeenan commented 6 years ago

From what recall on macOS CoreBluetooth only gives a couple of fields like device name, rssi and maybe manufacturer data. Would need to double check.

WayneKeenan commented 6 years ago
2018-05-04 16:47:19,677  DEBUG -         macos_adapter.py:176 - centralManager_didDiscoverPeripheral_advertisementData_RSSI_(): Found: 
name=Chromecast rssi=84 data={
    kCBAdvDataIsConnectable = 0;
    kCBAdvDataManufacturerData = <e000010c ca338ec1>;
} 
holitics commented 6 years ago

OK, @ttu I updated accordingly. Running the code now uses the CoreBluetoothAdapter, but I seem to get no manufacturer data...

Advertisement(flags=0x00, name='N03BG', txpower=None, uuid16s=[UUID16(0xfeaf)], uuid128s=[], rssi=-73, mfg_data=None)

WayneKeenan commented 6 years ago

I'm probably not passing it thru inside bleson.

WayneKeenan commented 6 years ago

Or, manufacturer data is not in the advertisement but in a scan response for this device. I forget what's in the spec, and I don't recall bleson doing scan request/response handling

WayneKeenan commented 6 years ago

Can anyone confirm what the Ruuvi tag Advertisment packet is and what a ScanResponse would show please?

ojousima commented 6 years ago

The actual payload data is in advertisement packet as Manufacturer Specific Data. Some special firmwares include things such as complete name in scan response, but it does not matter for this application. Example packet from HCIdump: 04 3E 25 02 01 03 01 86 BC F2 71 67 E1 19 02 01 04 15 FF 99 04 03 65 17 5F CB 25 FF F1 FF F5 03 DC 0B 77 00 00 00 00 B8

Ruuvi data is 15 FF 99 04 03 65 17 5F CB 25 FF F1 FF F5 03 DC 0B 77 00 00 00 00 B8, i.e. "21 bytes of manufacturer specific data from Ruuvi Innovations" + the actual payload + RSSI

rwfnf commented 6 years ago

In regards to my issue above, I figured out the problem. It was actually to do with Bleson, there was a bug in Bleson that disabled continuous monitoring on linux systems that was patched 17 days ago: https://github.com/TheCellule/python-bleson/pull/41

Installing via "pip3 install bleson" didn't include this patch, so I git cloned and installed: git clone https://github.com/TheCellule/python-bleson.git sudo pip3 install -e . --process-dependency-links

ttu commented 4 years ago

Optional Bleason communicaton released. Check README and issue #78.