Data updates slower than in the Victron app device list

[DrRob]

Hi,

More a question than an issue – in the first screen of the Victron Android app ("Device list"), the overview values that are displayed without connecting to a device seem to update a lot quicker than this library receives updates. It's maybe once a second in the app, whereas the victron-ble library seems to receive updated values a lot slower, and it seems to vary by device – I have an Orion Smart, SmartSolar and SmartShunt – it's about every 10s for the Orion Smart and the SmartSolar, and many 10s of seconds for the SmartShunt. Is that to be expected? Does the app do something to increase the broadcast update rate even before connecting to a specific device?

Thanks, Rob.

[bobemoe]

I too have exact same issue with a SmartSolar I'm seeing updates at about 10s too. I'm not sure how to resolve it here as I ended up writing my own script using bluepy instead of bleak so can answer the question:

Does the app do something to increase the broadcast update rate even before connecting to a specific device?

No. The advertising data is frequent, I can detect it at a rate of 1s without having to ask for faster data. I do have to interrupt the scanner and start it again once I get the data to avoid waiting for the scanner timeout to restart.

Maybe something similar needs to be done here.

I'd much prefer to use this app as its much cleaner than my hacked together script!

[DrRob]

I found the source of the problem here: https://github.com/keshavdv/victron-ble/blob/main/victron_ble/scanner.py#L34

By removing the use of the _seen_data buffer I'm now getting updates every second or so, and they're not duplicated values either. The problem with this code is that it's only looking for duplicated keys and is ignoring the fact that the values may have changed.

[DrRob]

Ah, sorry, it's not quite as simple as that. I had to do that AND call stop() then start() inside the callback, similar to what bobemoe described above. I forgot I'd put those two lines of code in there.

[Volker-NDE]

Hi Rob, I've experienced the same problem with my SmartShunt 500. Program is running pretty well, but BLE data is only updated every 11 seconds. I tried to copy your solution, but it didn't work out, interval is still unchanged. Where exactly did you place the stop() and start() code? Can you post that part of your code? 😊🙏

[DrRob]

Where exactly did you place the stop() and start() code? Can you post that part of your code? 😊🙏

What I should have done was fork the repo, apply my changes, rebuild the package and install it. What I actually did was edit the Python package in-situ. I'm a bad person.

On my Pi, in /usr/local/lib/python3.9/dist-packages/victron_ble, in __pycache__, I did sudo rm *.pyc, and then I edited scanner.py, in BaseScanner, in _detection_callback:

        #if not data or not data.startswith(b"\x10") or data in self._seen_data:
        if not data or not data.startswith(b"\x10"):
            return

And then in my script, which uses the library, in class Scanner(BaseScanner), in the callback, which in my case unpacks the data and then rebroadcasts it to an MQTT server, I added stop() and start():

    def callback(self, ble_device: BLEDevice, raw_data: bytes):
        try:
            logger.info(f"Received data from {ble_device.address.lower()}: {raw_data.hex()}")
            try:
                device = self.get_device(ble_device, raw_data)
            except AdvertisementKeyMissingError:
                return
            except UnknownDeviceError as e:
                logger.error(e)
                return
            parsed = device.parse(raw_data)

            asyncio.ensure_future(super().stop())
            asyncio.ensure_future(super().start())

            asyncio.ensure_future(self.client.publish("victron/" + ble_device.address, payload=json.dumps(parsed, cls=DeviceDataEncoder)))

        except MqttError as error:
            logger.error(f'MqttError "{error}". Restarting...')
            sys.exit(1)
        except:
            traceback.print_exception(*sys.exc_info())
            logger.error('Restarting...')
            sys.exit(1)

Sorry that's a bit hand-wavey – I hope it helps.

[Volker-NDE]

Yes, that definitely help, thanks for your quick reply! Now I am down to an interval of about one second. 😃 Indeed, I also decided to change scanner.py directly. I even made a copy of it a module in my project. As my BLE connection is running in a dedicated thread, I had to add some code to realize the data handover from async function to thread. 🙂