(Request) BT4 legacy and BT5 extended Remote ID sniffing

[alphafox02]

Carrying over from a comment made on YouTube, I’d like to formally request here as it seems in line with what Sniffle could potentially detect.

I’ve tested the following application on android https://github.com/dronetag/drone-scanner

Using a Samsung A7 Lite I was able to detect a holy stone remote ID Bluetooth transmitter emitting both BT4 and BT5 extended packets. https://store.holystone.com/products/holy-stone-drone-remote-id-module

I thought perhaps code within the drone scanner application could be adopted to sniffle in order for sniffle to accomplish the same thing (both detect and decode).

Sharing here as a suggestion as I have a CatSniffer and the holy stone transmitter in my possession.

[alphafox02]

Optionally sending decoded results to a network interface in json or some other standard would be great to for linkage of sniffle to another application I’m thinking of.

[sultanqasim]

You could use this Wireshark plugin for ASTM F3411 decoding with Sniffle: https://github.com/opendroneid/wireshark-dissector

You can use either the Sniffle Wireshark extcap plugin for real-time decoding, or use the Sniffle command line utilities to record a PCAP and open it with Wireshark.

[alphafox02]

I actually had this very thought a week or so ago and thank you for the reminder. I went back to the computer where I already had sniffle 1.7 loaded to the CatSniffer and Lua plugins in place. However I did not have the transmitter at that point, so I just fired it up.

I checked extended advertisement and long range and started sniffle in Wireshark. I do in fact see the packets being broadcast, however, they are not showing as opendroneID as they do with the sample pcap in the dissector project you pointed at. There maybe is some kind of difference in the standard? But this is promising. See side by side comparison.

[alphafox02]

A better side by side. I'm not sure what makes the opendroneId appear when opening the sample vs me streaming in via Sniffle. I'm assuming a difference in the packet provided with the opendroneID dissector vs the real life RemoteID being transmitted.

[sultanqasim]

Could you try with the latest Sniffle host software? I think it will (mostly) work even with the v1.7 firmware. One difference I saw between your screenshots is the labelling of auxiliary advertising PDUs (AUX_ADV_IND etc) which I recently improved in the host software.

If that doesn’t suffice, could you share a PCAP with me to investigate?

[sultanqasim]

To make the latest host software work with v1.7 firmware, you would need to revert: https://github.com/nccgroup/Sniffle/commit/4f379e0217e7c51c3c4f7c88115feeea18c615f0

Alternatively, you could update the CatSniffer firmware, but be extra careful to select the build variant matching your hardware, else you could lock yourself out of the bootloader. From my understanding, CatSniffer v1.0 to v2.1 uses the CC1352P1, while CatSniffer v3.x uses the CC1352P7.

[alphafox02]

I will sit down, hopefully tonight and test all of this and I will be very careful about the firmware. Also happy to provide a pcap ASAP.

[alphafox02]

Quick update. I flashed 1.9.3 to the CatSniffer with no issue. Pulled down the latest code from this repo. Sadly neither the command line or use of WireShark produced any results of any bluetooth. Also now when trying to use Wireshark with ALL channel option left to default it complains that a MAC address or IRQ must be filled in. This was not the case with 1.7.

If I drop back to 1.7 firmware and the older tool set everything starts working again. So perhaps I should try your other technique of using the older firmware and reverting a change to this repo.

[alphafox02]

Using the older firmware, but the newer Sniffle code. I reverted the sniffle_hw.py file to look like this after removing

marker_data = randbytes(4) self.cmd_marker(marker_data)

def mark_and_flush(self):
        # use marker to zero time, flush every packet before marker
        # also tolerate errors from incomplete lines in UART buffer
        recvd_mark = False
        while not recvd_mark:
            self.cmd_marker()
            timeout = time() + 0.5
            while time() < timeout:
                mtype, _, _ = self._recv_msg(True)
                if mtype == 0x12: # MarkerMessage
                    recvd_mark = True
                    break

This allowed WireShark to run, however the complaint about using a MAC address or IRQ when attempting to leave channel set all remained. I then manually set each of the three channels and obtained results with the extended and long range options selected. However, the results looked the same as with 1.7 best I can tell and the OpenDroneID drop down did not appear.

[alphafox02]

I will try to pass a pcap once I get to a different area or once I confirm gps location is not being acquired during capture.

[sultanqasim]

Quick update. I flashed 1.9.3 to the CatSniffer with no issue. Pulled down the latest code from this repo. Sadly neither the command line or use of WireShark produced any results of any bluetooth. Also now when trying to use Wireshark with ALL channel option left to default it complains that a MAC address or IRQ must be filled in. This was not the case with 1.7.

If I drop back to 1.7 firmware and the older tool set everything starts working again. So perhaps I should try your other technique of using the older firmware and reverting a change to this repo.

If you’re not getting any results with the v1.9.3 firmware on the CatSniffer, it might be that the CatSniffer is expecting a different baud rate or different pin assignments. I’ll look to see what the CatSniffer devs changed in their fork.

Regarding the wireshark extcap plugin, change the channel selection to auto instead of all. The all channel mode requires a target MAC address specified. Auto channel mode sniffs on channel 37 if no target MAC address or IRK is specified, or hops between advertising channels (like all channel mode) if a target is specified and extended advertising reception is turned off.

[sultanqasim]

Assuming you have a CatSniffer v3.x (with the CC1352P7), try the following firmware. It's firmware v1.9.3 for the CC1352P7 but with 921600 baud communications. Unzip before flashing of course.

sniffle_cc1352p7_1M.hex.gz

[alphafox02]

Okay, that firmware combined with the latest code from the repo (with mod you told me to make) does in fact get things back running with the CatSniffer. What’s interesting is even though I have extended advertisements and long range selected, also auto, I get all sorts of activity in wireshark beyond just the holy stone transmitter. Where as before with 1.7 from CatSniffer people and the 1.7 sniffle with all, extended, and long range selected I would only see the holy stone transmitter. Not sure why the difference.

It also seems maybe the packet doesn’t contain as much as just one did before? Here’s a picture to try and illustrate what I’m saying. This is with 1.9 firmware you provided, latest source with the revert you said to do (I tried it without reverting and nothing appeared in Wireshark).

[alphafox02]

I tried posting from the phone and things did not go so well. Hopefully things make sense, but for sure the changes you made to the firmware helped get things back moving on the CatSniffer.

[sultanqasim]

Thanks for testing that. It’s a bit strange that you needed to revert that change to mark_and_flush because the v1.9.3 firmware should work with the latest host software unmodified. I wonder if the new firmware I shared didn’t install correctly. It’s also strange that it’s sniffing on the normal 1M PHY instead of using the long range PHY, even though you have the long range PHY selected. Sniffing on the 1M PHY is why you’re seeing all that extra stuff and seeing shorter advertisements.

For sniffing long range (extended) advertising, the v1.7 and v1.9.3 firmware should function the same, aside from v1.9.3 not needing that revert I mentioned for v1.7 compatibility.

[alphafox02]

I'm retracing my steps, but the first thing that's confusing me is where I even got this firmware sniffle_CC1352P_7_v1.7.hex

I thought this was from the CatSniffer Firmware repo and hence my testing of Sniffle 1.7, but I just went back to the CatSniffer Firmware repo and looked and the firmware for the last 4 months is named sniffle.hex (so not sure where I got the firmware?).

I do however have a folder named Sniffle-1.7 which I'm positive I downloaded from your releases section.

This combo is what first seemed to work well. I should point out that it also had 5 optional check boxes in WireShark. Advertisments only, Extended, Hop, Long, and Ignore encrypted... which I think in 1.9 at least one of those are gone?

I'm going to reflash everything and get a better baseline of where I'm at.

[alphafox02]

I reflashed the firmware you provided. I went back to WireShark with the latest code from your repo (no edits). I figured out the issue is each time I'm making a change with the check boxes and/or starting stopping nothing then appears to work when I start again. I may have been in this state when I went to test.

So what I've found is that I unplug the CatSniffer. Plug it back in. Make the changes, like check extended and long range, click start on WireShark and I do in fact get just the Holy Stone Transmitter packets! So it would appear everything is now working, minus the fact when I stop and make changes to WireShark and start again, I would have to unplug the CatSniffer.

Actually just now I stopped WireShark and clicked Start. Zero packets. I unplug CatSniffer, plug back in, then click start and it saw all "anonymous" packets. But I'm repeating the process of stopping, unplugging, starting and haven't seen the Holy Stone transmitter again. I'll keep testing this, something seems strange with this behavior.

[sultanqasim]

I'm retracing my steps, but the first thing that's confusing me is where I even got this firmware sniffle_CC1352P_7_v1.7.hex

I thought this was from the CatSniffer Firmware repo and hence my testing of Sniffle 1.7, but I just went back to the CatSniffer Firmware repo and looked and the firmware for the last 4 months is named sniffle.hex (so not sure where I got the firmware?).

I do however have a folder named Sniffle-1.7 which I'm positive I downloaded from your releases section.

This combo is what first seemed to work well. I should point out that it also had 5 optional check boxes in WireShark. Advertisments only, Extended, Hop, Long, and Ignore encrypted... which I think in 1.9 at least one of those are gone?

I'm going to reflash everything and get a better baseline of where I'm at.

In the latest host software, I removed the hop checkbox, as it's equivalent to the new "all channel" mode.

[sultanqasim]

I reflashed the firmware you provided. I went back to WireShark with the latest code from your repo (no edits). I figured out the issue is each time I'm making a change with the check boxes and/or starting stopping nothing then appears to work when I start again. I may have been in this state when I went to test.

So what I've found is that I unplug the CatSniffer. Plug it back in. Make the changes, like check extended and long range, click start on WireShark and I do in fact get just the Holy Stone Transmitter packets! So it would appear everything is now working, minus the fact when I stop and make changes to WireShark and start again, I would have to unplug the CatSniffer.

Actually just now I stopped WireShark and clicked Start. Zero packets. I unplug CatSniffer, plug back in, then click start and it saw all "anonymous" packets. But I'm repeating the process of stopping, unplugging, starting and haven't seen the Holy Stone transmitter again. I'll keep testing this, something seems strange with this behavior.

When using the wireshark plugin, I always stop capture before changing settings. I'm not sure what would happen if you tried to change things while a capture is running. You shouldn't need to reset the sniffer to change settings though, something is a bit odd. Maybe your way of using it is triggering some bug, or maybe it's some CatSniffer-specific behaviour.

EDIT: I don't think Wireshark even allows changing settings mid capture. The host software (extcap plugin in this case) reconfigures the Sniffer every time you start a capture. I'm not sure why it would break every time you change settings after an initial capture, that's still perplexing to me.

[alphafox02]

It may be something CatSniffer specific. I'm going to test 1.7 here again in a second, but I can say with WireShark 4.2.4 when I click stop, wait for a second or so till it stops, then click start again there's no packets. I then perform, at the moment random, unchecking of extended and long range while it's stopped, plug back in and finally get all normal packets flowing again. Then I'll stop, unplug, check extended and long range, then click start again and finally get the expected packets flowing again.

I know it probably sounds crazy. I should just record it. I don't know the reason behind it all. I should also note that the OpenDroneID drop down is not appearing.

[alphafox02]

I drop back to 1.7 and Sniffle 1.7 code and I can start without issue, takes a long time to actually stop (I remember this now, seems near instant to stop with 1.9), but I can then click and start again without issue/without unplugging and plugging back in. It also seems like the Source and Destination columns are flipped around. Either that or my eyes are getting tired of looking at the screen.

I can also stop, change settings to extended and long, click start again without an issue. Very strange. But something for sure different between 1.7 and 1.9 I think.

[sultanqasim]

Regarding the v1.7 extcap plugin taking a long time to stop, cherry-picking these two commits included in later releases would fix it:

• 09aacf7d18f0e810e07cfeb653fd67a2ff578b8f
• 713d5a9dfc03c42e33125a04fbfd1eb9f03b6b9e

The sniffer not working after you stop it until you plug in again is very strange, probably a CatSniffer specific bug. You said that the CatSniffer developers own build of Sniffle firmware v1.7 allows stopping and restarting without issue? The CatSniffer has a RP2040 microcontroller between the host and the CC1352P acting as a UART man-in-the-middle; maybe it's doing something that breaks message sending from the host to my Sniffle v1.9.3 firmware.

I don't know if the CatSniffer developers changed something else in their own Sniffle firmware build beyond the baud rate, or if this change is causing breakage for the CatSniffer: https://github.com/nccgroup/Sniffle/commit/dec6240bb4cb48f4f169d81a3a618e69213a85ec

[alphafox02]

I’ll try those commits but regarding the 1.7 CatSniffer firmware. I’m not understanding where I got it or why I got onto using 1.7 to begin with. When I go back to the CatSniffer repo I can’t find any sign of it mentioning Sniffle 1.7 and the only firmware I find is called sniffle.hex

Yet on my computer I have Sniffle-1.7 directory and then this random file sniffle_CC1352P_7_v1.7.hex

I have to dig around more to see where I got this and why was I using 1.7. I can upload it here if it helps any.

[sultanqasim]

With regards to the OpenDroneID dropdown not appearing in Wireshark when capturing those advertisements, it's probably either a matter of configuring Wireshark to use the decoder, or maybe something about how Sniffle is labelling the advertisements slightly differently than the Nordic sniffer.

[alphafox02]

I’ll keep trying. I have the Nordic thing coming in the mail so that way I can determine if there’s any configuration needed in Wireshark, because right now if I open the example pcap provided in the sample files it just shows up.

Until I can record a pcap however you need me to, is there any way you can play back the one here in such a way that sniffle can process it? https://github.com/opendroneid/wireshark-dissector

Or would it be better that I record in such a way that would allow you to transmit it over the air as if it was coming from the holy stone and then you could use sniffle to process it in real time?

[alphafox02]

Here's the firmware that I can't recall where it came from or if I just simply renamed it. sniffle_firmware.zip

[sultanqasim]

I’ll keep trying. I have the Nordic thing coming in the mail so that way I can determine if there’s any configuration needed in Wireshark, because right now if I open the example pcap provided in the sample files it just shows up.

Until I can record a pcap however you need me to, is there any way you can play back the one here in such a way that sniffle can process it? https://github.com/opendroneid/wireshark-dissector

Or would it be better that I record in such a way that would allow you to transmit it over the air as if it was coming from the holy stone and then you could use sniffle to process it in real time?

Thanks, Sniffle does have advertising transmission capability, so I can just use a second Sniffle device to replay those advertisements from the sample to test. I'll try that when I have a moment.

[alphafox02]

I've not been able to figure out where I got the firmware shared above.

I went back to the official CatSniffer site and the sniffle.hex firmware they provide does not work with 1.7 or 1.9, so I have no idea what version of Sniffle they left off with supporting since no version is mentioned in their guides.

When I go to Sniffle releases and look under 1.7 there's not even a .hex available, so I'm at a complete lose. The firmware however appears to work really well with the 1.7 release.

I'll try and revert the change you mentioned above that you think may be causing an issue with CatSniffer and see if I can build a 1.9+ firmware myself.

[sultanqasim]

I've not been able to figure out where I got the firmware shared above.

I went back to the official CatSniffer site and the sniffle.hex firmware they provide does not work with 1.7 or 1.9, so I have no idea what version of Sniffle they left off with supporting since no version is mentioned in their guides.

When I go to Sniffle releases and look under 1.7 there's not even a .hex available, so I'm at a complete lose. The firmware however appears to work really well with the 1.7 release.

I'll try and revert the change you mentioned above that you think may be causing an issue with CatSniffer and see if I can build a 1.9+ firmware myself.

One possibility is that the CatSniffer RP2040 firmware's serial passthrough implementation is too slow and causes message receive timeouts with the Sniffle firmware. If that's the case, reverting the firmware change I mentioned above to return to operation without receive timeouts might fix it. The CatSniffer serial passthrough code didn't look particularly efficient to me, so it's plausible that it's causing issues.

It would be best if the CatSniffer developers themselves updated their Sniffle firmware version compatible with their device. I would be happy to officially support the CatSniffer hardware if they test for me and share their patches, or provide me with a device to test. They also ought to release the code they used for the Sniffle firmware build they posted on their GitHub firmware binary repository for GPL compliance. They have an outdated Sniffle fork here but that doesn't seem to be the actual code they used for their build because that fork is still running at 2M baud while their serial passthrough firmware expects 921600 baud.

I'd suggest using a Sonoff CC2652P dongle as a cheaper and better supported way to use Sniffle.

[alphafox02]

I’ve got one coming Friday, so I’ll be better able to test Sniffle and hopefully things like Remote ID for the moment. I’ll also try to rebuild firmware for the CatSniffer with that one change you mentioned. The mystery still remains as to where I got that one firmware, if I ever figure it out and there’s source provided I’ll make sure to share.

[alphafox02]

I'm all setup with a build environment, but I'm not sure how you pumped out the test Catsniffer firmware. When I run make or make PLATFORM=CC1352P74_1M I seem to end up with a sniffle.out file. I reverted the changes to do a quick test on the CatSniffer.

EDIT: I see the .hex, it was there all along.

[alphafox02]

I did a Git pull to get the very latest Sniffle. Reverted this change https://github.com/nccgroup/Sniffle/commit/dec6240bb4cb48f4f169d81a3a618e69213a85ec

Built the firmware with make PLATFORM=CC1352P74_1M

Flashed the CatSniffer with sniffle.hex and now it works, or so it appears, perfectly with WireShark. I can start/stop no issues. Stop, make changes to include extended/long range, and start right back up and receive results as expected from the Holy Stone transmitter.

The only thing that appears remaining is to sort out the OpenDroneID stuff.

[sultanqasim]

I did a Git pull to get the very latest Sniffle. Reverted this change dec6240

Built the firmware with make PLATFORM=CC1352P74_1M

Flashed the CatSniffer with sniffle.hex and now it works, or so it appears, perfectly with WireShark. I can start/stop no issues. Stop, make changes to include extended/long range, and start right back up and receive results as expected from the Holy Stone transmitter.

The only thing that appears remaining is to sort out the OpenDroneID stuff.

Excellent, thank you for testing this. I'm debating between implementing relaxed timeouts (either for everyone or just the CatSniffer build variant) or just removing the timeouts altogether for the CatSniffer-specific build variant (as you did with that revert).

I'm curious as to how slow the CatSniffer serial passthrough firmware is, particularly the timing gap between characters sent to the CC1352 over UART. Doing USB to UART serial passthrough efficiently in firmware is non-trivial, and using Arduino APIs with associated overhead from abstractions certainly doesn't help.

[alphafox02]

Just wanted to let you know I blew away messanger.c, did a pull git of the latest, rebuilt firmware without any edits and the behavior is the same good results I mentioned above. So the extra time out helped.

[sultanqasim]

Excellent, I was just about to share with you a build to test but you already did. Thanks for all your help!

[alphafox02]

No problem! Once the nRF52840 arrives I will sniff the remote id live and try to determine if the actual real standard is any different then the example captures. My Samsung Tablet with the DroneScanner app that’s based on the OpenDrone ID code does find the holy stone transmitter on BT5 extended. So I suppose if the nRF finds it with OpenDrone dissector we’re closer, but if it doesn’t, maybe just the dissectors are not correct.

[sultanqasim]

I figured out the issue with the OpenDroneID dissector. It just hadn't implemented support for the header format used by Sniffle and other open source BLE sniffers. Adding support for it was fairly simple (aside from me needing to teach myself a little Lua): https://github.com/sultanqasim/wireshark-dissector/commit/a570a1081b2736d251d68aaff11815f4ffad80b3

[alphafox02]

I just noticed this response. I'll go check it now, this is exciting! I'll let you know asap how it works.

[alphafox02]

It appears that everything, to include LAT/LONG being broadcast by the Holy Stone RemoteID transmitter is being decoded properly! This is awesome. Will this become native in Sniffle cli tools or will it require the Wireshark piece? Should I close ticket as it seems like it's working on the BT5 Extended/Long range? I know no the samsung tablet that it bounces between BT4 Legacy and BT5, so It's possible maybe the Holy Stone is transmitting both?

[sultanqasim]

If you sniff advertising on the regular (1M) PHY, you should see any legacy advertisements from the Holy Stone device. I'd consider this request resolved with my change to that Wireshark dissector. As for decoding this using the Sniffle Python CLI, in the past I've flip-flopped in my mind on adding such functionality. For now, I'm leaning towards letting Wireshark handle any higher level protocol decode, and limiting Sniffle's functionality to capture and decode of link layer information.

Less scope creep will help me to focus on making the basic functionality of Sniffle as good as I can, and avoid reinventing the wheel when Wireshark can decode things nicely.

[alphafox02]

Makes sense, thank you again for all the help and making this possible. The Sonoff device showed up today. I flashed it in like under a minute, fired up WireShark, and it worked perfectly. I used the release build instead of building my own, but all functioned without issue. Super simple! I'll be testing more with Sniffle now.