search

sparkfun / SparkFun_RTK_Firmware

Centimeter precision GPS/GNSS using L1/L2 signals broadcast over Bluetooth SPP (using the ESP32) in an easy to use enclosure.
https://docs.sparkfun.com/SparkFun_RTK_Firmware/
Other
81 stars 46 forks source link

BLE high frequency of disconnects #761

Open bendauphinee opened 5 months ago

bendauphinee commented 5 months ago

Subject of the issue

Over 8 hours yesterday, I ran a survey with the new 4.0 firmware. I've had occasional disconnects before, but the new version seems particularly unstable, with well over 50 disconnects over the course of my day. After adjusting the update frequency down from 0.25 sec to 0.33 sec, it seemed to stabilize some, but still had issues.

I tried noticing a pattern, but there was none apparent, with disconnects happening because of:

I guess maybe as a first step, if y'all count either generate a build to log whatever data you might need to diagnose this, or tell me what to turn on to log it to SD card, we can start there. I have another long survey run this next weekend likely, so it would be a good time to do this.

Your workbench

tonycanike commented 5 months ago

Can you tell us more about your hardware configuration please?

Which specific SparkFun hardware are you using? Facet? Facet LB? Surveyor? Express? Breakout board? etc? Are you using two SparkFun units in Base/Rover? Or are you using one and getting connections from a network source?

By "transmitting NTRIP", are you using IP to transmit your own RTCM from your base to your rover, or are you getting RTCM via NTRIP from a network provider and transmitting it to your rover? (The problem could have been in the network or at the RTCM provider.)

Can you describe what you mean by "disconnect" ?
Are you saying SWMaps on the iPhone disconnects from the bluetooth connection with the Facet (for example) ? Or is the NTRIP data stream interrupted (eg are you losing your internet connection) ? Or are you saying you are losing RTK Fix? Or something else?

bendauphinee commented 5 months ago

Great stack of question! You can find my detailed writeup here, but I'll summarize:

Using a SparkFun GPS-RTK-SMA Breakout – ZED-F9P transmitting via RTK Base to RTK2GO. Receiving via SW Maps, and passing it into my SparkFun RTK Facet via SW Maps.

By disconnect I mean there seems to be some Bluetooth disconnect happening, so it drops out of SW Maps. I had it do this once after reconnecting even before I had reconnected to NTRIP. Usually it reconnects with no issue when I connect back to it in SW Maps, but every once in a while it would lag and refuse to connect again until rebooted.

RTK performance was fine, and it would always be in RTK Fix right up until a connection drop. NTRIP performance is also fine, with both base and my phone on stable LTE connections.

I'm assuming it's BLE bandwidth related, due to the fact that turning down the update frequency cut way back on the disconnections. This is in contrast to 3.10, where I would only hit occasional disconnects, with the default update frequency.

nseidle commented 1 month ago

We've see a variety of issues with BLE on the v2.x of the ESP32 Arduino core. Without a way to replicate your exact issue/setup, I cannot explain why v3.x of the RTK Firmware was better. We've have had a lot more success on RTK Everywhere where we have more PSRAM on those products, so we are able to move to v3.0.x of the ESP32 core.

To clarify, are you running on your own hardware? Not a RTK product? If you're on your own hardware and up for some hacking, you could run RTK Everywhere on an ESP32 dev board with PSRAM.

If you can give us a way to replicate I'm happy to take a look.

bendauphinee commented 1 month ago

To clarify, are you running on your own hardware? Not a RTK product?

I'm running y'all's RTK Facet, I just dumped the reported hardware for reference.

If you can give us a way to replicate I'm happy to take a look.

I don't know that I can give you a way to replicate, which is why I suggested if you can do me a logging build, I can run it and give you the results if it's still flaky.

nseidle commented 1 month ago

Ah! I saw the Using a SparkFun GPS-RTK-SMA Breakout and was confused.

Are you running any settings that are of note? Did you turn on any additional messages?

bendauphinee commented 1 month ago

Other than being in BLE mode, tweaking the frequency down, and transmitting NTRIP back to device, no settings of note.

YashuSystems commented 1 month ago

I just acquired an RTK Express+ since I need the ZED-F9R. I also have a Facet w/ ZED-F9P which I obtained about a year ago. I have v4.1 running in both and I'm using the latest SW Maps as of this post date.... with iPhone 13.

I am seeing these BLE disconnects on the Express+ but not on the Facet. The disconnects are unacceptably frequent (typ less than 60 sec to disconnect after the associate from SW maps). The disconnect issue seems more prevalent once I start streaming RTCM corrections from the SW Maps client. So, this topology I was planning to use for my rover is now essentially inutile with the Express+.

I am unsure if there is some kind issue with the ZED-F9R vs. ZED-F9P. At this point, I am trying to determine if the Express+ has hardware issues and if I need to exchange it for a new unit or something. At this time, I do not have time to trace this BLE disconnect issue with my Ellisys Vanguard. Maybe in a few weeks or something(?).

Advise if Sparkfun plans to investigate and resolve this issue with a FW update. Otherwise I will need to create my own corrections stream path and simply deem the Express+ BLE link as unreliable. I also do not have time to solve the ESP32 FW -or- HW issue.

_**We've see a variety of issues with BLE on the v2.x of the ESP32 Arduino _core._** ... please elaborate

FYI.. I had to reposition the OLED on the Express+ since the enclosure cutouts registration on the Hammond enclosure were way off ( > 2mm) and the graphic overlay viewing window was therefore not aligned obscuring clear view of all OLED pixels. The hot glue attachment should also changed to screws for these rather expensive units. I modified my Express+ unit to use self-tapping screws.

YashuSystems commented 1 month ago

Here is a Vanguard trace of the anomalous single byte from the Express+ when it is streaming NMEA sentences to SW Maps. SW Maps then looks to instigate a disconnection sequence or something. Express+_SingleByteXmitDuringGPS-OutputStreamAnomaly_EllisysVanguard

pkramer509 commented 2 days ago

Dont have anything to add other than running out of box express+ 4.0 the BLE to ios/SW maps is functionally useless as it doesn't stay connected long enough to do anything useful.