Reconnecting because we received 0 bytes from the socket even though it said there was data available 5 times

[Hadain]

Hi,

I struggle to get valid correction data, and I have run out of ideas to solve the issue. I entered my credentials in the launch file and defined the service provider (Norway, CPOS) (host:port/mountpoint).

I see that the node is connected to the server, but the connection reset since 0 bytes were read from the packages while data was available 5 times.

It is confirmed that the login is fine, the service is available, and it works with other GNSS devices.

I am running ROS noetic on a Raspberry pi 3, with Ubuntu Server 20. UFW is configured to allow all in and out connections to the host IP.

I would appreciate any suggestions.

[arpaterson]

I am also seeing this with SaPOS, in Germany. It is my first attempt to use ntrip_client or NTRIP casters in general.

Be sure not to spam the server with log in attempts.

My initial wild guess is about VRS. Is the NTRIP caster waiting for an nmea message so it can supply a virtual reference station near your rover?

[olvdhrm]

I am running the package on ROS 2 Humble and am coming across with the same problem.

It was able to connect to the given IP address and port successfully, however, the following warning appears: Reconnecting because we received 0 bytes from the socket even though it said there was data available 5 times

Has this problem been addressed? Would really appreciate the help.

[arpaterson]

I am running the package on ROS 2 Humble and am coming across with the same problem.

It was able to connect to the given IP address and port successfully, however, the following warning appears: Reconnecting because we received 0 bytes from the socket even though it said there was data available 5 times

Has this problem been addressed? Would really appreciate the help.

I don’t know how deeply familiar you are with the working of RTK, and NTRIP.

Does your NTRIP caster require you to send your rover’s position - a $GPGGA message?

Casters may need the rover position for a few reasons - for VRS, it is used to set up a virtual reference station right next to your rover, for ‘nearest’ it is used to determine the nearest physical reference station and thus supply you with correction data from the right one, but the simplest reason is validity - even if there is only one reference station, if you are too far from it, it should not give you correction data.

If you haven’t supplied ntrip_client with GPGGA (it will consume and forward nmea/Sentence messages if recall) you might want to look into that.

Kumarrobotics ublox_gps outputs these unprocessed sentences alongside NavSatFix as of a certain recent commit for example.

[olvdhrm]

I have just recently started looking into RTK and NTRIP hence am relatively new to the topic.

The configuration used for the NTRIP client connection (host, port, mount point, and credentials) has previously been tested and used from the u-center application and was able to provide the RTCM corrections. So, I was ruling out the possibility of error due to the NTRIP caster - is this alright?

Yes, I am currently trying to use this package alongside the ublox driver package by kumarrobotics (ublox_gps).

Are there specific configurations regarding the $GPGGA message required through the package(s) or should it be within the antenna and configured through the u-center?

Thank you @arpaterson

[olvdhrm]

Hello, I ended up using a fix to NMEA converter and was able to receive the RTCM data after passing the NMEA sentence to the NTRIP Client. Thank you!

[arpaterson]

You shouldn’t need a translator node, if you use ublox_gps from this commit or later : https://github.com/KumarRobotics/ublox/commit/fc9fb4f0a4b4c52b988b50fdc4f47aff20aa2f79

Ublox_gps now outputs nmea sentences from the receiver without changing them, if configured correctly, ready for ntrip_client to forward to the caster. Before this commit I had written a node that reassembled a GGA sentence well enough, but it’s no longer needed.

Try that commit

[arpaterson]

If the problem is solved, you can close this issue.

[TheExplorer95]

I'm facing the same problem right now and would appreciate if @arpaterson could point out which parameter has to be set within the config file. Currently, I'm using this as a base and tested some other configurations, but that did not work out. Thanks in advance, been trying to figure out how to get the raw NMEA sentences for a few days now.

[TheExplorer95]

@arpaterson just found your kumbarRobotics/Ublox issue, which relates to the problem outlined here link. Would you mind sharing your f9p config file for reference?

[yulan0215]

NMEA converter

Hi, how did you solve this problem? Thx

[Hadain]

Hi, Thanks for the feedback, I followed arpaterson suggestion, and in my case, the problem was indeed, that the NTRIP caster required the rover's position. Strangely, only the GPGGA format worked.

My receiver is not providing raw NMEA messages, so I had to update the _subscribenmea(self, nmea) function to consume the processed message type and then send the NMEA sentence to the server. My solution includes a separate class to handle the subscriptions and the incoming messages. I run it in a separate thread, then when I got a new message, I translate it into a GPGGA message then send it to the caster.

ntrip_client/scripts/ntrip_ros.py:

from ntrip_client.nmea_subscriber import NMEASubscriber

class NTRIPRos:
  def __init__(self):
    [..]
    self._subscriber = NMEASubscriber( arguments ) #Initiate the new subscriber class

    #Initialize publisher thread to periodically send data to CPOS server
    self.worker_rate = 1 #hz
    self.worker = threading.Thread(target=self.send_rtcm_to_server)
    [..]

   def run(self):
     [..]
    self._subscriber.start_subscriber()  #start the subscriber
    self.worker.start() #Start the publisher thread to periodically send data to CPOS server
    [..]

  def send_rtcm_to_server(self):
    rate = rospy.Rate(5) # ROS Rate at 5Hz
    rospy.loginfo("Service: send_rtcm_to_server is initiated")
    while not rospy.is_shutdown():
      if not self._subscriber.latest_sentence == None:
        rospy.logdebug("Sending data: {}".format(self._subscriber.latest_sentence))
        self._client.send_nmea(self._subscriber.latest_sentence)
      rate.sleep()

new file: /src/ntrip_client/nmea_subscriber.py

import logging
import rospy
from time import strftime, gmtime

from nmea_msgs.msg import Sentence, Gpgga
from gps_common.msg import GPSFix

class NMEASubscriber:
    def __init__(self, msg_type, logerr=logging.error, logwarn=logging.warning, loginfo=logging.info, logdebug=logging.debug):
        # Bit of a strange pattern here, but save the log functions so we can be agnostic of ROS
        self._logerr = logerr
        self._logwarn = logwarn
        self._loginfo = loginfo
        self._logdebug = logdebug

        self._subs_msg_type = msg_type

        self.latest_sentence = None

        self._subs_msg_type_sw = {
            "Sentence": (Sentence, self.subscribe_nmea),
            "GPSFix": (GPSFix, self.subscribe_gpsfix),
        }

        self._supported_msg_types = {
            "Sentence": Sentence,
            "GPSFix": GPSFix,
        }

        if self._subs_msg_type in self._subs_msg_type_sw:
            self._logdebug("Message type for subscription is supported (type: {})".format(self._subs_msg_type))
            self._loginfo("Message type for subscription is supported (type: {})".format(self._subs_msg_type))

        else:
            self._logerr("Not supported Message type for subscription (type: {})".format(self._subs_msg_type))
            self._loginfo("Supported types: {}".format(self._supported_msg_types))

    def start_subscriber(self):
        # Start the subscriber
        rospy.loginfo("Subscriber initiated")
        self._subscriber = rospy.Subscriber('nmea', 
                                      (self._subs_msg_type_sw.get(self._subs_msg_type, (Gpgga, self.subscribe_gpgga)))[0],
                                      (self._subs_msg_type_sw.get(self._subs_msg_type, (Gpgga, self.subscribe_gpgga)))[1],
                                      queue_size=10)

    def _ToDMM(self, dd, latlon):
        dd1 = abs(float(dd))
        cdeg = int(dd1)
        minsec = dd1 - cdeg
        cmin = minsec * 60
        if latlon == "lat":
            return "{:02d}{:08.5f}".format(cdeg,cmin)
        else:
            return "{:03d}{:08.5f}".format(cdeg,cmin)

    def _calc_checksum(self, string):
        calculated_checksum =0
        for char in string[1:]:
            calculated_checksum ^= ord(char)
        hashid = "{:x}".format(calculated_checksum)
        nmea_sentence = "{}*{}".format(string,hashid)
        return nmea_sentence

    def gpsfix_to_gpgga(self, data, now):

        # if true, RTK (GBAS,4), else NoFix(0), unagFix(1), SBAS(2)
        quality =  data.status.status+2 if (data.status.status == 2) else (data.status.status+1 if (data.status.status < 17) else 3)

        lati = self._ToDMM(data.latitude, "lat")
        long = self._ToDMM(data.longitude, "lon")
        sentence = (  f"$GPGGA,"
                    f"{strftime('%H%M%S', gmtime(now.secs))}."
                    f"{int((now.nsecs/1000000)):02d},"
                    f"{lati},"
                    f"N,"
                    f"{long},"
                    f"E,"
                    f"{quality},"
                    f"{data.status.satellites_used:02d},"
                    f"{data.hdop:.2f},"
                    f"{data.altitude:.2f},"
                    f"M,"
                    f"40.66,"
                    f"M,,"
        )
        return self._calc_checksum(sentence)

    def nmea_sentence_relay(self, nmea):
        # Just extract the NMEA from the message, and send it right to the server
        #self._client.send_nmea(nmea.sentence)
        return nmea.sentence

    def subscribe_nmea(self, data):
        # Just extract the NMEA from the message, and send it right to the server
        #self._client.send_nmea(nmea.sentence)
        self.latest_sentence = self.nmea_sentence_relay(data)

    def subscribe_gpsfix(self, data):
        now = rospy.get_rostime()
        self.latest_sentence = self.gpsfix_to_gpgga(data, now)

I hope this will help with similar issues.

[yulan0215]

self._subscriber = NMEASubscriber( arguments ) #Initiate the new subscriber class

Hello, I am sorry I had a stupid question here: self._subscriber = NMEASubscriber( arguments ) #Initiate the new subscriber class

here, what should I write for the arguments? Because the error from my side was the arguments was not defined... Or it will be better if you can provide the Full Script of: ntrip_ros.py:

[Hadain]

I have to cut out this section from my code, but you can see the arguments in the class definition: `class NMEASubscriber: def init(self, msg_type, logerr=logging.error, logwarn=logging.warning, loginfo=logging.info, logdebug=logging.debug):

Bit of a strange pattern here, but save the log functions so we can be agnostic of ROS`

so the minimum argument is the msg_type, where at the moment the accepted types are: "Sentence": Sentence, "GPSFix": GPSFix,

so a definition would be like this:

self._subscriber = NMEASubscriber(msg_type="GPSFix")

[yulan0215]

I have to cut out this section from my code, but you can see the arguments in the class definition: class NMEASubscriber: def __init__(self, msg_type, logerr=logging.error, logwarn=logging.warning, loginfo=logging.info, logdebug=logging.debug): # Bit of a strange pattern here, but save the log functions so we can be agnostic of ROS

so the minimum argument is the msg_type, where at the moment the accepted types are: "Sentence": Sentence, "GPSFix": GPSFix,

so a definition would be like this:

self._subscriber = NMEASubscriber(msg_type="GPSFix")

Thanks for your help and right now I got the RTCM msg... However, another problem occurred, I publish the rtcm ROS msg to feed my Ublox here: https://github.com/KumarRobotics/ublox/blob/master/ublox_gps/src/node.cpp and in this script, I added my RTCM rostopic here: https://github.com/KumarRobotics/ublox/blob/4f107f3b82135160a1aca3ef0689fd119199bbef/ublox_gps/src/node.cpp#L1908 However, I did not get the RTK solution, when I check my /ublox_gps/fix, the status was 0 which means I did not get RTK solution from RTCM, do you have any idea about that?

[yulan0215]

I have to cut out this section from my code, but you can see the arguments in the class definition: class NMEASubscriber: def __init__(self, msg_type, logerr=logging.error, logwarn=logging.warning, loginfo=logging.info, logdebug=logging.debug): # Bit of a strange pattern here, but save the log functions so we can be agnostic of ROS so the minimum argument is the msg_type, where at the moment the accepted types are: "Sentence": Sentence, "GPSFix": GPSFix, so a definition would be like this: self._subscriber = NMEASubscriber(msg_type="GPSFix")

Thanks for your help and right now I got the RTCM msg... However, another problem occurred, I publish the rtcm ROS msg to feed my Ublox here: https://github.com/KumarRobotics/ublox/blob/master/ublox_gps/src/node.cpp and in this script, I added my RTCM rostopic here: https://github.com/KumarRobotics/ublox/blob/4f107f3b82135160a1aca3ef0689fd119199bbef/ublox_gps/src/node.cpp#L1908 However, I did not get the RTK solution, when I check my /ublox_gps/fix, the status was 0 which means I did not get RTK solution from RTCM, do you have any idea about that?

Another problem was that when I ran the Ntrip I saw the problem here: Do you also have any idea about that? Thx

[Hadain]

I have to cut out this section from my code, but you can see the arguments in the class definition: class NMEASubscriber: def __init__(self, msg_type, logerr=logging.error, logwarn=logging.warning, loginfo=logging.info, logdebug=logging.debug): # Bit of a strange pattern here, but save the log functions so we can be agnostic of ROS so the minimum argument is the msg_type, where at the moment the accepted types are: "Sentence": Sentence, "GPSFix": GPSFix, so a definition would be like this: self._subscriber = NMEASubscriber(msg_type="GPSFix")

Thanks for your help and right now I got the RTCM msg... However, another problem occurred, I publish the rtcm ROS msg to feed my Ublox here: https://github.com/KumarRobotics/ublox/blob/master/ublox_gps/src/node.cpp and in this script, I added my RTCM rostopic here: https://github.com/KumarRobotics/ublox/blob/4f107f3b82135160a1aca3ef0689fd119199bbef/ublox_gps/src/node.cpp#L1908 However, I did not get the RTK solution, when I check my /ublox_gps/fix, the status was 0 which means I did not get RTK solution from RTCM, do you have any idea about that?

I am glad, that you managed to receive the RTCM messages from the server, so we can assume that this issue is solved. However, I am not familiar with this driver. I would check whether the messages are in the correct format. Also, check that the published topic is in the same format as the subscriber expects. It might be that you need to make some adjustments, for example, add or remove the header.

[Hadain]

I have to cut out this section from my code, but you can see the arguments in the class definition: class NMEASubscriber: def __init__(self, msg_type, logerr=logging.error, logwarn=logging.warning, loginfo=logging.info, logdebug=logging.debug): # Bit of a strange pattern here, but save the log functions so we can be agnostic of ROS so the minimum argument is the msg_type, where at the moment the accepted types are: "Sentence": Sentence, "GPSFix": GPSFix, so a definition would be like this: self._subscriber = NMEASubscriber(msg_type="GPSFix")

Thanks for your help and right now I got the RTCM msg... However, another problem occurred, I publish the rtcm ROS msg to feed my Ublox here: https://github.com/KumarRobotics/ublox/blob/master/ublox_gps/src/node.cpp and in this script, I added my RTCM rostopic here: https://github.com/KumarRobotics/ublox/blob/4f107f3b82135160a1aca3ef0689fd119199bbef/ublox_gps/src/node.cpp#L1908 However, I did not get the RTK solution, when I check my /ublox_gps/fix, the status was 0 which means I did not get RTK solution from RTCM, do you have any idea about that?

Another problem was that when I ran the Ntrip I saw the problem here: Do you also have any idea about that? Thx

As the error message says, the format of the fed NMEA sentence is wrong. Are you using the relay (so you get the NMEA from your receiver directly), or are you using my code to translate the ROS topic (GPSfix) message into an NMEA sentence?

[yulan0215]

I have to cut out this section from my code, but you can see the arguments in the class definition: class NMEASubscriber: def __init__(self, msg_type, logerr=logging.error, logwarn=logging.warning, loginfo=logging.info, logdebug=logging.debug): # Bit of a strange pattern here, but save the log functions so we can be agnostic of ROS so the minimum argument is the msg_type, where at the moment the accepted types are: "Sentence": Sentence, "GPSFix": GPSFix, so a definition would be like this: self._subscriber = NMEASubscriber(msg_type="GPSFix")

Thanks for your help and right now I got the RTCM msg... However, another problem occurred, I publish the rtcm ROS msg to feed my Ublox here: https://github.com/KumarRobotics/ublox/blob/master/ublox_gps/src/node.cpp and in this script, I added my RTCM rostopic here: https://github.com/KumarRobotics/ublox/blob/4f107f3b82135160a1aca3ef0689fd119199bbef/ublox_gps/src/node.cpp#L1908 However, I did not get the RTK solution, when I check my /ublox_gps/fix, the status was 0 which means I did not get RTK solution from RTCM, do you have any idea about that?

Another problem was that when I ran the Ntrip I saw the problem here: Do you also have any idea about that? Thx

As the error message says, the format of the fed NMEA sentence is wrong. Are you using the relay (so you get the NMEA from your receiver directly), or are you using my code to translate the ROS topic (GPSfix) message into an NMEA sentence?

Seems to be that I directly use my own NMEA maybe that is the problem, could you tell me how to modify to use my own NMEA message? Thx~

[yulan0215]

I have to cut out this section from my code, but you can see the arguments in the class definition: class NMEASubscriber: def __init__(self, msg_type, logerr=logging.error, logwarn=logging.warning, loginfo=logging.info, logdebug=logging.debug): # Bit of a strange pattern here, but save the log functions so we can be agnostic of ROS so the minimum argument is the msg_type, where at the moment the accepted types are: "Sentence": Sentence, "GPSFix": GPSFix, so a definition would be like this: self._subscriber = NMEASubscriber(msg_type="GPSFix")

Thanks for your help and right now I got the RTCM msg... However, another problem occurred, I publish the rtcm ROS msg to feed my Ublox here: https://github.com/KumarRobotics/ublox/blob/master/ublox_gps/src/node.cpp and in this script, I added my RTCM rostopic here: https://github.com/KumarRobotics/ublox/blob/4f107f3b82135160a1aca3ef0689fd119199bbef/ublox_gps/src/node.cpp#L1908 However, I did not get the RTK solution, when I check my /ublox_gps/fix, the status was 0 which means I did not get RTK solution from RTCM, do you have any idea about that?

Another problem was that when I ran the Ntrip I saw the problem here: Do you also have any idea about that? Thx

As the error message says, the format of the fed NMEA sentence is wrong. Are you using the relay (so you get the NMEA from your receiver directly), or are you using my code to translate the ROS topic (GPSfix) message into an NMEA sentence?

Seems to be that I directly use my own NMEA maybe that is the problem, could you tell me how to modify to use my own NMEA message? I used the nmea messgae from here: https://github.com/KumarRobotics/ublox/tree/master/ublox_msgs/msg

[Hadain]

I can't see which NMEA message you would use from the linked msgs, since there are ca. 40 ROS msg definitions, but none is a raw NMEA message.

You can write your own translator like this:

1) add your message keyword and handling function name in the _self._subs_msg_typesw dictionary:

self._subs_msg_type_sw = {
            "Sentence": (Sentence, self.subscribe_nmea),
            "GPSFix": (GPSFix, self.subscribe_gpsfix),
            "YourMessagekeyword": (ROSmsgType, self.subscribe_yourmsg)
        }

2) add the subscriber function:

    def subscribe_yourmsg(self, data):
        now = rospy.get_rostime()
        self.latest_sentence = self.your_translator_function(data, now)

3) then write the translator function, similar to the _gpsfix_togpgga(self, data, now) function:

def your_translator_function(self, data, now):

        # put together the string you need
        sentence = (  "proper NMEA message, following standards"
        )
        return self._calc_checksum(sentence) # This adds the checksum to the end of the sentence