Unable to receive RTCM Corrections using rtk_examples.py

[acwu11]

Describe the bug

Unable to receive RTCM corrections after following the rtk_examples.py on ZED_F9K receiver at baud rate of 38400. We replaced default fields with personal login for NTRIP client but cannot tell if 1) the NTRIP connection was unsuccessful or 2) the connection was successful but there is a problem in receiving RTCM corrections.

From the 'NAV-SAT' message read from the receiver, the rtcmCorrUsed_xx field is always 0, but no additional error is thrown.

Other information:

• pygnssutils version -- latest as of Oct. 14. 2022

• Sample output (after disabling the print messages from the "read thread")

• Expected Behavior * We expect to see a print field starting with "NTRIP >> ..." in the terminal when RTCM correction messages are received.

  • Desktop *

• Operating System: Linux

• Type of Connection: USB

GNSS/GPS Device:

• Device Model/Generation: u-blox ZED-F9K - 00B - 00
• Firmware Version: N/A
• Protocol: N/A.

[semuadmin]

Hi @acwu11

Sorry you're having difficulties. I would be inclined to start with a basic gnssntripclient command line example first, just to establish if you're able to connect to the caster and receive RTCM data. Try the example here: https://github.com/semuconsulting/pygnssutils#gnssntripclient.

First establish if you can download the caster's sourcetable (this is normally available without authentication):

gnssntripclient server=rtgpsout.unavco.org reflat=55.3 reflon=-160.24 ggamode=1 verbosity=2

where 55.3 and 160.24 are your fixed ARP reference coordinates (substitute as appropriate). You should see something like this in response:

2022-10-15 08:48:43.826172: Closest mountpoint to reference location (55.3, -160.24) = AB07_RTCM3, 16.17 km
2022-10-15 08:48:43.826265: Complete sourcetable follows...
['AB07_RTCM3', 'AB07_RTCM3', 'RTCM 3.1', '1004(1),1005(60),1007(60),1012(1),1033(60)', '2', 'GPS', 'UNAVCO', 'USA', '55.35', '-160.48', '0', '0', 'TRIMBLE NETRS', 'None', 'B', 'N', '0', '', '']
etc.

If the response times out, you could try increasing the waittime parameter to, say, 10 seconds (it's 3 seconds by default).

If this works, pick a suitable mountpoint MPT (e.g. _AB07RTCM3) and login credentials USERID/PASSWD and see if the caster returns valid RTCM data:

gnssntripclient server=rtgpsout.unavco.org mountpoint=AB07_RTCM3 user=USERID password=PASSWD reflat=55.3 reflon=-160.24 ggamode=1 ggainterval=60 verbosity=2

If authentication fails, you should see a corresponding HTTP 401 Unauthorized error. If all's well, you should see a stream of RTCM messages at the terminal.

Try this first and let me know how you get on.

[acwu11]

Hi @semuadmin Thanks! I tried these steps and was able to download the sourcetable. However, there was an authentication failure in the second step with terminal output : HTTP/1.1 403 Forbidden.

Does this indicate that it is a problem with the login? Would you have any suggested solution methods? For reference, we use the same credentials through u-center to establish the NTRIP Client.

[semuadmin]

Hi @semuadmin Thanks! I tried these steps and was able to download the sourcetable. However, there was an authentication failure in the second step with terminal output : HTTP/1.1 403 Forbidden.

Does this indicate that it is a problem with the login? Would you have any suggested solution methods? For reference, we use the same credentials through u-center to establish the NTRIP Client.

Hi @acwu11

Yes, an HTTP 403 error indicates an issue of some kind with authorisation (not necessarily with the login credentials). I would ordinarily expect an HTTP 401 if incorrect login credentials have been applied. An HTTP 403 may indicate that the caster is applying some kind of block on the specific client IP address or application identifier.

gnssntripclient has been tested using casters which require basic authentication (userid and password) and it should work ok. I'm afraid I can't account for why it doesn't work in your case. I have applied for my own access credentials to the UNAVCO caster in order to test it myself, and am awaiting their response.

I take it the same caster, mountpoint and login credentials work fine in the u-center NTRIP client?

[acwu11]

@semuadmin

Thanks in advance for checking with UNAVCO on your end. Just reverified and yes, the same caster, mountpoint, and login credentials work in u-center NTRIP client.

[semuadmin]

@semuadmin

Thanks in advance for checking with UNAVCO on your end. Just reverified and yes, the same caster, mountpoint, and login credentials work in u-center NTRIP client.

@acwu11 I have now received and tested UNAVCO login credentials but I am getting the same error as you - an HTTP/1.1 403 Forbidden message. I have raised a query with UNAVCO but my best guess is that either a) their caster is blocking unrecognised NTRIP clients such as 'gnssntripclient', or b) my implementation of the NTRIP 2 protocol is somehow faulty (though, as I say, it works fine with many other authenticated casters). I'll continue to investigate.

[dborden0]

In the same example I am running into an error connecting to a local ntrip server that I have an account for. I get this error using the command line to connect:

D:\git\gps_rtk>poetry run gnssntripclient server=167.131.109.57 port=9882 user=user password=pwd mountpoint=IMAX_GG_RTCM3_AG waittime=20 verbosity=2 ggamode=1 ggainterval=60 reflat=42.38 reflon=-122.89 connected data b'HTTP/1.1 200 OK\r\nNtrip-Version: Ntrip/2.0\r\nCache-Control: no-store, no-cache, max-age=0\r\nPragma: no-cache\r\nServer: NTRIP Spider/7.8.0.9445\r\nDate: Wed, 19 Oct 2022 04:08:04 GMT\r\nConnection: close\r\nContent-Type: gnss/data\r\n\r\n' Exception in thread Thread-1: Traceback (most recent call last): File "c:\users\dbord\appdata\local\programs\python\python38-32\lib\threading.py", line 932, in _bootstrap_inner self.run() File "c:\users\dbord\appdata\local\programs\python\python38-32\lib\threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "c:\users\dbord\appdata\local\pypoetry\cache\virtualenvs\gps-rtk-o6opi8md-py3.8\lib\site-packages\pygnssutils\gnssntripclient.py", line 464, in _read_thread rc = self._do_header(self._socket, stopevent) File "c:\users\dbord\appdata\local\pypoetry\cache\virtualenvs\gps-rtk-o6opi8md-py3.8\lib\site-packages\pygnssutils\gnssntripclient.py", line 508, in _do_header data = sock.recv(DEFAULT_BUFSIZE) socket.timeout: timed out

[dborden0]

Also, if I use the PyGPSClient with the same parameters, I receive data from the NTRIP server with no issues, which I believe uses the same NTRIP Client?

[dborden0]

I believe the issue is that the NTRIP server I am using only provides data if you provide your lat/lon from the receiver to the server. PyGPSClient does this, but the sample application doesn't because it passes in None for the App to the GNSSNTRIPClient class.

[semuadmin]

Hi @dborden0

Thanks for the information. You are correct in saying that many NTRIP casters will only provide NTRIP data if valid client coordinates are provided via an NMEA GGA sentence, but I'm not sure this wholly explains the issue originally reported here.

For example, I'm still getting an HTTP/1.1 403 Forbidden error from the UNAVCO caster regardless of whether I send a GGA sentence (with fixed or live coordinates), either from PYGPSClient or from the gnssntripclient CLI - so something else appears to be blocking the client in this instance. I've double-checked my code and as far as I can tell it is fully compliant with the NTRIP 2.0 protocol standard, but I may have overlooked something.

I'll continue to investigate, but at the end of the day only the NTRIP service providers themselves can fully explain the reasons for their response codes. They may, for example, block certain NTRIP User-Agent designators, or restrict responses to those clients within a specific geofence.

I should perhaps emphasise that rtk_example.py is only intended to be an illustration - not a fully mature NTRIP client.

By way of background, the code in gnssntripclient.py was originally part of the core PyGPSClient GUI application, but I decided to refactor it as a separate package for the benefit of those who may want to use its functionality outside a tkinter GUI application. This is why it accepts an optional calling 'app' parameter, but the calling 'app' doesn't necessarily have to be a tkinter application - it could be any other user-written Python class which provides the same methods/attributes (i.e. appmaster, update_ntrip_status, get_coordinates).

[wdimmit]

The UNAVCO caster seems to require Connection: Close\r\n be appended to the end of the GET request. See line 306 in gnssntripclient.py. I'm not sure if this breaks compatibility with other casters, as I've only tested with UNAVCO.

[semuadmin]

OK so finally got round to doing what I should have done in the first place - put a Wireshark intercept on the HTTP GET command from u-center's NTRIP client and compare it to mine:

Turns out the UNAVCO caster was objecting to my User-Agent string for some reason. I've amended the string and it all appears to work fine now. Sorry it's taken a while to resolve this.