semuconsulting / pygnssutils

Python GNSS CLI utility library for reading, parsing and broadcasting NMEA, UBX, RTCM3, NTRIP and SPARTN protocols
BSD 3-Clause "New" or "Revised" License
82 stars 23 forks source link

Can't connect to NTRIP server #80

Closed akloeker closed 6 days ago

akloeker commented 1 month ago

I'm using a ZED Box (basically an Nvidia Orin NX with builtin ublox ZED-F9P GNSS-RTK receiver) and am currently trying to use this script to get an RTK fix.

When using gpsd with this command sudo gpsd -nG ntrip://<user>:<password>:@caster.axio-net.eu:2101/07-AXIO -s 115200 /dev/ttyACM0 -N -D 4 the NTRIP service works just fine. I.e. gpsd outputs

gpsd:INFO: PPS:ntrip://<user>:<password>: ntpshm_link_activate: 1
gpsd:INFO: device ntrip://<user>:<password>: activated

However using the provided script, I always end up with the following error: CRITICAL - pygnssutils.gnssntripclient - Error connecting to caster.axio-net.eu:2101/mountpoint='07-AXIO': HTTP/1.0 401 Unauthorized I made more than 100% sure that the user credentials, server address, port, etc. are correct. There is definitely no problem on this side.

The next step was to try the gnssntripclient with the following command: gnssntripclient -S caster.axio-net.eu -P 2101 -M 07-AXIO --ntripuser <user> --ntrippassword <password> --verbosity 3

Interestingly I get another error:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/pyubx_test/venv/lib/python3.8/site-packages/pygnssutils/gnssntripclient.py", line 493, in _read_thread
    self._do_connection(settings, stopevent, output)
  File "/home/user/pyubx_test/venv/lib/python3.8/site-packages/pygnssutils/gnssntripclient.py", line 579, in _do_connection
    rc = self._do_header(self._socket, stopevent, output)
  File "/home/user/pyubx_test/venv/lib/python3.8/site-packages/pygnssutils/gnssntripclient.py", line 620, in _do_header
    data = sock.recv(DEFAULT_BUFSIZE)
socket.timeout: timed out

The same error also appears when using the pygpsclient.

semuadmin commented 1 month ago

Hi @akloeker ,

Sorry you're having problems. Couple of observations off the top of my head:

  1. The Stackoverflow link may be a bit stale now. There is a more recent version of the rtk_example.py script here - use this one instead.

  2. Are you running gpsd and the Python utilities on the same platform? If so, they can sometimes conflict. Ensure you have completely shut down gpsd and any other daemon processes which may be trying to use the same serial or TCP ports before running any Python scripts. This is a common issue.

  3. Maybe try with an rtk2go.com mountpoint (e.g. 'MunichBaseO') using your email address and a nominal password, just to see if it's a local issue.

gnssntripclient -S rtk2go.com -P 2101 -M MunichBaseO --ntripuser me@myaddress.com --ntrippassword password --verbosity 3
2024-08-02 18:38:40.549 - INFO - pygnssutils.gnssntripclient - Streaming RTCM data from rtk2go.com:2101/MunichBaseO ...
2024-08-02 18:38:41.751 - INFO - pygnssutils.gnssntripclient - RTCMMessage received: 1004
2024-08-02 18:38:41.751 - DEBUG - pygnssutils.gnssntripclient - <RTCM(1004, DF002=1004, DF003=0, etc...

Other than this, then assuming caster.axio-net is a standard NTRIP 2.0 caster, I can't think of any obvious reason why the gnssntripclient utility wouldn't work - it's been tested successfully with many other casters.

I'll endeavour to register for an account with Axio / Trimble Pivot Web to test it myself - I'm awaiting registration confirmation now and will respond further as and when I can get access.

akloeker commented 1 month ago

Thank you very much for the immediate reply!

  1. I upgraded to the newest version of the script but as expected it didn't change much.
  2. gpsd is completely shutdown. If not, this results in a permission error on the /dev/ttyACM0 device,
  3. rtk2go.com works without problems. Using this provider I get an RTK fix (albeit with low accuracy).

So the problem really seems to be provider specific. I'm looking forward to your further replies when you get access to Axio!

semuadmin commented 1 month ago
3. rtk2go.com works without problems. Using this provider I get an RTK fix (albeit with low accuracy).

So the problem really seems to be provider specific. I'm looking forward to your further replies when you get access to Axio!

Still waiting for a response from Axio, but some further observations...

Taking a further look at the sourcetable entry for the mountpoint you're attempting to access - caster.axio-net.eu:2101/07-AXIO - I note that:

  1. It's using a relatively old version of the RTCM protocol - 3.1.
  2. The 'nmea' field is set to '1' i.e. "Client must send NMEA GGA message with approximate position to Caster" - are you doing this (i.e. do you have ggainterval and ggamode set appropriately - type gnssntripclient -h for help)?
  3. The 'fee' field is set to'Y', implying that a fee is required for the service (this may be spurious, especially if it appears to work OK with gpsd).
{'mountpoint': '07-AXIO', 'id': 'AXIO-PED', 'format': 'RTCM 3.1', 'format_details': '1004(1), 1005(10), 1008(10), 1012(1), 1029(61), 1030(10), 1031(10), 1032(30), 1033(30)', 'carrier': '2', 'nav_system': 'GPS+GLO', 'network': 'AXIO-NET', 'country': 'DEU', 'lat': '48', 'lon': '11', 'nmea': '1', 'solution': '1', 'generator': 'AXIO-NET', 'compr_encryp': 'none', 'authentication': 'B', 'fee': 'Y', 'bitrate': '0'}

Compare this with the rtk2go.com/MunichBase0 sourcetable entry:

{'mountpoint': 'MunichBaseO', 'id': 'Munchen', 'format': 'RTCM 3.3', 'format_details': '1004(1),1005(10),1008(10),1012(1),1019(4),1020(2),1033(10),1042(4),1046(1),1077(1),1087(1),1097(1),1107(1),1127(1),1230(30)', 'carrier': '2', 'nav_system': 'GPS+GLO+GAL+BDS+SBS', 'network': 'SNIP', 'country': 'DEU', 'lat': '48.13', 'lon': '11.67', 'nmea': '1', 'solution': '0', 'generator': 'sNTRIP', 'compr_encryp': 'none', 'authentication': 'B', 'fee': 'N', 'bitrate': '10380'}
akloeker commented 1 month ago

I can report a few partial successes.

The 'nmea' field is set to '1' i.e. "Client must send NMEA GGA message with approximate position to Caster" - are you doing this (i.e. do you have ggainterval and ggamode set appropriately - type gnssntripclient -h for help)?

I think this is the key why it failed so far. Setting ggamode and ggainterval to 1 and setting a suitable reference positions leads to the client sometimes getting NTRIP data. In all other cases, it just timeouts as before. I tried a lot today to figure out if there is some deterministic behavior behind the time outs but so far I couldn't find any clue when exactly a timeout happens. However seems that the chance for a successful connection improves the more accurate my reference position and separation relative to my actual position and separation. But even with a very accurate reference position the client crashes like every second time.

Using ggamode = 0 (live position) never works, even when using the rtk_example.py script and having a normal GNSS fix.

It's using a relatively old version of the RTCM protocol - 3.1.

There is also another mountpoint AX-PED which provides RTCM 3.2, however it operates on port 2102 without HTTPS. In the rtx_example.py script it just works as the other mountpoint, however for the gnssntripclient it always tries to connect with HTTPS, which then fails. Even setting -H or --https to 0 has no effect. It seems that this flag is ignored.

That are my results from today, maybe I have some time tomorrow to run some more experiments.

semuadmin commented 1 month ago

Thanks for the additional info.

The 'nmea' field is set to '1' i.e. "Client must send NMEA GGA message with approximate position to Caster" - are you doing this (i.e. do you have ggainterval and ggamode set appropriately - type gnssntripclient -h for help)?

I think this is the key why it failed so far. Setting ggamode and ggainterval to 1 and setting a suitable reference positions leads to the client sometimes getting NTRIP data. In all other cases, it just timeouts as before.

That's consistent with my expectation. Some casters (like rtk2go.com) are fairly relaxed about receiving NMEA GGA data even when the 'gga' sourcetable flag is set to '1', but for most it's a mandatory requirement when this flag is set.

Using ggamode = 0 (live position) never works, even when using the rtk_example.py script and having a normal GNSS fix.

Go figure??? Some NTRIP casters have geofenced services - that is, they will only respond if the reported position is within their designated geographical catchment area, though I've no idea if this is the issue here.

There is also another mountpoint AX-PED which provides RTCM 3.2, however it operates on port 2102 without HTTPS. In the rtx_example.py script it just works as the other mountpoint, however fpr the gnssntripclient it always tries to connect wie HTTPS, which then fails. Even setting -H or --https to 0 has no effect. It seems that this flag is ignored.

Yeah, my bad - it's set up this way because u-blox/Thingstream uses 2102 (rather than 443) as its HTTPS port and I had assumed this was probably true for other casters - but evidently it's not! 2101 is the NTRIP 2.0 standard so using 2102 is arguably non-compliant, but it's easy enough to handle - there's already a fix in for this in RC 1.0.32.

akloeker commented 1 month ago

Some NTRIP casters have geofenced services - that is, they will only respond if the reported position is within their designated geographical catchment area, though I've no idea if this is the issue here.

I don't think that this is the problem here. When I use the exact same values from the live position as fixed reference, it sometimes works. Does the REFSEP value gets set when using live position? I found out, that this value also has to be correct to make the caster work.

Apart from that, sadly I couldn't generate any more meaningful insights.

semuadmin commented 1 month ago

Does the REFSEP value gets set when using live position? I found out, that this value also has to be correct to make the caster work.

Just to clarify, when running gnssntripclient as a CLI utility (as opposed to invoking the GNSSNTRIPClient class in a Python script), the only mode available is GGAFIXED i.e. fixed reference coordinates. GGLIVE requires a calling application to implement a get_coordinates() method which returns (lat, lon, alt, sep) - when running as a CLI there is no calling application.

I'll make this clearer in the documentation.

Are you getting consistent results with GGAFIXED, or are there still intermittent failures?

akloeker commented 1 month ago

Just to clarify, when running gnssntripclient as a CLI utility (as opposed to invoking the GNSSNTRIPClient class in a Python script), the only mode available is GGAFIXED i.e. fixed reference coordinates. GGLIVE requires a calling application to implement a get_coordinates() method which returns (lat, lon, alt, sep) - when running as a CLI there is no calling application.

I tried it with the rtk_example.py script but without success.

Are you getting consistent results with GGAFIXED, or are there still intermittent failures?

There are still intermittent failures, I haven't found a 100% reliable setup yet.

semuadmin commented 1 month ago

There are still intermittent failures, I haven't found a 100% reliable setup yet.

OK have you tried with a proprietary NTRIP client like BNC?

Ok FYI I'm still awaiting a response from Axio/Trimble re NTRIP Caster login. This seems to involve convoluted procedure requiring reqistration for both a Trimble ID, a Trimble Pivot Web account and an AXIO-NET portal account - are you aware of any short-cuts to access?

akloeker commented 1 month ago

OK have you tried with a proprietary NTRIP client like BNC?

Yes with several proprietary RTK GNSS INS solutions we never had any problems.

Ok FYI I'm still awaiting a response from Axio/Trimble re NTRIP Caster login. This seems to involve convoluted procedure requiring reqistration for both a Trimble ID, a Trimble Pivot Web account and an AXIO-NET portal account - are you aware of any short-cuts to access?

Sadly no. I wasn't involved in the registration process and it's already been a few years since we got access.

semuadmin commented 1 month ago

FYI there have been a few enhancements to gnssntripclient in v1.1.0 (including NTRIP 2.0 GGA processing) - no idea if they'll have any bearing on this issue, but may be worth another try with this new version.

python3 -m pip install --upgrade pygnssutils

I'm still awaiting a login for the Axio caster and am currently unable to reproduce your issue in any other caster.