search

semuconsulting / pygnssutils

Python GNSS CLI utility library for reading, parsing and broadcasting NMEA, UBX, RTCM3, NTRIP and SPARTN protocols
BSD 3-Clause "New" or "Revised" License
84 stars 25 forks source link

gnssserver: TypeError with string output format #31

Closed IanBurwell closed 1 year ago

IanBurwell commented 1 year ago

Describe the bug

I am trying to use the gnssserver as an NTRIP caster so that I can get RTCM corrections into ROS (using ntrip_client). In doing so, I realized that I needed the output format to be a string of sorts and thus am using --format 16 (or --format 1). However, this causes an error (below) as the string seems to not be encoded.

pygnssutils version: 1.0.9

gnssserver output:

$ export PYGPSCLIENT_USER=user
$ export PYGPSCLIENT_PASSWORD=password
$ gnssserver --inport "/dev/ttyACM0" --baudrate 9600 --hostip localhost --outport 2101 --ntripmode 1 --protfilter 4 --format 16
2023-05-23 15:24:22.375871: Starting server (type CTRL-C to stop)...
2023-05-23 15:24:22.375897: Starting input thread, reading from /dev/ttyACM0...
2023-05-23 15:24:22.376497: Parsing GNSS data stream from: Serial<id=0x7f6a2c929fd0, open=True>(port='/dev/ttyACM0', baudrate=9600, bytesize=8, parity='N', stopbits=1, timeout=3, xonxoff=False, rtscts=False, dsrdtr=False)...

2023-05-23 15:24:22.876833: Starting output thread, broadcasting on localhost:2101...
2023-05-23 15:24:26.969079: Client ('127.0.0.1', 41154) has connected. Total clients: 1
2023-05-23 15:24:27.019219: Client ('127.0.0.1', 41154) has disconnected. Total clients: 0
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 41154)
Traceback (most recent call last):
  File "/usr/lib/python3.8/socketserver.py", line 683, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.8/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/home/rover/.local/lib/python3.8/site-packages/pygnssutils/socket_server.py", line 221, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/lib/python3.8/socketserver.py", line 747, in __init__
    self.handle()
  File "/home/rover/.local/lib/python3.8/site-packages/pygnssutils/socket_server.py", line 282, in handle
    self._write_from_mq()
  File "/home/rover/.local/lib/python3.8/site-packages/pygnssutils/socket_server.py", line 406, in _write_from_mq
    self.wfile.write(raw)
  File "/usr/lib/python3.8/socketserver.py", line 826, in write
    self._sock.sendall(b)
TypeError: a bytes-like object is required, not 'str'
----------------------------------------
^C2023-05-23 15:24:31.491631: Stopping server...
2023-05-23 15:24:31.492141: Streaming terminated, 9 messages processed with 0 errors.

2023-05-23 15:24:31.976689: Server shutdown.

If using --format 1 the error becomes TypeError: a bytes-like object is required, not 'RTCMMessage'

ntrip_ros output (if helpful):

$ roslaunch ntrip_client ntrip_client.launch host:=localhost mountpoint:=pygnssutils authenticate:=true username:=user password:=password ntrip_version:=2 debug:=true
... logging to /home/rover/.ros/log/6e28188c-f99f-11ed-8545-f58b20dd1639/roslaunch-base-station-102233.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
WARNING: disk usage in log directory [/home/rover/.ros/log] is over 1GB.
It's recommended that you use the 'rosclean' command.

started roslaunch server http://base-station:38795/

SUMMARY
========

PARAMETERS
 * /ntrip_client/ntrip_client_node/authenticate: True
 * /ntrip_client/ntrip_client_node/ca_cert: 
 * /ntrip_client/ntrip_client_node/cert: 
 * /ntrip_client/ntrip_client_node/host: localhost
 * /ntrip_client/ntrip_client_node/key: 
 * /ntrip_client/ntrip_client_node/mountpoint: pygnssutils
 * /ntrip_client/ntrip_client_node/ntrip_version: 2
 * /ntrip_client/ntrip_client_node/password: password
 * /ntrip_client/ntrip_client_node/port: 2101
 * /ntrip_client/ntrip_client_node/reconnect_attempt_max: 10
 * /ntrip_client/ntrip_client_node/reconnect_attempt_wait_seconds: 5
 * /ntrip_client/ntrip_client_node/rtcm_frame_id: odom
 * /ntrip_client/ntrip_client_node/rtcm_timeout_seconds: 4
 * /ntrip_client/ntrip_client_node/ssl: False
 * /ntrip_client/ntrip_client_node/username: user
 * /rosdistro: noetic
 * /rosversion: 1.16.0

NODES
  /ntrip_client/
    ntrip_client_node (ntrip_client/ntrip_ros.py)

auto-starting new master
process[master]: started with pid [102252]
ROS_MASTER_URI=http://localhost:11311/

setting /run_id to 6e28188c-f99f-11ed-8545-f58b20dd1639
process[rosout-1]: started with pid [102273]
started core service [/rosout]
process[ntrip_client/ntrip_client_node-2]: started with pid [102280]
[DEBUG] [1684869866.922012]: init_node, name[/ntrip_client/ntrip_client_node], pid[102280]
[DEBUG] [1684869866.924688]: binding to 0.0.0.0 0
[DEBUG] [1684869866.927172]: bound to 0.0.0.0 44277
[DEBUG] [1684869866.929877]: ... service URL is rosrpc://base-station:44277
[DEBUG] [1684869866.932152]: [/ntrip_client/ntrip_client_node/get_loggers]: new Service instance
[DEBUG] [1684869866.936572]: ... service URL is rosrpc://base-station:44277
[DEBUG] [1684869866.938897]: [/ntrip_client/ntrip_client_node/set_logger_level]: new Service instance
[ERROR] [1684869867.021289]: Invalid response received from http://localhost:2101/pygnssutils
[ERROR] [1684869867.024328]: Response: 
[ERROR] [1684869867.027056]: Unable to connect to NTRIP server
[INFO] [1684869867.029923]: Stopping RTCM publisher
[INFO] [1684869867.032297]: Disconnecting NTRIP client
[ntrip_client/ntrip_client_node-2] process has died [pid 102280, exit code 1, cmd /opt/ros/noetic/lib/ntrip_client/ntrip_ros.py __name:=ntrip_client_node __log:=/home/rover/.ros/log/6e28188c-f99f-11ed-8545-f58b20dd1639/ntrip_client-ntrip_client_node-2.log].
log file: /home/rover/.ros/log/6e28188c-f99f-11ed-8545-f58b20dd1639/ntrip_client-ntrip_client_node-2*.log
^C[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

To Reproduce

Steps to reproduce the behaviour:

  1. Using an ArduSimple simpleRTK2B base station with a few RTCM messages enabled via USB
  2. ROS noetic and ntrip_client as a client
  3. Run the above commands

Expected Behaviour

Either a graceful failure or no error.

Desktop (please complete the following information):

GNSS/GPS Device (please complete the following information as best you can):

image

Additional context

My one goal is to provide the ublox_gps ROS driver with RTCM messages from a local base station and the best way I've found is via an NTRIP caster (such as gnssserver) and the ntrip_client package. I would use pyGPSclient but it sends binary data only (to my knowledge) which ntrip_client won't accept. Thus any advice along with discussing the mentioned bug would be warmly welcome.

semuadmin commented 1 year ago

Hi @IanBurwell,

I'm not in a position to provide support or guidance on the particular ntrip client or ublox_gps libraries you refer to, but RTCM is a binary protocol and any compliant NTRIP client (including RTK-compatible GNSS devices such as the F9* series) would expect RTCM data in binary format, not string format.

To use gnssserver as an NTRIP caster, you need to be using binary output format (--format 2), which is the default anyway, so you can simply omit the --format argument.

FYI the pygnssutils library also provides an ntrip client - gnssntripclient - which does accept binary input. You may like to take a look at this.

I may be misunderstanding your objectives here, but you appear to be using an unnecessarily complicated toolchain if you're simply trying to upload RTCM data from an F9P base station to another ublox GNSS device. You don't need the ntrip-client or ublox-gps libraries - you could simply send the binary RTCM output from gnssserver directly to the receiver - see for example https://github.com/semuconsulting/pygnssutils/blob/main/examples/rtk_example.py.

HTH

IanBurwell commented 1 year ago

Ah, my misunderstanding came from not knowing that NTRIP was solely binary and that the library I am using must either not be in compliance or something else entirely. I thus understand why --format caused an issue. It could be nice to catch that user error but I understand what went wrong and am happy to close this issue. The reason I have a complex chain is due to the rover GPS driver hogging the port and only accepting RTCM via ROS messages (so the chain is just a clunky way of converting RTCM-->ROS).

Thanks, -Ian

semuadmin commented 1 year ago

The user error is there for a reason, but I've updated the README to make things clearer.

I had a quick look at the documentation for ntrip_client and I'm pretty sure it's designed to accept binary RTCM input from an NTRIP caster. I can see no reason why it shouldn't work with gnssserver running in NTRIP mode with the default binary output format - have you actually tried it in this configuration?

NB: as explained in the README, you need to ensure your F9P base device is operating in Base Station Mode (either FIXED or SURVEY_IN) - gnssserver itself doesn't set this up automatically (as different users will have different configuration requirements), but code examples are available e.g. https://github.com/semuconsulting/PyGPSClient/blob/master/examples/f9p_basestation.py

ROS is open source so if I can find the time, I may look into supporting a ROS output format directly in pygnssutils in a future release.

Are you happy to close this issue now?

IanBurwell commented 1 year ago

My confusion with strings was due to a string decoding error without stack trace and a gross assumption. I will look further into it to get to the root of my problem.