search

opengisch / QField

A simplified touch optimized interface for QGIS
http://www.qfield.org
GNU General Public License v2.0
832 stars 227 forks source link

Use QField with `gpsd` on Linux #4981

Closed Jopie01 closed 9 months ago

Jopie01 commented 9 months ago

Describe the issue

I'm using Fedora 39 / Debian 12 system and have an U-blox ZED-F9P GPS receiver which is connected using USB. I'm using the gpsd daemon to get the GPS data from the receiver but also sending NTRIP data to the receiver for RTK corrections. The daemon also makes the (corrected) GPS data available on localhost and port 2947. Everything works correctly, I got a RTK FIX and I can see the data with cgps -s coming in. Also QGis displays the position and data as well.

However, QField on the other hand is connecting but won't display any position or data. I'm using qfield-v3.1.9-linux-x64.AppImage and don't know how to debug this issue. When I start QField from the terminal I see

"TcpReceiver: Initiating connection to address localhost (port 2947)"

and on the gpsd side:

gpsd:CLIENT: => client(0) len 123: {"class":"VERSION","release":"3.25.1~dev","rev":"3.25.1~dev-2024-01-23T11:05:50.460472","proto_major":3,"proto_minor":15}

then it al stops, no messages anymore from QField or gpsd sending data.

Looking at QGis, connection log on gpsd side:

gpsd:CLIENT: => client(0) len 123: {"class":"VERSION","release":"3.25.1~dev","rev":"3.25.1~dev-2024-01-23T11:05:50.460472","proto_major":3,"proto_minor":15}
gpsd:CLIENT: <= client(0): ?WATCH={"enable":true,"nmea":true,"raw":true};
gpsd:CLIENT: => client(0) len 647: {"class":"DEVICES","devices":[{"class":"DEVICE", ....},{"class":"DEVICE", ....}]}
{"class":"WATCH","enable":true,"json":false,"nmea":true,"raw":0,"scaled":false,"timing":false,"split24":false,"pps":false}
........

Then lots of NMEA messages are flowing over the screen and QGis displays the GPS location and data.

Reproduction steps

First get gpsd up and running. I followed https://gpsd.io/building.html to get the newest version. Test the connection with gpsmon or cgps and observe the GPS data. Then start QField and add a new external GPS receiver from localhost and port 2947.

Expected behavior

When QField connects to gpsd it should display the location and other GPS data.

Observed behavior

No GPS data or location is show. No warning or error

Desktop (please complete the following information)

It seems QField just expects a stream of data after the connection, but gpsd doesn't do that. As seen in the log of QGis, the 'client' should send a message to say "OK, start streaming!". QGis does that, QField not. The connection stays open because when I quit QField I get

gpsd:CLIENT: <= client(0): eof read

Which is exactly the same message when I quit QGis.

nirvn commented 9 months ago

QField hasn't implemented the gspd protocol (yet? :) ), labeling as enchantment request.

@Jopie01 , always good to hear about QField Linux user. Are you able to connect to the device through other ways?

Jopie01 commented 9 months ago

Thanks for the fast reply!

Are you able to connect to the device through other ways?

That's going to be hard, I wanted to use gpsd because it solves all the nasty problems when also using a NTRIP connection. Now I have to create several virtual serial devices, one to get the NTRIP data to the receiver and another to get the GPS data into QField.

I'm not a C++ programmer (but Python), but just for the record I found some resources:

I hope someone will someday come along and implement this.

Typing all the above, it came to my mind that I can create a simple Python script to stream the data. It will sit between gpsd and QField, does the protocol things and then just streams the GPS data which can be picked up by QField. For now that will be the way to go for me.

nirvn commented 9 months ago

@Jopie01 , give the test appimage here (https://github.com/opengisch/QField/actions/runs/7634783636?pr=4982) a try, I think that'll unlock gpsd for you in QField.

Out of curiosity, what are you mapping out with QField?

Jopie01 commented 9 months ago

give the test appimage here (https://github.com/opengisch/QField/actions/runs/7634783636?pr=4982) a try, I think that'll unlock gpsd for you in QField.

I gave it a try and got another message now about a readonly device.

"TcpReceiver: Initiating connection to address localhost (port 2947)"
QIODevice::write (QTcpSocket): ReadOnly device

Out of curiosity, what are you mapping out with QField?

For some years now I'm supporting the creation of a labyrinth in a corn field so the farmer can create some extra income. We are drawing the labyrinth first in QGis and export a KML file which can be read by AgOpenGPS. And because I have the GPS system I wanted now map my solar frame in the field. So import a top view (DXF) of the solar frame into QGis, place it in the correct position and then use QField for the field by walking to the right GPS position.

For all that I'm using a refurbished Surface Pro 3 tablet, also installed Debian on it and wanted to start using QField.

Jopie01 commented 9 months ago

Hmm, this is becoming a bit more weird. I just thought to use the NMEA test script from this repo https://github.com/opengisch/QField/blob/master/test/nmea_server/nmeaserver.py to see if feeding it https://github.com/opengisch/QField/blob/master/test/nmea_server/happy.txt will give some results. Unfortunately nothing is happening on the QField side. I started the 'server' which runs on port 1958 and QField is connecting and the 'server' starts streaming, but nothing happens in QField. No data is showing up when "Show position information" is checked. I also checked the "Log NMEA sentences from device to file" but the files stays empty.

Tested with both version 3.1.9 and the test appimage. Both the same result. Is this something specific for the appimage?

nirvn commented 9 months ago

@Jopie01 , try the appimage in this new test build (https://github.com/opengisch/QField/actions/runs/7638848064?pr=4982).

nirvn commented 9 months ago

@Jopie01 , should I take your emojis as a sign things work?

Jopie01 commented 9 months ago

should I take your emojis as a sign things work?

Yes, you did it! It works! Thanks a lot! But now I have to see where the discrepancy is coming from. I see a difference between the JSON and NMEA. The JSON gives me a more precise location and accuracy then the NMEA gives. It can be a gpsd thing.

nirvn commented 9 months ago

@Jopie01 , OK, glad to hear it's helped you out.

BTW, I find your usage of QField and your setup (Linux on a tablet) super interesting. Would you be interested in writing a success story that could be added here (https://docs.qfield.org/success-stories/). I feel like it could inspire others into create more labyrinths, or using QField :wink:

Jopie01 commented 9 months ago

I find your usage of QField and your setup (Linux on a tablet) super interesting.

At the moment it's just the idea. I have to try it out. My main concern is that I want to 'lock' onto a GPS point, walk to it and QField tells me how close I am and sounds a alarm when I reached it within centimeters.

The reason for Linux is (for me) very simple, That's my main OS and daily driver. Also Linux gives me much more possibilities then other closed systems. And of course freedom. The reason why I'm using a Surface Pro 3 is because AgOpenGPS is Windows only, so I needed for that a Windows tablet. Also it had to be more or less cheap so I shopped for a refurbished tablet which ultimately lead to the Surface Pro 3, which is basically a 2-in-1 laptop. And when Windows runs somewhere, Linux runs even better :smile:

Would you be interested in writing a success story that could be added here (https://docs.qfield.org/success-stories/).

I have to test my workflow because I don't think it's implemented in QField at the moment. lock onto a target, walk to it and give feedback when in very close distance. I can write a story about the labyrinth, but there is no QField involved, maybe this year.

nirvn commented 9 months ago

@Jopie01 , if you haven't done so yet, you should read this QField documentation page: https://docs.qfield.org/how-to/navigation/ -- I think it'll help you achieve your workflow.

Jopie01 commented 9 months ago

you should read this QField documentation page: https://docs.qfield.org/how-to/navigation/

That's exactly what I was looking for! Thanks for reminding me to RTM. Now I need to take my system and try it out.

Jopie01 commented 9 months ago

I don't know if this is something for a new issue. I've done some testing but walk into the problem that I'm missing some accuracy data. The fix stays on 'Fix3D' where I have a full RTK fix. I don't know if this is a big problem, but I'm missing the H. Accuracy and V. Accuracy and without them, the "stakeout" precise view isn't working.

So the question is, which NMEA sentences is QField using?

nirvn commented 9 months ago

@Jopie01 , if you want to get a good understand on how QField (and QGIS) handles NMEA information, check this source code: https://github.com/qgis/QGIS/blob/master/src/core/gps/qgsnmeaconnection.cpp#L229-L246

I've highlighted the horizontal/vertical accuracy fetching in the link above. They are taken from NMEA GST sentences.

Jopie01 commented 9 months ago

Thanks for the link! Will take a deeper look at it.

the horizontal/vertical accuracy fetching in the link above. They are taken from NMEA GST sentences.

This tells it all because gpsd didn't send any GST sentence. I need to do a bit more research if and when gpsd is sending the different sentences.

nirvn commented 9 months ago

@Jopie01 , do let us know if you figure out how to have gpsd stream accuracy values.

Jopie01 commented 9 months ago

let us know if you figure out how to have gpsd stream accuracy values.

Location of gpsd is https://gitlab.com/gpsd/gpsd and I created an issue there as well to ask for more NMEA sentences https://gitlab.com/gpsd/gpsd/-/issues/272. Their NMEA creation is located at https://gitlab.com/gpsd/gpsd/-/blob/master/gpsd/pseudonmea.c but I don't know when the different functions are called. I also don't understand exactly how and when send the NMEA sentences.

They have https://gpsd.gitlab.io/gpsd/NMEA.html where they explain the different NMEA messages.

Jopie01 commented 9 months ago

To make this fully complete: My U-blox receiver outputs the compact binary UBX data which gpsd is converting into NMEA messages. However, the UBX data doesn't have anything to generate the GST sentence from. So I have to reconfigure the U-blox to output NMEA which is then passed through by gpsd.

An enhancement would be to be able to fallback to CEP when the GST sentence is not available. I've just came across an article and an update on that article about accuracy in the GPS world. https://www.gpsworld.com/gps-accuracy-lies-damn-lies-and-statistics/

https://web.archive.org/web/20150520062144/https://www.gpsworld.com/gpsgnss-accuracy-lies-damn-lies-and-statistics-1134/ (unfortunately you have to use webarchive otherwise no images are shown)

m-kuhn commented 9 months ago

@Jopie01 this in the end concerns more the QGIS NMEA parser than QField, feel free to open a feature request https://github.com/qgis/QGIS/issues

bettellam commented 3 months ago

@Jopie01

I'm not a C++ programmer (but Python), ....

You can get some inspiration from here: https://github.com/semuconsulting/PyGPSClient