Migrate Ruby gpsd to Python3

[km4ack]

This may be worth investigating.

see this post

[btombaugh]

i have followed the instruction in Kevin's document in the other post, and updated successfully

[btombaugh]

This leads to another question -- if that Python script replaces the two Ruby scripts grid and get_grid, which seemed to updated the file /run/user/1000/gridinfo.txt, do we need to disable or replace the process that was performing that update? i.e. is the Ruby script still running in the background? is gridinfo.txt used for anything other than updating the location in Conky?

[starbasessd]

I did not, not knowing that file was being used for anything. If it is, it's easy enough to add a line to update that file.

[starbasessd]

What data is in that file? Just gridsquare, or more?

[btombaugh]

yes, /run/user/1000/gridinfo.txt contains the grid square, to 8 char. Before I had gpsd working, or if the GPS has no 3D coordinate fix, it showed "NO GPS"

[starbasessd]

What else uses that file, do you know? @km4ack?

[km4ack]

The gridinfo.txt file is exclusively used by Conky to display the Grid Square in Conky.

[starbasessd]

Do you need it? Do you want any mods?

[btombaugh]

so is there a process that can be disabled or removed if we replace with the python script? I see that it's still being updated... Looks like:

Update GPS maidenhead for conky

/1 * /home/pi/bin/conky/grid in crontab

[starbasessd]

Yes, the .conkyrc file pulls the data from the .py script whenever it runs/updates, rather than force running the ruby grid and get_grid scripts, if you updated the .conkyrc line.

[btombaugh]

I've commented out the crontab entry, and placed the PyGridsquare.py script into /home/pi/bin/conky. I updated the .conkyrc to call /home/pi/bin/conky/PyGridsquare.py, so it should not have to be invoking the Ruby scripts any longer. Seems to work fine... Thanks for your responses!

[starbasessd]

Glad to be of help. This was my attempt to minimize / standardize and remove (for me) extraneous packages. I put the .py file in /usr/bin so I can call it manually from a command line, too, without remembering the path. (still need the full path when calling from within scripts or as root...)

[km4ack]

One word of caution. At one time we called the ruby script direct from Conky to get the gridsquare. This has the potential to cause issues in some cases. It would be better if the python script was run by cron and send the output (grid square) to a simple text file like the ruby script is doing now. BAP uses /run/user/$UID/filename for many of these type operations to reduce read/write cycles to the SD card.

[starbasessd]

In my humble ( or not ) opinion, running it every minute is major overkill, especially if you save the location/grid square to a file, too. If you are using a geo-tracker app while in motion, it shouldn't be reading this script anyway. My 6 digit grid square covers a lot of miles both north - south and east - west. If I covered either distance faster than 10 minutes I'd be breaking speed limits... As to read-write cycles, I think it's a toss up, but there would be a lot more write cycles to 'update' that txt file, which is what actually causes most of the damage (think 'write once, read many' for SDCards) than reading the script, figuring out what the grid square is, and updating the conky display overlay. IIRC the issue with calling ruby from conky was it stopped all updates while the ruby scripts were running pausing the screen, etc, it wasn't able to background it. The python script doesn't do it or at least not nearly as bad as ruby did.

[km4ack]

I can't disagree with you on running it every minute being overkill. The primary reason for this was to get it in conky as quick as possible after boot. A better approach might be braking it into two commands. One that runs at boot and another that updates it every X number of minutes.

As for the read/write cycles, the dir /run/user/$UID/ is held in RAM and not written to the SD card.

[starbasessd]

Can be the same script in crontab: @reboot \<path to>/PyGridsquare.py * * * * * \<path to>/PyGridsquare.py [will run every minute] \ \ \ \ \ Change the first to \5 or *\10 for 5 minutes or 10 minutes respectively I can give you the modded section in a little bit to write the gridsquare to that txt file.

[starbasessd]

PyGetGridsquare.py

---

!/usr/bin/env python3

from gpsdclient import GPSDClient import maidenhead as mh

client = GPSDClient(host="")

for result in client.dict_stream(convert_datetime=True): if result["class"] == "TPV": lat = result.get("lat", "n/a") lon = result.get("lon", "n/a") gridsquare = mh.to_maiden(lat, lon) print(gridsquare) file1 = open("/run/user/1000/gridinfo.txt","w") file1.write(str(gridsquare)) file1.close() break

---

This will save the gridsquare in the file /run/user/1000/gridinfo.txt

[starbasessd]

In the .conkrc file change ${font Arial:bold:size=18}${color White}Gridsquare ${alignr}${color Yellow}${execi 25 /PyGetGridsquare.py | cut -c1-8} to ${font Arial:bold:size=18}${color White}Gridsquare ${alignr}${color Yellow}${exec cat /run/user/1000/gridinfo.txt} save and if PyGetGridsquare.py created the data file, it should display it, instead of the direct output.

[btombaugh]

that seems to be working. I indented the Python code for the for loop and if statement. The filename for the crontab needs to be PyGetGridsquare.py, which I also had to chmod +x. It worked after a reboot and came up with the grid right away. I set it for every */5... Thanks again, I think this is a good solution.

[starbasessd]

Yeah, I hate markdown language (what github uses for this message posting...) it is such a pain to post code (or edit wiki pages when I support motionEye/motionEyeOS), sorry. I could post a modded text script like from the other site

[starbasessd]

Here's the instruction set from the other site modified with the above info: PyGetGridsquare.txt Naming error in line: sudo chmod +x //PyGridsquare.py needs to be: sudo chmod +x //PyGetGridsquare.py for consistency.

[btombaugh]

Kevin, the update instructions look good. Two comments:

If the Ruby grid scripts are replaced with the Python script, and the updates made in the crontab to call the Python script instead of the Ruby script, then I don't think that there needs to be any change made in .conkyrc, since it's already going to cat the same file. One should remove or comment the crontab entry for the Ruby script, so that it isn't running both.

Second, you should put your name and contact info in the instructions! Nicely done, sir!

[starbasessd]

No problem, writing version 1.0 of document, however, it was intended as a set of instructions for Jason to include into Build-A-Pi, and for a few to 'test' to see if it is worth it. My personal setup for field includes 3-5 Pis networked, so one would be a host/router/hub/Access Point, and any other machines, tablets, whatever could get GPS and Time (among other services), using a single GPS dongle. I also have a number of PiZeroWs acting as security cameras (with the motionEye/motionEyeOS that I support here on GitHub). If anyone feels the need to contact me, they can at KD9EFV at GMAIL dot COM. I am an Enterprise (Level 3) IT Help Desk, best known for acting as go between for end users and developer/programmers. Think: Brunette between 2 Blondes. I run into issues that need outside the box solutions.

[starbasessd]

Building a Pi Buster clean (updated to today) then will install Build-a-Pi to see what the current default installs, then modify my instructions for it, for use by 'current' users' to update/upgrade/install instead of...

[starbasessd]

@btombaugh @km4ack Would it be better to put the PyGetGridsquare.py call in the system crontab, or do you have a need for it in the user (Pi) crontab?

[btombaugh]

In my opinion, it would be better to put things in shared/common locations, rather than a user's home folder, in the event that anyone creates their own login instead of using "pi" as the only user... So I would vote for the system crontab.

However, since the conky scripts and config are all in the ~/pi directory, unless everything is moved, leaving it the user's directory is probably ok, too...

[starbasessd]

I wasn't referring to the file location (I was taught always use /usr/bin or /usr/sbin for custom apps \<sigh>) I was asking which crontab to use to call the program itself. There is a crontab for each user (crontab -e) and a system crontab (sudo crontab -e). If pi is logged in, both the pi user crontab and the system crontab are run. If johndoe is logged in, his crontab and system are run but not pi's. If no one is logged in, only system crontab is run.

[btombaugh]

right, sorry, I wasn't clear... I would think that best practice would be to put apps and other files into shared locations like /usr/bin, /usr/local, or /opt, depending on the flavor... If we did that, then using the system/root crontab would be better, as it removes any dependency on which user account is used. But because right now all of the files are already in the ~/pi directory, there is already a dependency that would preclude us from removing the user pi anyway, as everything would break...

Since we're working on a pi that fits in the palm of your hand, and not a large, multi-user system, there seems like less need to avoid putting things in the default user's home directory, and that user's crontab. In other words, I think it's safe to assume that 99.9% of users would be logging into their RPi as "pi" so it's fine to keep things as they are.

[starbasessd]

Interesting. In the field, I have multiple users all the time, using email, shared files and documents, KanBan, Whiteboards, etc. At home, I use Conky on all my desktop machines, whether Pi, PC, VM, whatever. (Also use neofetch on CLI machines and added a report for Gridsquare \<grin>). I never 'remove' the Pi user, anyway. Sometimes rename it... I guess it's harder for me to switch back and forth from a Debian type environment, and the RPiOS environment.

[starbasessd]

Latest instructions for end user implementation. v0.9.9 PyGetGridsquare.txt

[starbasessd]

@km4ack BTW, 3.1.3 and Bullseye: They moved vcgencmd (used for temperature monitoring in conky) from /opt/vc/bin to /usr/bin...

[km4ack]

Finally got around to testing and noticed that if a GPS isn't connected, the script will hang. Can you add a timeout and echo out "NO GPS" if a GPS isn't attached to the Pi?

[starbasessd]

Sure.

[starbasessd]

On my systems, whether acting as a host for the GPS, or showing the Gridsquare from another host, it doesn't 'hang', it just shows (blank) if there isn't any data from a GPS. Is this why you want it to show 'NO GPS'?

[km4ack]

I am working on something outside of BAP. This is the code I am using:

#!/usr/bin/env python3

from gpsdclient import GPSDClient
import maidenhead as mh

client = GPSDClient(host="localhost")

for result in client.dict_stream(convert_datetime=True):
    if result["class"] == "TPV":
        lat = result.get("lat", "n/a")
        lon = result.get("lon", "n/a")
        gridsquare = mh.to_maiden(lat, lon)
        print(gridsquare)
        break

When I run the script from the command line without a GPS connected to the pi, it will hang and doesn't return to prompt in the terminal. I assumed it would do the same thing if writing the grid to the text file.

[starbasessd]

OK, I see the problem, occurs in latest version, too, if run separately. Will advise change.

[starbasessd]

OK, Version 0.9.10 of instructions, including an updated PyGetGridsquare.py script. I wrote the previous version forgetting the full functions of the If loop and For loop in Python. I am also forcing "No GPS" if no local or network GPS Service found PyGetGridsquare.txt .

[km4ack]

That Works! Thanks!!

[km4ack]

I spoke too soon. Now when I have a GPS connected and it has a lock, I am getting "NO GPS" On a positive note, it doesn't hang.

[starbasessd]

Checking.

[starbasessd]

Building Pi3B+, Bullseye 10/30/2021, fully updated. Using install instructions 1/22/2022 for the PyGetGridsquare.py, will report shortly.

[starbasessd]

OK, running into a brick wall. I can check for /dev/ttyACM0 but not all USB GPSs connect there. I can check for gpsd, but my primary use case uses a network server, so it wouldn't find gpsd locally. Almost everything I can check for seems to be a 1-up kind of thing. If you want to drop me an email, can we discuss your use case?

[km4ack]

I like the python code and actually prefer it over ruby if it will function the same way with Conky. Currently the ruby script is reporting either the gridsquare or "no gps" to text file which is then displayed by conky. I would like to see the same behavior with the python script thus giving the operator some sort of indication if they either forgot to plug in the GPS or if the GPS doesn't have a lock.

[starbasessd]

One issue is this allows both local and network sources, where Ruby didn't. Thinking Hard.

[km4ack]

Yep. I like that it will work over the network. I am not opposed to a two script solution like I currently use with the ruby script. If you can just get it to break out of the script if there's no GPS data available. That way it wouldn't have to record "no gps". If it simply returns empty, I can work with it.

[km4ack]

This may be useless but there is a TPV object called "mode". Could you read it first to determine if it has a fix or not and exit out if no fix is found? Maybe exit if mode doesn't equal 3? reference - https://gpsd.gitlab.io/gpsd/gpsd_json.html

[starbasessd]

The logic keeps falling back to testing for the GPSd service whether local or network. I think I found a way, minor re-write I think. more later.

[starbasessd]

OK. Skull sweat. Version 0.9.11 with new (hopefully more robust version of PyGetGridsquare.py). Still writes gridsquare to /run/user/1000/gridinfo.txt and now "NO GPS" as appropriate. Not as clean as before, but should be able to follow it. Had to break up client line. PyGetGridsquare.txt

[starbasessd]

Don't ask for much, huh? LOL :) I'll see what I can do for NO FIX...

[starbasessd]

@km4ack You ask, you receive: PyGetGridsquare.txt Reports "NO GPS" if it doesn't see a gpsd service running at host Reports "No FIX" if TPV mode = 1 or 2 Reports gridsquare if it has a GPS service with a 3D fix. Let me know... Looking for ibuprofen for head ache... grin