twillis449 / ALBUS_ionosphere

software to determine ionosphere TEC and RM from GPS receiver data
Other
18 stars 9 forks source link

Undo py3.6 breaking change. Add requests library to installation #21

Closed bennahugo closed 1 year ago

bennahugo commented 1 year ago

Fixes #20

bennahugo commented 1 year ago

Alright tests pass again. @twillis449 perhaps if possible push things through the CI server via pull requests so we can catch obvious breakage (more tests are needed --- with output report testing --- I will get to that once the writeup is done)

twillis449 commented 1 year ago

This is weird. In line 214 of GPS_stations .py you had:

 print('input file is ', filename)
# dunnon what encoding was used on this file but it is not UTF 8.
# strip whatever is not ASCII further down
fp = open(filename, "r")
counter = 0
try:
    site_id_found = 0
    for line in fp:

line = ''.join(map(chr, map(lambda x: x if x < 127 else ord(' '), line)))

blah blah BUT I noticed that originally this file was still being opened with the infamous 'rb' flag ie. fp = open(filename, "rb") which was causing your failure with python 3.6. on other files but I guess not ...

I wonder if the first line of the gps_pos_default.snx file which is %=SNX 2.01 IGS 13:094:19351 IGS 00:000:00000 00:000:00000 P 00000 0 which starts with a % sign is causing you trouble. What happens if you insert a comment # before the % sign?

bennahugo commented 1 year ago

It is not UTF8 encoded -- there is some weird accented characters in there One of the location names had a non-ascii character in there for instance and this caused it to break. The only way to read that is in binary mode and then stripping anything that is not 127 or less out of it.

bennahugo commented 1 year ago

Here is the offending line

list(filter(lambda l: any(map(lambda x: x > 127, l)), lines)) [b' bscn A BSCN00FRA P Besan\xc3\xa7on FRA 05 59 21.8 47 14 48.8 358.6 \n', b' spt7 A SPT700SWE P Bor\xc3\xa5s SWE 12 53 29.2 57 42 53.7 221.1 \n']

It appears as follows

bscn  A BSCN00FRA P         Besançon   FRA  05 59 21.8   47 14 48.8    358.6 
twillis449 commented 1 year ago

Ah - OK :) - might be a few other things like this in either the GNSS global stations or the EUREF stations.

twillis449 commented 1 year ago

BTW - if we ever get around to updating and submitting the ALBUS reference paper I will be adding your name to the list of authors. You've already done a lot more than some of the original authors. :)

twillis449 commented 1 year ago

BTW - when you upgrade to Python 3.10 this issue will go away as I guess it seems to have non-ascii understanding built in.