search

lofar-astron / RMextract

extract TEC, vTEC, Earthmagnetic field and Rotation Measures from GPS and WMM data for radio interferometry observations
GNU General Public License v3.0
31 stars 22 forks source link

months problem #23

Closed twillis449 closed 5 years ago

twillis449 commented 5 years ago

I installed RMextract. But I've encountered some really weird problem - It seems unable to generate codg file names if the month being specified is 4 (April) or less. e,g

following does work for month >= 5, fails for month <= 4 START_TIME="2016/4/09 00:00:50.1234" # fails END_TIME="2016/4/09 01:00:50.456" # fails

but following works START_TIME="2016/5/09 00:00:50.1234" # works END_TIME="2016/5/09 01:00:50.456" # works

See the test script

i'm using ubuntu 16.04 linux

test_times.py.gz

twillis449 commented 5 years ago

I get the same errors on an ubuntu 18.04 system

BTW: Its also a bit annoying that all those 'DEBUG:root ...' messages are presently turned on by default.

maaijke commented 5 years ago

Strange indeed, I cannot reproduce this. Could it be that the files on the server are corrupted, or that you have a corrupted IONEX file stored somewhere? It would be helpful if you could give the reported error.

twillis449 commented 5 years ago

I will put in some additional debug statements. I see that you have switched to using ftplib instead of the older (and simpler to me!) pycurl interface. So your coding has become somewhat harder for me to understand. I don't think this has anything to do with corrupted IONEX files because at least then the system should download something which would cause a local crash when trying to read and extract data from the file on my machine, What's happening is that it just reports that it could not find any 'codg' file to download. I'll get back to you with some more detailed output.

maaijke commented 5 years ago

I switched to ftplib to make the search path more general, such that if the server decides to change the path there is still a fair change to find the files. The problem is that I cannot reproduce the issue.

twillis449 commented 5 years ago

I have an old hard drive here which I will use to do a completely new install of ubuntu 18,04 wiping all previous stuff of the disk. Then I'll see what I get.

twillis449 commented 5 years ago

OK - found I already had a bare-bones install of ubuntu 18.04. After installing all supporting packages I ran the agw_examples.py script but changed the final start and end times to

START_TIME="2014/1/09 00:00:00" END_TIME="2014/1/09 23:59:59" I then get:

llis@twillis-Latitude-E6420:~/Downloads/RMextract-0.3/examples$ python agw_examples.py We will need PyEphem to perform calculations! the accuracy of results might decease a bit PyEphem will be used to perform calculations! observing in direction RA, DEC in radians 5.15199451295 -0.349065850399 pointing [ 5.15199451 -0.34906585] ('ephem start and end time ', 56666.0, 56666.99998842599) INFO:root:Retrieving data for 2014 or 14008 INFO:root:Retrieving data from gnss/products/ionex/2014 INFO:root: Traceback (most recent call last): File "agw_examples.py", line 64, in result = getRM(use_azel=use_azel,radec=pointing,out_file='test_report',object=OBJECT,start_time=START_TIME,end_time=END_TIME, timestep=TIME_STEP,stat_positions= upington_position,useEMM=True,TIME_OFFSET=TIME_OFFSET) File "/usr/local/lib/python2.7/dist-packages/RMextract-0.1-py2.7-linux-x86_64.egg/RMextract/getRM.py", line 153, in getRM ionexf=ionex.getIONEXfile(time=date_parms,server=server,prefix=prefix,outpath=ionexPath) File "/usr/local/lib/python2.7/dist-packages/RMextract-0.1-py2.7-linux-x86_64.egg/RMextract/getIONEX.py", line 544, in getIONEXfile return _get_IONEX_file(time, server, prefix, outpath, overwrite) File "/usr/local/lib/python2.7/dist-packages/RMextract-0.1-py2.7-linux-x86_64.egg/RMextract/getIONEX.py", line 507, in _get_IONEX_file prefix) AssertionError: No files found on ftp://cddis.gsfc.nasa.gov/gnss/products/ionex/ for codg

maaijke commented 5 years ago

THank you. THe before after April 9 issue must be related to the dayofyear going from 99 to 100. I see that I have put in a possibly related fix about a month ago. Could you check out the latest code directly from master (I also changed logging level to ERROR in that same fix, so it seems you do not have it). I created a V0.4 for you as well.

twillis449 commented 5 years ago

Yea team! Looks like you've fixed things with v 0.4. :)

The only odd thing I've noticed is that you generate an error message

/usr/local/lib/python2.7/dist-packages/RMextract-0.1-py2.7-linux-x86_64.egg/RMextract/getIONEX.py:188: RuntimeWarning: invalid value encountered in divide weights = weights / _steps

I'm not sure what you are doing there but it does not seem to affect the outcome.

twillis449 commented 5 years ago

Closing this issue.