demiangomez / Parallel.GAMIT

Python wrapper to parallelize GAMIT executions
BSD 3-Clause "New" or "Revised" License
36 stars 17 forks source link

read o.Z files natively #13

Closed smalleyr closed 6 years ago

smalleyr commented 6 years ago

I'm bringing in the CAP campaign data - the 1993 data is in rinex v1 and rnx2crx only works with v2 and above. The current solution is to use teqc to convert it all to rinex v2 (I'm testing the results of both now)

smalleyr commented 6 years ago

pyPlotETM: got following error message that seems to have no effect as there are no "missing" plots (alphabetical list of bra net stns goes from aplj to apsa with on station between).

Successfully plotted bra.aplj /Volumes/Sierra750GB/usr/local/pyconda/lib/python2.7/site-packages/numpy/linalg/linalg.py:1487: RuntimeWarning: divide by zero encountered in true_divide return s[..., 0]/s[..., -1] Successfully plotted bra.apsa

Got 4 errors like this, pdlg had 5 occupations, the other 3 had 6 or 7 occupations, no plot of course

Successfully plotted cer.pdbl Error during processing of cer.pdlg Traceback (most recent call last): File "../classes/pyPlotETM.py", line 52, in main etm = pyPPPETM.ETM(cnn, stn['NetworkCode'], stn['StationCode'], False) File "/Volumes/UsersDrive/Users/smalley/Working.Parallel.GAMIT/classes/pyPPPETM.py", line 624, in init self.Jumps = JumpsTable(cnn, NetworkCode, StationCode, ppp.t, add_antenna_jumps=self.Periodic.params) File "/Volumes/UsersDrive/Users/smalley/Working.Parallel.GAMIT/classes/pyPPPETM.py", line 294, in init DOP = np.diag(np.linalg.inv(np.dot(self.A.transpose(), self.A))) File "/Volumes/Sierra750GB/usr/local/pyconda/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 513, in inv ainv = _umath_linalg.inv(a, signature=signature, extobj=extobj) File "/Volumes/Sierra750GB/usr/local/pyconda/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 90, in _raise_linalgerror_singular raise LinAlgError("Singular matrix") LinAlgError: Singular matrix

Successfully plotted cer.pdrg

Finally, here were missing "successfully plotted files", I put the output of pyPlotETM into a file called "z"

capybara:production smalley$ grep Succ z | wc 499 1497 14970 capybara:production smalley$ ls *png | wc 415 415 5395

more comments will come by email.

smalleyr commented 6 years ago

more on pyPlotETM

First, I do not understand the -json option, what is JSON? Does a 0, an integer, follow JSON, as another parameter, or is it a possible value for JSON?

"too" (at the end of the help, option -json) sounds weird. Better to use "as well" or

capybara:run_dir smalley$ python ../classes/pyPlotETM.py -h usage: pyPlotETM.py [-h] [-np] [-dir DIRECTORY] [-json JSON] stnlist [stnlist ...]

Plot ETM for stations in the database

positional arguments: stnlist List of networks/stations to plot given in [net].[stnm] format or just [stnm] (separated by spaces; if [stnm] is not unique in the database, all stations with that name will be plotted). Use keyword 'all' to plot all stations in all networks. If [net].all is given, all stations from network [net] will be plotted

optional arguments: -h, --help show this help message and exit -np, --noparallel Execute command without parallelization -dir DIRECTORY, --directory DIRECTORY Directory to save the resulting PNG files. If not specified, assumed to be the production directory -json JSON, --json JSON Export ETM adjustment to JSON. 0 no time series; otherwise, export time series too

capybara:run_dir smalley$ python ../classes/pyPlotETM.py all -json 0 ==== Starting JobServer(pp) ====

Checking requirements at the local node... -- capybara.ceri.memphis.edu: Test passed! Checking requirements at the remote nodes... -- carpincho.ceri.memphis.edu: Test passed! Parallel Python started with the following nodes: -- IP/Name CPUs local 7 141.225.157.178:60000 7

Successfully plotted bra.alar Successfully plotted bra.amco Successfully plotted bra.amhu Successfully plotted bra.amte Successfully plotted bra.aplj /Volumes/Sierra750GB/usr/local/pyconda/lib/python2.7/site-packages/numpy/linalg/linalg.py:1487: RuntimeWarning: divide by zero encountered in true_divide return s[..., 0]/s[..., -1] Successfully plotted bra.apsa

and I now get a very large number of this error

Error during processing of cer.vied Traceback (most recent call last): File "../classes/pyPlotETM.py", line 60, in main json.dump(etm.todictionary(False), f, indent=4, sort_keys=False) File "/Volumes/UsersDrive/Users/smalley/Working.Parallel.GAMIT/classes/pyPPPETM.py", line 752, in todictionary etm['Linear'] = {'tref': self.Linear.tref, 'params': self.Linear.values.tolist()} AttributeError: ETM instance has no attribute 'Linear'

the json file is 0 long and there is no png file. When I ran it without the -json flag, it reported successful plotting for vied, but there was no png file (one of the ~80 files reported as successful with no output).

I also am getting this message

Error during processing of cer.cris Traceback (most recent call last): File "../classes/pyPlotETM.py", line 52, in main etm = pyPPPETM.ETM(cnn, stn['NetworkCode'], stn['StationCode'], False) File "/Volumes/UsersDrive/Users/smalley/Working.Parallel.GAMIT/classes/pyPPPETM.py", line 624, in init self.Jumps = JumpsTable(cnn, NetworkCode, StationCode, ppp.t, add_antenna_jumps=self.Periodic.params) File "/Volumes/UsersDrive/Users/smalley/Working.Parallel.GAMIT/classes/pyPPPETM.py", line 294, in init DOP = np.diag(np.linalg.inv(np.dot(self.A.transpose(), self.A))) File "/Volumes/Sierra750GB/usr/local/pyconda/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 513, in inv ainv = _umath_linalg.inv(a, signature=signature, extobj=extobj) File "/Volumes/Sierra750GB/usr/local/pyconda/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 90, in _raise_linalgerror_singular raise LinAlgError("Singular matrix") LinAlgError: Singular matrix

and I think there are the ones that were failing on the first run (without the -json 0) and not reporting successful plotting

smalleyr commented 6 years ago

running pyPlotETM on just bra I get the divide by zero error (shown in previous) and two of "AttributeError: ETM instance has no attribute 'Linear'" errors on continuous stations.

smalleyr commented 6 years ago

No errors in rms network

smalleyr commented 6 years ago

1 error in sir

demiangomez commented 6 years ago

This is NOT how you are supposed to submit issues. "Machine gun" submission is NOT helpful.