Closed njarboe closed 5 years ago
moonshoes87
commented
5 years ago I suspect that we're attempting to do a lot more plots now that I've removed result_type, so we're getting never before seen problems. Yay. I've made several fixes, which should seriously cut down on the number of contributions that are raising errors. @njarboe, I think it would be reasonable at this point to re-run plots on the server at this point, and see if there are more problems lurking.
njarboe
commented
5 years ago @moonshoes87 Pulled from master this morning on the server and am running all the contributions again. Below are few errors from the beginning of the run. Also have included the error and stdout+error logs for this run up to now. The plots on the server won't be updated till all of them are finished. Looks like the 0 lat problem pops up, but also some others that may or may not be data corruption errors. batchLogError.txt batchLog.txt
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//polemap_magic.py", line 344, in
moonshoes87
commented
5 years ago @njarboe, thanks.
How much work is it for you to restart the process and try again? I've got some fixes based on the errors you sent me, but I haven't had a good way to test them since I can't easily find the failing contributions. If it's straightforward, I'll just push these few little fixes and you can restart. If it's annoying to restart, then I will try to track down the contribution ids of some of the problem contributions and do more in depth testing.
The good news is I think we will end up with a lot more plots since we aren't filtering out by result_type anymore, once we get all this straightened out.
njarboe
commented
5 years ago @moonshoes87 To restart is very easy. Takes less than a minute. If you push the changes I'll pull them onto the server and start again.
moonshoes87
commented
5 years ago Ok -- changes are up. I've been trying to test some locally, and so far the fixes seem fine. Update me when you have more errors!
njarboe
commented
5 years ago Ok. Running from the start. I'll let you know if/when some error show up.
njarboe
commented
5 years ago @moonshoes87 Pulled the latest and am re running. About half way through. One problem file so far:
working on: Mexico
-I- writing sites records to /home/magic/contributions/batchPlots/12365/tmp_sites.txt
-I- 110 records written to sites file
-I- writing locations records to /home/magic/contributions/batchPlots/12365/tmp_locations.txt
-I- 2 records written to locations file
-I- No sample data found
-I- No measurement data found
-I- result file found sites.txt
-I- working on site directions
number of datapoints: 110 Mexico
55 Dec/inc pairs found
number of individual directions: 55
number of tilt corrected directions: 0
number of geographic directions: 55
number of sample directions: 0
number of no coordinates directions: 0
eqarea_magic.py -f tmp_sites.txt -fsp tmp_specimens.txt -fsa tmp_samples.txt -flo tmp_locations.txt -sav -fmt png -crd g
-I- working on VGP map
vgpmap_magic.py -f tmp_sites.txt -prj moll -res c -sym ro 5 -sav -fmt png
-I- Look for intensities
-I- No intensities found
-I- working on pole map
polemap_magic.py -sav -fmt png -rev gv 40
file is /home/magic/contributions/afterVersion3/magic_contribution_12366.txt
startIndex is 59
65
endIndex is 65
diffIndex is 65-59-1
Dirctory is 12366
/home/magic/contributions/batchPlots/12366
working on: 'contribution'
1 records written to file ./contribution.txt
contribution data put in ./contribution.txt
working on: 'locations'
1 records written to file ./locations.txt
locations data put in ./locations.txt
working on: 'sites'
4 records written to file ./sites.txt
sites data put in ./sites.txt
working on: 'samples'
6 records written to file ./samples.txt
samples data put in ./samples.txt
working on: 'specimens'
6 records written to file ./specimens.txt
specimens data put in ./specimens.txt
working on: 'measurements'
215 records written to file ./measurements.txt
measurements data put in ./measurements.txt
working on: 'ages'
4 records written to file ./ages.txt
ages data put in ./ages.txt
-I- Using online data model
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//zeq_magic.py", line 586, in
moonshoes87
commented
5 years ago I added a fix -- don't restart until the CI tests have run and the above commit turns green.
moonshoes87
commented
5 years ago Well, CI tests are failing with a segfault, unrelated to this fix. I'm trying to fix that, so you can go ahead and restart make_magic_plots as you see fit.
njarboe
commented
5 years ago Here are a few more from before the above fix. These contributions are from when we had better data checking so I hope they are software bugs, but let me know if they are due to data problems. I'll run it again after you take a look at these? Thanks.
I put a "----------" line between errors.
file is /home/magic/contributions/afterVersion3/magic_contribution_12578.txt
startIndex is 59
65
endIndex is 65
diffIndex is 65-59-1
Dirctory is 12578
/home/magic/contributions/batchPlots/12578
working on: 'contribution'
1 records written to file ./contribution.txt
contribution data put in ./contribution.txt
working on: 'locations'
2 records written to file ./locations.txt
locations data put in ./locations.txt
working on: 'ages'
2 records written to file ./ages.txt
ages data put in ./ages.txt
-I- Using online data model
gridlines only supported for PlateCarree and Mercator plots currently
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//polemap_magic.py", line 347, in
/home/magic/contributions/batchPlots/12742 working on: 'contribution' 1 records written to file ./contribution.txt contribution data put in ./contribution.txt working on: 'locations' 2480 records written to file ./locations.txt locations data put in ./locations.txt working on: 'sites' 455 records written to file ./sites.txt sites data put in ./sites.txt working on: 'samples' 889 records written to file ./samples.txt samples data put in ./samples.txt working on: 'specimens' 1073 records written to file ./specimens.txt specimens data put in ./specimens.txt working on: 'measurements' 37919 records written to file ./measurements.txt measurements data put in ./measurements.txt working on: 'criteria' 13 records written to file ./criteria.txt criteria data put in ./criteria.txt working on: 'ages' 70 records written to file ./ages.txt ages data put in ./ages.txt -I- Using online data model
working on: 169
-I- writing measurements records to /home/magic/contributions/batchPlots/12742/tmp_measurements.txt
-I- 184 records written to measurements file
-I- writing specimens records to /home/magic/contributions/batchPlots/12742/tmp_specimens.txt
-I- 4 records written to specimens file
-I- writing samples records to /home/magic/contributions/batchPlots/12742/tmp_samples.txt
-I- 6 records written to samples file
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs/make_magic_plots.py", line 455, in
working on: Point Camden Gabbro
-I- writing locations records to /home/magic/contributions/batchPlots/12767/tmp_locations.txt
-I- 3 records written to locations file
-I- No sample data found
-I- No measurement data found
-I- working on pole map
polemap_magic.py -sav -fmt png -rev gv 40
file is /home/magic/contributions/afterVersion3/magic_contribution_12768.txt
startIndex is 59
65
endIndex is 65
diffIndex is 65-59-1
Dirctory is 12768
/home/magic/contributions/batchPlots/12768
working on: 'contribution'
1 records written to file ./contribution.txt
contribution data put in ./contribution.txt
working on: 'locations'
2 records written to file ./locations.txt
locations data put in ./locations.txt
working on: 'sites'
29 records written to file ./sites.txt
sites data put in ./sites.txt
working on: 'samples'
14 records written to file ./samples.txt
samples data put in ./samples.txt
working on: 'specimens'
114 records written to file ./specimens.txt
specimens data put in ./specimens.txt
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//histplot.py", line 113, in
-W- 12768 plotting the entire contribution as one location
working on: Younger Volcanic Series, Mauritius
-I- writing locations records to /home/magic/contributions/batchPlots/12988/tmp_locations.txt
-I- 3 records written to locations file
-I- No sample data found
-I- No measurement data found
-I- working on pole map
polemap_magic.py -sav -fmt png -rev gv 40
file is /home/magic/contributions/afterVersion3/magic_contribution_12989.txt
startIndex is 59
65
endIndex is 65
diffIndex is 65-59-1
Dirctory is 12989
/home/magic/contributions/batchPlots/12989
working on: 'contribution'
1 records written to file ./contribution.txt
contribution data put in ./contribution.txt
working on: 'locations'
244 records written to file ./locations.txt
locations data put in ./locations.txt
working on: 'sites'
4113 records written to file ./sites.txt
sites data put in ./sites.txt
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//histplot.py", line 113, in
working on: Japan -I- writing specimens records to /home/magic/contributions/batchPlots/13356/tmp_specimens.txt -I- 58 records written to specimens file -I- writing samples records to /home/magic/contributions/batchPlots/13356/tmp_samples.txt -I- 58 records written to samples file -I- writing sites records to /home/magic/contributions/batchPlots/13356/tmp_sites.txt -I- 20 records written to sites file -I- writing locations records to /home/magic/contributions/batchPlots/13356/tmp_locations.txt -I- 1 records written to locations file using specimen coordinates -I- No measurement data found -I- result file found sites.txt -I- working on site directions number of datapoints: 20 Japan 10 Dec/inc pairs found number of individual directions: 10 number of tilt corrected directions: 10 number of geographic directions: 0 number of sample directions: 0 number of no coordinates directions: 0 eqarea_magic.py -f tmp_sites.txt -fsp tmp_specimens.txt -fsa tmp_samples.txt -flo tmp_locations.txt -sav -fmt png -crd t -I- working on VGP map -I- No vgps found -I- Look for intensities histplot.py -b 1 -xlab 'Intensity (uT)' -sav -f intensities.txt -F LO:_Japan_TY:_intensitieshistogram:.png working on hysteresis specimens.txt no hysteresis data found working on anisotropy specimens.txt No anisotropy data found -I- working on pole map -I- No poles found file is /home/magic/contributions/afterVersion3/magic_contribution_13357.txt startIndex is 59 65 endIndex is 65 diffIndex is 65-59-1 Dirctory is 13357 /home/magic/contributions/batchPlots/13357 working on: 'contribution' 1 records written to file ./contribution.txt contribution data put in ./contribution.txt working on: 'locations' 1 records written to file ./locations.txt locations data put in ./locations.txt working on: 'sites' 16 records written to file ./sites.txt sites data put in ./sites.txt working on: 'specimens' 15 records written to file ./specimens.txt specimens data put in ./specimens.txt working on: 'ages' 16 records written to file ./ages.txt ages data put in ./ages.txt -I- Using online data model Traceback (most recent call last): File "/home/magic/anaconda3/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 3064, in get_loc return self._engine.get_loc(key) File "pandas/_libs/index.pyx", line 140, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/index.pyx", line 162, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/hashtable_class_helper.pxi", line 1492, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas/_libs/hashtable_class_helper.pxi", line 1500, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'specimen'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs/make_magic_plots.py", line 455, in
working on: Pilansberg Dykes
-I- writing locations records to /home/magic/contributions/batchPlots/13630/tmp_locations.txt
-I- 1 records written to locations file
-I- No sample data found
-I- No measurement data found
-I- working on pole map
polemap_magic.py -sav -fmt png -rev gv 40
file is /home/magic/contributions/afterVersion3/magic_contribution_13631.txt
startIndex is 59
65
endIndex is 65
diffIndex is 65-59-1
Dirctory is 13631
/home/magic/contributions/batchPlots/13631
working on: 'contribution'
1 records written to file ./contribution.txt
contribution data put in ./contribution.txt
working on: 'locations'
1 records written to file ./locations.txt
locations data put in ./locations.txt
working on: 'sites'
1 records written to file ./sites.txt
sites data put in ./sites.txt
working on: 'samples'
9 records written to file ./samples.txt
samples data put in ./samples.txt
working on: 'specimens'
9 records written to file ./specimens.txt
specimens data put in ./specimens.txt
working on: 'measurements'
91 records written to file ./measurements.txt
measurements data put in ./measurements.txt
working on: 'ages'
1 records written to file ./ages.txt
ages data put in ./ages.txt
-I- Using online data model
1 saved in LO:_DiscoverySite_SI:_JH5O_SA:_5s1-f1_SP:_5s1-f1_TY:eqarea.png
2 saved in LO:_DiscoverySite_SI:_JH5O_SA:_5s1-f1_SP:_5s1-f1_TY:zijd.png
3 saved in LO:_DiscoverySite_SI:_JH5O_SA:_5s1-f1_SP:_5s1-f1_TY:demag.png
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//zeq_magic.py", line 586, in
moonshoes87
commented
5 years ago @njarboe, one note for the server: make sure matplotlib=2.2.3. The default is now 3.0, and that breaks things. 3.0.2 should be ok, once it is released.
moonshoes87
commented
5 years ago @njarboe, I believe I've resolved most of these issues. There are still two outstanding plotting errors that I know about (polemap_magic.py #475 and thellier_magic.py #474) , which I'm getting Lisa's input on. Also, for what it's worth, Travis CI tests are passing again. Feel free to re-run make_magic_plots, although at least contribution 12578 will still have errors.
njarboe
commented
5 years ago @moonshoes87, I haven't upgraded for awhile and have matplotlib=2.2.2
That seems to work OK. How would you switch to 2.2.3 and not to 3.0 if I need to?
I'll re-run now.
moonshoes87
commented
5 years ago To install a specific version, the syntax is: conda install matplotlib=2.2.3
moonshoes87
commented
5 years ago @njarboe, I'm closing this issue as it has gotten very long. Please open a new issue with the next set of problems.
@moonshoes87 I pulled the most recent pmagpy onto the pmagpy server and ran through the files again with make_magic_plots. Included is the error log and the plain output log. I have include those files and you can reach the output plots and other related files for a contribution produced in the process at: https://pmagpy.earthref.org/plots/$CONTRIBUTION_NUMBER https://pmagpy.earthref.org/plots/12089/ for example. batchLogError.txt batchLog.txt
Looking at the batchLogError.txt there are a few issues that repeat themselves. I'm sure some problems are due to data missing or incorrect, but here are a few examples from the start of the error log: (I'm not sure the best way to report these bugs @moonshoes87. Let me know if you have ideas on that I can use.)
Traceback (most recent call last): File "/home/magic/git/PmagPy/programs//histplot.py", line 117, in
main()
File "/home/magic/git/PmagPy/programs//histplot.py", line 96, in main
Nbins = old_div(len(D), binsize)
TypeError: len() of unsized object
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//histplot.py", line 117, in
main()
File "/home/magic/git/PmagPy/programs//histplot.py", line 96, in main
Nbins = old_div(len(D), binsize)
TypeError: len() of unsized object
-W- 12089 Derbyshire equarea_magic.py 4 dec/inc pairs found, but no equal area plots were made
-W- 12089 Scotland equarea_magic.py 9 dec/inc pairs found, but no equal area plots were made
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//polemap_magic.py", line 348, in
main()
File "/home/magic/git/PmagPy/programs//polemap_magic.py", line 192, in main
isign = abs(lats[-1]) / lats[-1]
ZeroDivisionError: float division by zero
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//histplot.py", line 117, in
main()
File "/home/magic/git/PmagPy/programs//histplot.py", line 96, in main
Nbins = old_div(len(D), binsize)
TypeError: len() of unsized object
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//histplot.py", line 117, in
main()
File "/home/magic/git/PmagPy/programs//histplot.py", line 96, in main
Nbins = old_div(len(D), binsize)
TypeError: len() of unsized object
Traceback (most recent call last):
File "/home/magic/git/PmagPy/programs//histplot.py", line 117, in
main()
File "/home/magic/git/PmagPy/programs//histplot.py", line 96, in main
Nbins = old_div(len(D), binsize)
TypeError: len() of unsized object
-W- 12259 Mt Circe equarea_magic.py 1 dec/inc pairs found, but no equal area plots were made
-W- 12259 Mt Knobhead equarea_magic.py 1 dec/inc pairs found, but no equal area plots were made