search

eguil / Density_bining

Density bining code
2 stars 5 forks source link

binDensity crash - check inputs #40

Open durack1 opened 9 years ago

durack1 commented 9 years ago 
FileCount:  14
outPath:    /work/durack1/Shared/150313_data_density/historicalNat
outfile:    cmip5.MRI-CGCM3.historicalNat.r1i1p1.mo.ocn.Omon.density.ver-v20120510.latestX.WARN2.xml.nc
so:         cmip5.MRI-CGCM3.historicalNat.r1i1p1.mo.ocn.Omon.so.ver-v20120510.latestX.WARN2.xml
thetao:     cmip5.MRI-CGCM3.historicalNat.r1i1p1.mo.ocn.Omon.thetao.ver-v20120510.latestX.WARN2.xml
areacello:  cmip5.MRI-CGCM3.rcp85.r0i0p0.fx.ocn.fx.areacello.ver-v20110831.latestX.WARN2.xml
grdsize: 6756480
 ==> model: MRI-CGCM3  (grid size: 6756480 )
 ==> time interval:  0 1871
 ==> size of time chunk, number of time chunks (memory optimization) : 12 156

 --> time chunk (bounds) =  1 / 156  ( 0 11 ) MRI-CGCM3
Traceback (most recent call last):
  File ".//drive_density.py", line 262, in <module>
    densityBin(model[3],model[1],model[5],outfileDensity,debug=True,timeint='all')
  File "/export/durack1/git/Density_bining/binDensity.py", line 595, in densityBin
    nomask      = npy.equal(vmask_3D[0],0) ; # Returns boolean
IndexError: invalid index to scalar variable.

and

FileCount:  31                                                                                                                         
outPath:    /work/durack1/Shared/150313_data_density/rcp85                                                                             
outfile:    cmip5.MRI-CGCM3.rcp85.r1i1p1.mo.ocn.Omon.density.ver-v20120510.latestX.WARN2.xml.nc                                        
so:         cmip5.MRI-CGCM3.rcp85.r1i1p1.mo.ocn.Omon.so.ver-v20120510.latestX.WARN2.xml                                                
thetao:     cmip5.MRI-CGCM3.rcp85.r1i1p1.mo.ocn.Omon.thetao.ver-v20120510.latestX.WARN2.xml                                            
areacello:  cmip5.MRI-CGCM3.sstClim4xCO2.r0i0p0.fx.ocn.fx.areacello.ver-v20110831.latestX.xml                                          
grdsize: 6756480                                                                                                                       
 ==> model: MRI-CGCM3  (grid size: 6756480 )                                                                                           
 ==> time interval:  0 1139                                                                                                            
 ==> size of time chunk, number of time chunks (memory optimization) : 12 95                                                           

 --> time chunk (bounds) =  1 / 95  ( 0 11 ) MRI-CGCM3                                                                                 
Traceback (most recent call last):                                                                                                     
  File ".//drive_density.py", line 264, in <module>                                                                                    
    densityBin(model[3],model[1],model[5],outfileDensity,debug=True,timeint='all')                                                     
  File "/export/durack1/git/Density_bining/binDensity.py", line 595, in densityBin                                                     
    nomask      = npy.equal(vmask_3D[0],0) ; # Returns boolean                                                                         
IndexError: invalid index to scalar variable.                                                                                          
[durack1@crunchy Density_bining]$ cl
durack1 commented 9 years ago

Same issue, historical experiment:

 Wrote file:  /work/durack1/Shared/150313_data_density/historical/cmip5.MPI-ESM-P.historical.r1i1p1.an.ocn.Omon.density.ver-1.nc     
FileCount:  45                                                                                                                       
outPath:    /work/durack1/Shared/150313_data_density/historical                                                                      
outfile:    cmip5.MRI-CGCM3.historical.r1i1p1.mo.ocn.Omon.density.ver-v20120510.latestX.WARN2.xml.nc                                 
so:         cmip5.MRI-CGCM3.historical.r1i1p1.mo.ocn.Omon.so.ver-v20120510.latestX.WARN2.xml                                         
thetao:     cmip5.MRI-CGCM3.historical.r1i1p1.mo.ocn.Omon.thetao.ver-v20120510.latestX.WARN2.xml                                     
areacello:  cmip5.MRI-CGCM3.rcp85.r0i0p0.fx.ocn.fx.areacello.ver-v20110831.latestX.WARN2.xml                                         
grdsize: 6756480                                                                                                                     
 ==> model: MRI-CGCM3  (grid size: 6756480 )                                                                                         
 ==> time interval:  0 1871                                                                                                          
 ==> size of time chunk, number of time chunks (memory optimization) : 12 156                                                        

 --> time chunk (bounds) =  1 / 156  ( 0 11 ) MRI-CGCM3                                                                              
Traceback (most recent call last):                                                                                                   
  File ".//drive_density.py", line 262, in <module>                                                                                  
    densityBin(model[3],model[1],model[5],outfileDensity,debug=True,timeint='all')                                                   
  File "/export/durack1/git/Density_bining/binDensity.py", line 595, in densityBin                                                   
    nomask      = npy.equal(vmask_3D[0],0) ; # Returns boolean                                                                       
IndexError: invalid index to scalar variable.
eguil commented 9 years ago

ok I will look into this.

On 13/4/15 19:38, Paul J. Durack wrote:

Same issue, historical experiment:

| Wrote file: /work/durack1/Shared/150313_data_density/historical/cmip5.MPI-ESM-P.historical.r1i1p1.an.ocn.Omon.density.ver-1.nc FileCount: 45 outPath: /work/durack1/Shared/150313_data_density/historical outfile: cmip5.MRI-CGCM3.historical.r1i1p1.mo.ocn.Omon.density.ver-v20120510.latestX.WARN2.xml.nc so: cmip5.MRI-CGCM3.historical.r1i1p1.mo.ocn.Omon.so.ver-v20120510.latestX.WARN2.xml thetao: cmip5.MRI-CGCM3.historical.r1i1p1.mo.ocn.Omon.thetao.ver-v20120510.latestX.WARN2.xml areacello: cmip5.MRI-CGCM3.rcp85.r0i0p0.fx.ocn.fx.areacello.ver-v20110831.latestX.WARN2.xml grdsize: 6756480 ==> model: MRI-CGCM3 (grid size: 6756480 ) ==> time interval: 0 1871 ==> size of time chunk, number of time chunks (memory optimization) : 12 156

--> time chunk (bounds) = 1 / 156 ( 0 11 ) MRI-CGCM3 Traceback (most recent call last): File ".//drive_density.py", line 262, in densityBin(model[3],model[1],model[5],outfileDensity,debug=True,timeint='all') File "/export/durack1/git/Density_bining/binDensity.py", line 595, in densityBin nomask = npy.equal(vmask_3D[0],0) ; # Returns boolean IndexError: invalid index to scalar variable. |

— Reply to this email directly or view it on GitHub https://github.com/eguil/Density_bining/issues/40#issuecomment-92441688.

Eric Guilyardi IPSL/LOCEAN - Dir. Rech. CNRS Tour 45, 4eme, piece 406 UPMC, case 100 4 place Jussieu, F-75252 Paris Tel: +33 (0)1 44 27 70 76 Prof. Eric Guilyardi NCAS Climate Meteorology Department University of Reading Reading RG6 6BB - UK Tel: +44 (0)118 378 8315

             http://ncas-climate.nerc.ac.uk/~ericg
durack1 commented 9 years ago

@eguil these problems are most likely the inputs - the files include the .WARN2. identifier - so just leave this up to me.. I'm just restarting for the moment, will have to go and revisit once I get to the end of the available files.. We're still running out of memory on MIROC4h and MPI.. But again will just keep pushing this until I get to the end before digging deeper..

eguil commented 9 years ago

@durack1 ok fine

On 13/4/15 23:14, Paul J. Durack wrote:

@eguil https://github.com/eguil these problems are most likely the inputs - the files include the |.WARN2.| identifier - so just leave this up to me.. I'm just restarting for the moment, will have to go and revisit once I get to the end of the available files.. We're still running out of memory on MIROC4h and MPI.. But again will just keep pushing this until I get to the end before digging deeper..

— Reply to this email directly or view it on GitHub https://github.com/eguil/Density_bining/issues/40#issuecomment-92500812.

Eric Guilyardi IPSL/LOCEAN - Dir. Rech. CNRS Tour 45, 4eme, piece 406 UPMC, case 100 4 place Jussieu, F-75252 Paris Tel: +33 (0)1 44 27 70 76 Prof. Eric Guilyardi NCAS Climate Meteorology Department University of Reading Reading RG6 6BB - UK Tel: +44 (0)118 378 8315

             http://ncas-climate.nerc.ac.uk/~ericg
durack1 commented 9 years ago

And another: historicalNat cmip5.CESM1-CAM5.historicalNat.r1i1p1.an.ocn.Omon.density.ver-v20130909.nc

 --> time chunk (bounds) =  88 / 156  ( 1044 1055 ) CESM1-CAM5
*THETAO mean: +281.82; min: +270.23; max: +306.29
*THETAO mean: +08.67; min: -02.92; max: +33.14
  Total volume in z coordinates source grid (ref = 1.33 e+18) :  1.32186397697e+18
  Mean Temp./Salinity in z coordinates source grid            :  3.49228724533 34.7171399393
 Bining CPU analysis tc/tcdel =  87 12
    average cpu1  =  0.542500000001
    average cpu2  =  2.275
    average cpu3  =  0.321666666667
    average cpu4  =  27.07
    average cpu40 =  0.0908333333349
    average cpu5  =  0.598333333332
    CPU read T/S  =  7.07
    CPU comp. rho =  13.21
 Integral persistent values: 1.24750987849e+18 1.19516e+18 3.01982395738e+17
   % 95.8034683786 25.2671582723
  area global/atl/pac/ind  3.44248e+14 9.7772e+13 1.73523e+14 7.29536e+13
   T , S glob  3.44037747224 33.0646674819
   T , S atl   3.86833315611 31.8864366555
   T , S pac   3.09515285816 33.3947889029
   T , S ind   3.68796375976 33.8585274262
 Persistence CPU analysis t/nyrtc =  0 1
    cpu1 =  21.74
    cpu2 =  0.0
    cpu3 =  0.0
    cpu4 =  0.0
    cpu5 =  0.0
    cpu6 =  0.0
    cpu7 =  0.0
   CPU of chunk inits         = 21.91
   CPU of density bining      = 370.78
   CPU of masking and var def = 8.13
   CPU of annual mean compute = 4.83
   CPU of interpolation       = 5.96
   CPU of zonal mean          = 2.65
   CPU of persistence compute = 21.74
   CPU of chunk               = 436.0
   Max memory use 17.77614 GB
 --> time chunk (bounds) =  89 / 156  ( 1056 1067 ) CESM1-CAM5
*THETAO mean: +281.82; min: +270.04; max: +305.50
*THETAO mean: +08.67; min: -03.11; max: +32.35
  Total volume in z coordinates source grid (ref = 1.33 e+18) :  1.32186397697e+18
  Mean Temp./Salinity in z coordinates source grid            :  3.49275783464 34.7171368331
 Bining CPU analysis tc/tcdel =  88 12
    average cpu1  =  0.176666666667
    average cpu2  =  2.17083333333
    average cpu3  =  0.286666666666
    average cpu4  =  20.3491666667
    average cpu40 =  0.0716666666667
    average cpu5  =  0.449166666665
    CPU read T/S  =  3.81
    CPU comp. rho =  14.36
Traceback (most recent call last):
  File ".//drive_density.py", line 262, in <module>
    densityBin(model[3],model[1],model[5],outfileDensity,debug=True,timeint='all')
  File "/export/durack1/git/Density_bining/binDensity.py", line 969, in densityBin
    maskp = npy.ma.reshape(maskp, (N_s+1, latN*lonN))
  File "/usr/local/uvcdat/2014-11-24/lib/python2.7/site-packages/numpy/ma/core.py", line 6508, in reshape
    return a.reshape(new_shape, order=order)
ValueError: total size of new array must be unchanged
durack1 commented 9 years ago

These two issues are likely due to non-z vertical coordinates..

And another (this issue is also the case with r2i1p1 through r5i1p1):

FileCount:  41
outPath:    /work/durack1/Shared/150313_data_density/historical
outfile:    cmip5.MIROC5.historical.r1i1p1.mo.ocn.Omon.density.ver-1.nc
so:         cmip5.MIROC5.historical.r1i1p1.mo.ocn.Omon.so.ver-1.latestX.xml
thetao:     cmip5.MIROC5.historical.r1i1p1.mo.ocn.Omon.thetao.ver-1.latestX.xml
areacello:  cmip5.MIROC5.rcp85.r0i0p0.fx.ocn.fx.areacello.ver-1.latestX.xml
grdsize: 2867200
 ==> model: MIROC5  (grid size: 2867200 )
 ==> time interval:  0 1955
 ==> size of time chunk, number of time chunks (memory optimization) : 12 163
/usr/local/uvcdat/2014-11-24/lib/python2.7/site-packages/ESMP/src/ESMP_API.py:1241: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if (srcMaskValues != None):
/usr/local/uvcdat/2014-11-24/lib/python2.7/site-packages/ESMP/src/ESMP_API.py:1248: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if (dstMaskValues != None):

 --> time chunk (bounds) =  1 / 163  ( 0 11 ) MIROC5
Traceback (most recent call last):
  File ".//drive_density.py", line 262, in <module>
    densityBin(model[3],model[1],model[5],outfileDensity,debug=True,timeint='all')
  File "/export/durack1/git/Density_bining/binDensity.py", line 595, in densityBin
    nomask      = npy.equal(vmask_3D[0],0) ; # Returns boolean
IndexError: invalid index to scalar variable.

And another:

FileCount:  46
outPath:    /work/durack1/Shared/150313_data_density/historical
outfile:    cmip5.MRI-ESM1.historical.r1i1p1.mo.ocn.Omon.density.ver-v20130307.latestX.WARN2.xml.nc
so:         cmip5.MRI-ESM1.historical.r1i1p1.mo.ocn.Omon.so.ver-v20130307.latestX.WARN2.xml
thetao:     cmip5.MRI-ESM1.historical.r1i1p1.mo.ocn.Omon.thetao.ver-v20130307.latestX.WARN2.xml
areacello:  cmip5.MRI-ESM1.rcp85.r0i0p0.fx.ocn.fx.areacello.ver-v20140210.latestX.xml
grdsize: 6756480
 ==> model: MRI-ESM1  (grid size: 6756480 )
 ==> time interval:  0 1859
 ==> size of time chunk, number of time chunks (memory optimization) : 12 155
/usr/local/uvcdat/2014-11-24/lib/python2.7/site-packages/ESMP/src/ESMP_API.py:1241: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if (srcMaskValues != None):
/usr/local/uvcdat/2014-11-24/lib/python2.7/site-packages/ESMP/src/ESMP_API.py:1248: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if (dstMaskValues != None):

 --> time chunk (bounds) =  1 / 155  ( 0 11 ) MRI-ESM1
Traceback (most recent call last):
  File ".//drive_density.py", line 262, in <module>
    densityBin(model[3],model[1],model[5],outfileDensity,debug=True,timeint='all')
  File "/export/durack1/git/Density_bining/binDensity.py", line 595, in densityBin
    nomask      = npy.equal(vmask_3D[0],0) ; # Returns boolean
IndexError: invalid index to scalar variable.
durack1 commented 9 years ago

And another:

FileCount:  152
outPath:    /work/durack1/Shared/150313_data_density/historical
outfile:    cmip5.MIROC4h.historical.r2i1p1.mo.ocn.Omon.density.ver-1.nc
so:         cmip5.MIROC4h.historical.r2i1p1.mo.ocn.Omon.so.ver-1.latestX.xml
thetao:     cmip5.MIROC4h.historical.r2i1p1.mo.ocn.Omon.thetao.ver-1.latestX.xml
areacello:  cmip5.MIROC4h.rcp45.r0i0p0.fx.ocn.fx.areacello.ver-1.latestX.xml
grdsize: 56033280
 ==> model: MIROC4h  (grid size: 56033280 )
 ==> time interval:  0 671
 ==> size of time chunk, number of time chunks (memory optimization) : 12 56

 --> time chunk (bounds) =  1 / 56  ( 0 11 ) MIROC4h
trigger mask fix - EC-EARTH/MIROC4h
350524236
*THETAO mean: +281.47; min: +271.11; max: +309.97
*THETAO mean: +08.33; min: -02.04; max: +36.82
  Total volume in z coordinates source grid (ref = 1.33 e+18) :  1.36475536317e+18
  Mean Temp./Salinity in z coordinates source grid            :  4.03157081096 34.7199550623
 Bining CPU analysis tc/tcdel =  0 12
    average cpu1  =  1.42166666667
    average cpu2  =  21.6533333333
    average cpu3  =  2.38583333333
    average cpu4  =  236.875
    average cpu40 =  1.09749999999
    average cpu5  =  4.55416666667
    CPU read T/S  =  283.05
    CPU comp. rho =  248.47
0.0 0.0 0.0
0.0 0.0 0.0
  Total volume in rho coordinates source grid (ref = 1.33 e+18) :  1.31614338386e+18
  Mean Temp./Salinity in rho coordinates source grid            :  4.06755384743 34.8811525226
Traceback (most recent call last):
  File ".//drive_density.py", line 262, in <module>
    densityBin(model[3],model[1],model[5],outfileDensity,debug=True,timeint='all')
  File "/export/durack1/git/Density_bining/binDensity.py", line 961, in densityBin
    idxvm = 1-mv.masked_values(thickBin[inim:finm,:,:,:], valmask).mask
  File "/usr/local/uvcdat/2.2.0/lib/python2.7/site-packages/cdms2/MV2.py", line 867, in masked_values
    maresult = numpy.ma.masked_values(_makeMaskedArg(data), value, rtol=rtol, atol=atol, copy=copy)
  File "/usr/local/uvcdat/2.2.0/lib/python2.7/site-packages/numpy/ma/core.py", line 2193, in masked_values
    condition = umath.less_equal(mabs(xnew - value), atol + rtol * mabs(value))
MemoryError
durack1 commented 7 years ago

Revisit as specific test cases in #33 - test with new code and new ESMPy