Closed durack1 closed 5 years ago
Option added and input arguments slightly revisited:
densityBin(fileT,fileS,fileFx,fileV='none',outFile='out.nc',debug=True,timeint='all',\
mthout=False,gridfT='none',gridfS='none',gridfV='none'):
if fileV='none'
the MSF calculation is ignored and V read is not required.
Tested and ok
@eguil I'm still hitting problems with this:
(cdat80py2) duro@ocean:[Density_bining]:[issue71_durack1_UpdateObsUpdateESMPCalls*]:[11638]> python drive_obs.py
Processing: 0
areacello: ~Shared/190213_data_density/obs.EN4.historical.r0i0p0.mo.ocn.Omon.so.ver-1.latestX.xml
so: ~Shared/190213_data_density/obs.EN4.historical.r0i0p0.mo.ocn.Omon.thetao.ver-1.latestX.xml
thetao: ~Shared/190213_data_density/obs.JAMSTEC.historical.r0i0p0.mo.ocn.Omon.so.ver-1.latestX.xml
Warning: ID reference not found: bounds=bounds_latitude
Warning: ID reference not found: bounds=bounds_longitude
Warning: ID reference not found: bounds=bounds_pressure
Warning: ID reference not found: bounds=bounds_time
Warning: ID reference not found: bounds=depth_bnds
Warning: ID reference not found: bounds=bounds_lat
Warning: ID reference not found: bounds=bounds_lon
Warning: ID reference not found: bounds=time_bnds
Warning: ID reference not found: bounds=depth_bnds
Warning: ID reference not found: bounds=bounds_lat
Warning: ID reference not found: bounds=bounds_lon
Warning: ID reference not found: bounds=time_bnds
Traceback (most recent call last):
File "drive_obs.py", line 87, in <module>
densityBin(thetao,so,areacello,'none',outfileDensity,debug=True,timeint='all')
File "~git/Density_bining/binDensity.py", line 358, in densityBin
thetao_h = ft('thetao', time = slice(1,10)) ; # remove handle for non cmor files
File "~anaconda2/envs/cdat80py2/lib/python2.7/site-packages/cdms2/cudsinterface.py", line 33, in __call__
raise CDMSError("No such variable, " + id)
cdms2.error.CDMSError: No such variable, thetao
@durack1 can you put the files somewhere on crunchy ? It is probably due to non CMOR format. Is the temperature variable name thetao ?
@eguil it still seems to be an issue:
(cdat80py2) durack1@oceanonly:[Density_bining]:[issue71_durack1_UpdateObsUpdateESMPCalls*]:[11506]> python drive_obs.py
Processing: 0
areacello: /work/durack1/Shared/190213_data_density/obs.EN4.historical.r0i0p0.mo.ocn.Omon.areacello.ver-1.latestX.nc
so: /work/durack1/Shared/190213_data_density/obs.EN4.historical.r0i0p0.mo.ocn.Omon.so.ver-1.latestX.xml
thetao: /work/durack1/Shared/190213_data_density/obs.EN4.historical.r0i0p0.mo.ocn.Omon.thetao.ver-1.latestX.xml
Warning: ID reference not found: bounds=depth_bnds
Warning: ID reference not found: bounds=bounds_lat
Warning: ID reference not found: bounds=bounds_lon
Warning: ID reference not found: bounds=time_bnds
Warning: ID reference not found: bounds=depth_bnds
Warning: ID reference not found: bounds=bounds_lat
Warning: ID reference not found: bounds=bounds_lon
Warning: ID reference not found: bounds=time_bnds
Traceback (most recent call last):
File "drive_obs.py", line 89, in <module>
densityBin(thetao,so,areacello,'none',outfileDensity,debug=True,timeint='all')
File "/export/durack1/git/Density_bining/binDensity.py", line 364, in densityBin
vo_h = fv('vo' , time = slice(1,10)) ; #
File "/export/durack1/anaconda2/envs/cdat80py2/lib/python2.7/site-packages/cdms2/cudsinterface.py", line 33, in __call__
raise CDMSError("No such variable, " + id)
cdms2.error.CDMSError: No such variable, vo
I'll email the paths to data
is the call you make to density bin correct (arguments have been re-organised) ?
@eguil looks right to me:
densityBin(thetao,so,areacello,'none',outfileDensity,debug=True,timeint='all')
https://github.com/eguil/Density_bining/blob/master/binDensity.py#L244-L266
def densityBin(fileT,fileS,fileFx,fileV='none',outFile='out.nc',debug=True,timeint='all',mthout=False,gridfT='none',gridfS='none',gridfV='none'):
'''
The densityBin() function takes file and variable arguments and creates
density persistence fields which are written to a specified outfile
Author: Eric Guilyardi : Eric.Guilyardi@locean-ipsl.upmc.fr
Co-author: Paul J. Durack : pauldurack@llnl.gov : @durack1.
Created on Sun Sep 14 21:13:30 2014
Inputs:
------
- fileT(time,lev,lat,lon) - 4D potential temperature array
- fileS(time,lev,lat,lon) - 4D salinity array
- fileFx(lat,lon) - 2D array containing the cell area values
-> options:
- fileV(time,lev,lat,lon) - 4D meridional velocity array
- outFile(str) - output file with full path specified.
- debug <optional> - boolean value
- timeint <optional> - specify temporal step for binning <init_idx>,<ncount>
- mthout <optional> - write annual data (False) or all monthly data (True)
- gridfT <optional> - file to get T grid info from
- gridfS <optional> - file to get S grid info from
- gridfV <optional> - file to get V grid info from
...
ok I managed to run my latest version of binDensity (i.e. without your last changes) on the first year of EN4 and it is running. Here is my test script:
from binDensity import densityBin
obsThetao = '/data/ericglod/Density_binning/190213_data_density/180723_EN4.2.1.g10_monthly_TAndS_190001-190912_5-5350m.nc'
obsSo = '/data/ericglod/Density_binning/190213_data_density/180723_EN4.2.1.g10_monthly_TAndS_190001-190912_5-5350m.nc'
obsAreacello = '/data/ericglod/Density_binning/190213_data_density/obs.EN4.historical.r0i0p0.mo.ocn.Omon.areacello.ver-1.latestX.nc'
outfileDensity = '/home/ericglod/Density_bining/test/EN4.mon.ocean.Omon.density.nc'
densityBin(obsThetao,obsSo,obsAreacello,'none',outFile=outfileDensity,timeint='1,12')
Not sure what is going on on your side. Maybe we should do a call with screen sharing.
@eguil what the the version of CDAT that you're running this against? A conda env?
an old PMP installation version
Got any more details? If it's conda (probably not), you can peruse through /work/eguilyardi/anaconda2/envs/ - if it's CICLAD (looks like it), I'm not sure..
I am running on ciclad and it is not a conda install.
Probably time to make the change #69, download anaconda
conda create -n cdat81py2 -c cdat/label/v81 -c conda-forge cdat "python<3"
Eric I don’t know if this can help but we can actually easily create conda environments on Ciclad, I’ve been testing it with simple stuff and installing new packages which seems to work, should we test that option ? That way we should be able to install the new version of cdat
Le 7 mars 2019 à 21:49, Paul J. Durack notifications@github.com a écrit :
Probably time to make the change, download anaconda https://www.anaconda.com/distribution/#download-section conda create -n cdat81py2 -c cdat/label/v81 -c conda-forge cdat "python<3" — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/eguil/Density_bining/issues/74#issuecomment-470688515, or mute the thread https://github.com/notifications/unsubscribe-auth/AWkPCkK_2hBNuoejAXVLNwkwE5rMCvImks5vUXtcgaJpZM4a9Dcp.
@ysilvy it's a good move to update yourselves and the way you work. Creating conda environments means that you can have versions side by side, which is a real advantage, particularly as new/beta releases become available where you can validate older libraries/code works
Ok I managed to sync again binDensity on Crunchy. I will try there.
@eguil let me know
Ok let's try that (I will transfer all the EN4 data). I can now sync git on crunchy but the python is now v3 and binDensity is v2. Where is the cdat that would work for python2 ?
the program is running on ciclad (old set up) but I have issues with the negative masking values in En4. Is it easy to change this in the files or should I work around in the code ?
@eguil make sure you included the "python<3"
from the https://github.com/eguil/Density_bining/issues/74#issuecomment-470688515, otherwise it will default to py3
I know this is confusing but my previous comment was for my tests on crunchy where I did not install CDAT.
@eguil I'm having trouble following the thread of this discussion above, facetime/skype?
closing this
EN4 and other obs datasets have no corresponding velocity time series. Consequently current code won’t process obs.