PCMDI / cmor

Climate Model Output Rewriter
BSD 3-Clause "New" or "Revised" License
53 stars 32 forks source link

not able to determine if axis is level based on units #184

Closed aradhakrishnanGFDL closed 7 years ago

aradhakrishnanGFDL commented 7 years ago

What am I missing? The following is the tail of the stdout while running prepare with cmor3.2. Should I install genutil outside of this anaconda package?

/home/a1r/anaconda2/envs/CMOR/lib/python2.7/site-packages/cdms2/axis.py:769: UserWarning: genutil module not present, was not able to determine if axis is level based on units "genutil module not present, was not able to determine if axis is level based on units") CDMS error: Error on character time conversion, string = ?

CDMS error: Error on character time conversion, string = ?

Traceback (most recent call last): File "/home/a1r/anaconda2/envs/CMOR/bin/PrePARE", line 11, in load_entry_point('CMOR==3.2', 'console_scripts', 'PrePARE')() File "/home/a1r/anaconda2/envs/CMOR/lib/python2.7/site-packages/cmip6_cv/PrePARE/PrePARE.py", line 371, in main process.ControlVocab() File "/home/a1r/anaconda2/envs/CMOR/lib/python2.7/site-packages/cmip6_cv/PrePARE/PrePARE.py", line 329, in ControlVocab raise KeyboardInterrupt KeyboardInterrupt

dnadeau4 commented 7 years ago

Are you using CMOR 3.2.3? The entry_point did not work in CMOR 3.2.2

dnadeau4 commented 7 years ago

You can install genutil conda install -c conda-forge genutil

aradhakrishnanGFDL commented 7 years ago

Ok, so I did this: conda install -c conda-forge genutil

And, the following is my new error. Anything else I need to do?

Traceback (most recent call last): File "/home/a1r/anaconda2/envs/CMOR/bin/PrePARE", line 11, in load_entry_point('CMOR==3.2', 'console_scripts', 'PrePARE')() File "/home/a1r/anaconda2/envs/CMOR/lib/python2.7/site-packages/cmip6_cv/PrePARE/PrePARE.py", line 404, in main process.ControlVocab() File "/home/a1r/anaconda2/envs/CMOR/lib/python2.7/site-packages/cmip6_cv/PrePARE/PrePARE.py", line 362, in ControlVocab raise KeyboardInterrupt

dnadeau4 commented 7 years ago

@aradhakrishnanGFDL what is the version your are using? conda list | grep cmor

# packages in environment at ../anaconda2/envs/cmor3.2.4:
cmor                      3.2.4               np112py27_0    pcmdi
aradhakrishnanGFDL commented 7 years ago

3.2.4

dnadeau4 commented 7 years ago

You should have this entry point files. .../anaconda2/envs/cmor3.2.4/lib/python2.7/site-packages/CMOR-3.2-py2.7-linux-x86_64.egg-info/entry_points.txt

with the following lines.

[console_scripts]
PrePARE = cmip6_cv.PrePARE:main

Can you try this script? (cut/paste in python shell)

__requires__ = 'CMOR==3.2'
import re
import sys
from pkg_resources import load_entry_point
load_entry_point('CMOR==3.2', 'console_scripts', 'PrePARE')()
dnadeau4 commented 7 years ago

@aradhakrishnanGFDL This error can also due to a conflict with udunits2.xml export. Can you check that your environment is right? This is added when you activate your cmor env.

env | grep UDUNITS2
UDUNITS2_XML_PATH=.../anaconda2/envs/cmor3.2.4/share/udunits/udunits2.xml
doutriaux1 commented 7 years ago

@dnadeau4 you shouldn't need to have UDUNITS2_XML_PATH set at all. Why do you need this? We solved that in genutil.udunits.

doutriaux1 commented 7 years ago

@dnadeau4 you really should try very hard to not do extra modif of user env

doutriaux1 commented 7 years ago

@dnadeau4 it's something like at: https://github.com/UV-CDAT/genutil/blob/master/Src/udunits_wrap.c#L13

dnadeau4 commented 7 years ago

You need it for CMOR.

dnadeau4 commented 7 years ago

https://github.com/PCMDI/cmor/blob/master/Src/cmor.c#L433

doutriaux1 commented 7 years ago

but why? For the C?

doutriaux1 commented 7 years ago

It's conda yuo know where the file is, stick it at: https://github.com/PCMDI/cmor/blob/master/Src/cmor.c#L957

dnadeau4 commented 7 years ago

It's not always conda. People at MetOffice compile cmor manually. Do you know why @aradhakrishnanGFDL has this issue with genutil?

doutriaux1 commented 7 years ago

well if it's not conda then they wont have your UDUNITS2_XML_PATH set and it shouldn't interfer

doutriaux1 commented 7 years ago

@dnadeau4 let'stalk tomorrow in person, it'll be easier

dnadeau4 commented 7 years ago

@doutriaux1 will fix genutil or cmor for you.

aradhakrishnanGFDL commented 7 years ago

Thanks Denis.

dnadeau4 commented 7 years ago

@doutriaux1 Is this working now?

dnadeau4 commented 7 years ago

@aradhakrishnanGFDL this should work in 3.2.6. Reopen if needed.

aradhakrishnanGFDL commented 7 years ago

Hi,

I believe I still get the same error with 3.2.6. If I. may be missing, please help debug. We have an upcoming training (next week) at GFDL and it will be great to have this as part of our tool set.

/home/pcmdi/.conda/envs/cmor/lib/python2.7/site-packages/cdms2/axis.py:769: UserWarning: genutil module not present, was not able to determine if axis is level based on units "genutil module not present, was not able to determine if axis is level based on units") CDMS error: Error on character time conversion, string = ?

On Thu, Aug 31, 2017 at 12:45 PM, Denis Nadeau notifications@github.com wrote:

@aradhakrishnanGFDL https://github.com/aradhakrishnangfdl this should work in 3.2.6. Reopen if needed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PCMDI/cmor/issues/184#issuecomment-326355061, or mute the thread https://github.com/notifications/unsubscribe-auth/AZHW6tO53Zz9-mi6LrcZIZWDQRV20p7xks5sduNFgaJpZM4NwS64 .

-- Aparna Radhakrishnan Engility, NOAA/GFDL

Cell: 313-461-9948

dnadeau4 commented 7 years ago

@doutriaux1 what is the solution?

doutriaux1 commented 7 years ago

I don't know it's a cmor thing

dnadeau4 commented 7 years ago

@aradhakrishnanGFDL can you provide use with some code? I am very confused by your error, I don't have this error in any of my tests.

doutriaux1 commented 7 years ago

@aradhakrishnanGFDL can you send us your netcdf file as well? maybe an ncdump -h would be nice if posted here. Thx!

aradhakrishnanGFDL commented 7 years ago

Thanks Charles and Denis for the quick responses! (sorry in advance for the longgg post) Interesting- I got the genutil error for the following header -A. But, in one of our newer files (header-B), I don't see this particular error. One of the important changes in header-B is the correction of time:units, whereas header-B has-> time:units = "days since ?" ; This is one possible trigger. But, I am not sure what else I am missing otherwise. More notes on what was installed can be found below.

header-A

netcdf ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512 { dimensions: bnds = 2 ; lat = 180 ; lon = 288 ; time = UNLIMITED ; // (420 currently) variables: double bnds(bnds) ; bnds:long_name = "vertex number" ; double lat(lat) ; lat:long_name = "latitude" ; lat:units = "degrees_north" ; lat:bounds = "lat_bnds" ; lat:axis = "Y" ; lat:standard_name = "latitude" ; double lat_bnds(lat, bnds) ; double lon(lon) ; lon:long_name = "longitude" ; lon:units = "degrees_east" ; lon:bounds = "lon_bnds" ; lon:axis = "X" ; lon:standard_name = "longitude" ; double lon_bnds(lon, bnds) ; double time(time) ; time:long_name = "time" ; time:units = "days since ?" ; time:axis = "T" ; time:calendar = "julian" ; time:bounds = "time_bnds" ; time:description = "for time-mean fields" ; time:standard_name = "time" ; double time_bnds(time, bnds) ; time_bnds:long_name = "time axis boundaries" ; time_bnds:units = "days since 1979-01-01 00:00:00" ; time_bnds:missing_value = 1.e+20 ; time_bnds:_FillValue = 1.e+20 ; float ts(time, lat, lon) ; ts:long_name = "Surface Temperature" ; ts:units = "K" ; ts:missing_value = 1.e+20f ; ts:cell_methods = "area: time: mean" ; ts:cell_measures = "area: areacella" ; ts:standard_name = "surface_temperature" ; ts:interp_method = "conserve_order2" ; ts:_FillValue = 1.e+20f ; ts:original_name = "ts" ; ts:associated_files = "baseURL: http://nomads.gfdl.noaa.gov/CM2.X/associated_files.html areacella: areacella_fx_GFDL-AM4_c96L33_am4p0_cmip6Diag_r0i0p0.nc" ;

// global attributes: :title = "NOAA GFDL GFDL-AM4 model output prepared for CMIP6 c96L33_am4p0_cmip6Diag" ; :history = "Mon Jun 5 12:46:27 2017: ncatted -O -a license,global,o,c,CMIP6 model data produced by GFDL is licensed under a Creative Commons Attribution ShareAlike 4.0 International License ( https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law. /home/a1r/san/prepare/ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "Mon Jun 5 12:33:30 2017: ncatted -O -a license,global,o,c,CMIP6 model data produced by Lawrence Livermore PCMDI is licensed under a Creative Commons Attribution ShareAlike 4.0 International License ( https://creativecommons.org/licenses ) . Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url ( recorded as a global attribute in this file ) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information ( including any liability arising in negligence ) are excluded to the fullest extent permitted by law. ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "Mon Jun 5 12:19:18 2017: ncatted -O -a license,global,o,c,CMIP6 model data produced by GFDL is licensed under a Creative Commons Attribution ShareAlike 4.0 International License ( https://creativecommons.org/licenses ) . Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url ( recorded as a global attribute in this file ) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information ( including any liability arising in negligence ) are excluded to the fullest extent permitted by law. ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "Mon Jun 5 12:18:46 2017: ncatted -O -a license,global,o,c,CMIP6 model data produced by Lawrence Livermore PCMDI is licensed under a Creative Commons Attribution ShareAlike 4.0 International License ( https://creativecommons.org/licenses ) . Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url ( recorded as a global attribute in this file ) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information ( including any liability arising in negligence ) are excluded to the fullest extent permitted by law. ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "Thu Jun 1 11:05:58 2017: ncatted -O -a license,global,o,c,^CMIP6 model data produced by NOAA-GFDL is licensed under a Creative Commons Attribution.ShareAlike 4.0 International License ( https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) .The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law. /home/a1r/san/prepare/ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "Thu Jun 1 11:05:16 2017: ncatted -O -a license,global,o,c,This CMIP6 model data produced by NOAA-GFDL is licensed under a Creative Commons Attribution.ShareAlike 4.0 International License ( https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) .The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law. /home/a1r/san/prepare/ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "Thu Jun 1 11:03:54 2017: ncatted -O -a license,global,o,c,CMIP6 model data produced by NOAA-GFDL is licensed under a Creative Commons Attribution.*ShareAlike 4.0 International License ( https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) .The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law. /home/a1r/san/prepare/ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "Thu Jun 1 10:44:32 2017: ncatted -O -a source_type,global,o,c,AGCM /home/a1r/san/prepare/ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "Thu Jun 1 10:23:53 2017: ncatted -O -a experiment,global,o,c,amip /home/a1r/san/prepare/ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "Thu Jun 1 10:23:46 2017: ncatted -O -a experiment_id,global,o,c,amip /home/a1r/san/prepare/ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_000101-003512.nc\n", "File was processed by fremetar (GFDL analog of CMOR). TripleID: [exper_id_tapHVfqhIr,realiz_id_NFMP2wmZCu,run_id_xjMXuHqaT2]" ; :contact = "gfdl.climate.model.info@noaa.gov" ; :comment = "Experiment name = c96L33_am4p0_cmip6Diag" ; :tracking_id = "hdl:21.14100/46dd25b1-fd28-4983-834f-c2759d2c049d" ; :further_info_url = " http://furtherinfo.es-doc.org/CMIP6.NOAA-GFDL.GFDL-AM4.c96L33_am4p0_cmip6Diag.r1i1p1f1" ; :branch_method = "no parent" ; :branch_time_in_parent = 0. ; :creation_date = "2017-05-24T20:14:19Z" ; :Conventions = "CF-1.7 CMIP-6.0 UGRID-1.0" ; :table_id = "Amon" ; :sub_experiment = "none" ; :frequency = "mon" ; :forcing_index = 1 ; :physics_index = 1 ; :initialization_index = 1 ; :realization_index = 1 ; :parent_variant_label = "no parent" ; :parent_experiment_id = "no parent" ; :data_specs_version = "01.00.10" ; :external_variables = "area: areacella" ; :experiment_id = "amip" ; :experiment = "amip" ; :activity_id = "CMIP" ; :source_id = "GFDL-AM4" ; :source_type = "AGCM" ; :institution_id = "NOAA-GFDL" ; :institution = "National Oceanic and Atmospheric Administration, Geophysical Fluid Dynamics Laboratory, Princeton, NJ 08540, USA" ; :variable_id = "ts" ; :variant_info = "N/A" ; :mip_era = "CMIP6" ; :source = "AM4p0(2017): atmosphere:AM4(AM4,C96xL33); land:LM4.0(LM4.0,C96xL20)" ; :parent_activity_id = "no parent" ; :parent_mip_era = "no parent" ; :parent_source_id = "no parent" ; :sub_experiment_id = "none" ; :grid = "atmos data regridded from Cubed-sphere (c96) to 360x180 longitude/latitude; interpolation method: conserve_order2" ; :variant_label = "r1i1p1f1" ; :grid_label = "gr" ; :license = "CMIP6 model data produced by GFDL is licensed under a Creative Commons Attribution ShareAlike 4.0 International License ( https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ; :nominal_resolution = "100 km" ; :product = "model-output" ; :realm = "atmos" ; :references = "???Need to get text from MIPs leaders???" ; } header-B netcdf ts_Amon_GFDL-AM4_amip_r1i1p1f1_gr_198001-201412 { dimensions: bnds = 2 ; lat = 180 ; lon = 288 ; time = UNLIMITED ; // (420 currently) variables: double bnds(bnds) ; bnds:long_name = "vertex number" ; double lat(lat) ; lat:long_name = "latitude" ; lat:units = "degrees_north" ; lat:bounds = "lat_bnds" ; lat:axis = "Y" ; lat:standard_name = "latitude" ; double lat_bnds(lat, bnds) ; double lon(lon) ; lon:long_name = "longitude" ; lon:units = "degrees_east" ; lon:bounds = "lon_bnds" ; lon:axis = "X" ; lon:standard_name = "longitude" ; double lon_bnds(lon, bnds) ; double time(time) ; time:long_name = "time" ; time:units = "days since 1979-01-01 00:00:00" ; time:axis = "T" ; time:calendar = "julian" ; time:bounds = "time_bnds" ; time:description = "for time-mean fields" ; time:standard_name = "time" ; double time_bnds(time, bnds) ; time_bnds:long_name = "time axis boundaries" ; time_bnds:units = "days since 1979-01-01 00:00:00" ; time_bnds:missing_value = 1.e+20 ; time_bnds:_FillValue = 1.e+20 ; float ts(time, lat, lon) ; ts:long_name = "Surface Temperature" ; ts:units = "K" ; ts:missing_value = 1.e+20f ; ts:cell_methods = "area: time: mean" ; ts:cell_measures = "area: areacella" ; ts:standard_name = "surface_temperature" ; ts:interp_method = "conserve_order2" ; ts:_FillValue = 1.e+20f ; ts:original_name = "ts" ; ts:associated_files = "baseURL: http://nomads.gfdl.noaa.gov/CM2.X/associated_files.html areacella: areacella_fx_GFDL-AM4_amip_r0i0p0.nc" ;

// global attributes: :title = "NOAA GFDL GFDL-AM4 model output prepared for CMIP6 AMIP" ; :history = "File was processed by fremetar (GFDL analog of CMOR). TripleID: [exper_id_tapHVfqhIr,realiz_id_NFMP2wmZCu,run_id_xjMXuHqaT2]" ; :contact = "gfdl.climate.model.info@noaa.gov" ; :comment = "Experiment name = c96L33_am4p0_cmip6Diag" ; :tracking_id = "hdl:21.14100/2b05f167-4eec-4bad-9c19-e241b945bc12" ; :further_info_url = " http://furtherinfo.es-doc.org/CMIP6.NOAA-GFDL.GFDL-AM4.amip.r1i1p1f1" ; :branch_method = "no parent" ; :branch_time_in_parent = 0. ; :creation_date = "2017-07-07T18:38:59Z" ; :Conventions = "CF-1.7 CMIP-6.0 UGRID-1.0" ; :table_id = "Amon" ; :sub_experiment = "none" ; :frequency = "mon" ; :forcing_index = 1 ; :physics_index = 1 ; :initialization_index = 1 ; :realization_index = 1 ; :parent_variant_label = "no parent" ; :parent_experiment_id = "no parent" ; :data_specs_version = "01.00.10" ; :external_variables = "area: areacella" ; :experiment_id = "amip" ; :experiment = "AMIP" ; :activity_id = "CMIP" ; :source_id = "GFDL-AM4" ; :source_type = "AGCM" ; :institution_id = "NOAA-GFDL" ; :institution = "National Oceanic and Atmospheric Administration, Geophysical Fluid Dynamics Laboratory, Princeton, NJ 08540, USA" ; :variable_id = "ts" ; :variant_info = "N/A" ; :mip_era = "CMIP6" ; :source = "AM4p0(2017): atmosphere:AM4(AM4,C96xL33); land:LM4.0(LM4.0,C96xL20)" ; :parent_activity_id = "no parent" ; :parent_mip_era = "no parent" ; :parent_source_id = "no parent" ; :sub_experiment_id = "none" ; :grid = "atmos data regridded from Cubed-sphere (c96) to 360x180 longitude/latitude; interpolation method: conserve_order2" ; :variant_label = "r1i1p1f1" ; :grid_label = "gr" ; :license = "CMIP6 model data produced by NOAA GFDL is licensed under a Creative Common Attribution \"Share Alike\" 4.0 International License ( http://creativecommons.org/licenses/by/4.0/). Use of the data should be acknowledged following guidelines found at https://pcmdi.llnl.gov/home/CMIP6/CitationRequirements6-0.html. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file). The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ; :nominal_resolution = "100 km" ; :product = "model-output" ; :realm = "atmos" ; :references = "???Need to get text from MIPs leaders???" ; }

Here's more info. I did this:

conda create -n [YOUR_ENV_NAME_HERE] -c conda-forge -c pcmdi -c uvcdat cmor

tail of the stdout:

C Traceback: In function:

!!!!!!!!!!!!!!!!!!!!!!!!! ! ! Warning: The further info in attribute does not match the one found in your Control Vocabulary(CV) File. ! We found " http://furtherinfo.es-doc.org/CMIP6.NOAA-GFDL.GFDL-AM4.c96L33_am4p0_cmip6Diag.r1i1p1f1" and ! CV requires " http://furtherinfo.es-doc.org/CMIP6.NOAA-GFDL.GFDL-AM4.amip.none.r1i1p1f1" ! Check your Control Vocabulary file "/home/a1r/CMIP6_work/Tables/CMIP6_CV.json". ! ! !!!!!!!!!!!!!!!!!!!!!!!!!

/home/pcmdi/.conda/envs/cmor/lib/python2.7/site-packages/cdms2/axis.py:762: UserWarning: genutil module not present, was not able to determine if axis is level based on units "genutil module not present, was not able to determine if axis is level based on units") CDMS error: Error on character time conversion, string = ?

CDMS error: Error on character time conversion, string = ?

CDMS error: Error on character time conversion, string = ?

CDMS error: Error on character time conversion, string = ?

CDMS error: Error on character time conversion, string = ?

!!!!!!!!!!!!!!!!!!!!!!!!! ! Error: The input file is not CMIP6 compliant ! Check your file or use CMOR 3.x to achieve compliance for ESGF publication. !!!!!!!!!!!!!!!!!!!!!!!!!

netCDF header:

Where should I expect to see genutil python module?

/home/pcmdi/.conda/envs/cmor/lib/python2.7/site-packages/cdms2/????

Here's what I get when I run: conda install -c conda-forge genutil

Fetching package metadata ........... Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:

On Fri, Sep 1, 2017 at 1:45 PM, Charles Doutriaux notifications@github.com wrote:

@aradhakrishnanGFDL https://github.com/aradhakrishnangfdl can you send us your netcdf file as well? maybe an ncdump -h would be nice if posted here. Thx!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PCMDI/cmor/issues/184#issuecomment-326642751, or mute the thread https://github.com/notifications/unsubscribe-auth/AZHW6l9dJQOsH4efJklsi1ZL2613nPETks5seELIgaJpZM4NwS64 .

-- Aparna Radhakrishnan Engility, NOAA/GFDL

Cell: 313-461-9948

doutriaux1 commented 7 years ago

@dnadeau4 i don't think days since ? is a valid unit. It will definitely trip cdms.

doutriaux1 commented 7 years ago

I think that's why you get the genutil warning, when everything else fails we try to detect axes type based on units. And for this we need genutil.

dnadeau4 commented 7 years ago

@aradhakrishnanGFDL The problem are your units. You did not use cmor and your units are set to "days since ?" CDMS cannot convert that units and failed. It has nothing to do with CMOR or PrePARE.

dnadeau4 commented 7 years ago

@aradhakrishnanGFDL maybe your environment is broken, I just tried a new one and I was able to install genutil.

But you should never have days since ? in your file and your variable time_bnds normally have no attributes.

conda install -c conda-forge genutil
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /export/nadeau1/anaconda2/envs/cmor3.2.6:

The following NEW packages will be INSTALLED:

    genutil: 2.12-np112py27_0 conda-forge

Proceed ([y]/n)? y

genutil-2.12-n 100% |###################################################################################################| Time: 0:00:00 366.78 kB/s
(cmor3.2.6) nadeau1@:~$ 
aradhakrishnanGFDL commented 7 years ago

Yes, agreed. Like I mentioned earlier we spotted the issue with thorough QC'ing here in the lab and the newer files used for test publication DO NOT have this issue.

If we had genutil, prepare may have given a better error message as well I am sure.

Regarding the environment, sure I'll check with our sys admin. Thanks,

On Fri, Sep 1, 2017 at 5:34 PM, Denis Nadeau notifications@github.com wrote:

@aradhakrishnanGFDL https://github.com/aradhakrishnangfdl maybe your environment is broken, I just tried a new one and I was able to install genutil.

But you should never have days since ? in your file and your variable time_bnds normally have no attributes.

conda install -c conda-forge genutil Fetching package metadata ........... Solving package specifications: .

Package plan for installation in environment /export/nadeau1/anaconda2/envs/cmor3.2.6:

The following NEW packages will be INSTALLED:

genutil: 2.12-np112py27_0 conda-forge

Proceed ([y]/n)? y

genutil-2.12-n 100% |###################################################################################################| Time: 0:00:00 366.78 kB/s (cmor3.2.6) nadeau1@crunchy:~$

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PCMDI/cmor/issues/184#issuecomment-326688666, or mute the thread https://github.com/notifications/unsubscribe-auth/AZHW6tSkTbLy-aioXXjYBUoIzp0piTy_ks5seHhTgaJpZM4NwS64 .

-- Aparna Radhakrishnan Engility, NOAA/GFDL

Cell: 313-461-9948

doutriaux1 commented 7 years ago

@dnadeau4 if I recall correctly back from my CMOR days, the time units in the tables are/were "days since ?" and are later replaced by CMOR. See: https://github.com/PCMDI/cmip5-cmor-tables/blob/master/Tables/CMIP5_Amon#L144 for example. It's probably where @aradhakrishnanGFDL got these units. @aradhakrishnanGFDL you should really use CMOR.

dnadeau4 commented 7 years ago

That is what I think too. the ? should be replace with a year like 1979. We cannot guess.

dnadeau4 commented 7 years ago

CDMS try to take the string "?" and convert it to a date and failed.

aradhakrishnanGFDL commented 7 years ago

Thanks Charles and Denis! - Have a good holiday weekend.

On Friday, September 1, 2017, Denis Nadeau notifications@github.com wrote:

CDMS try to take the string "?" and convert it to a date and failed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PCMDI/cmor/issues/184#issuecomment-326694247, or mute the thread https://github.com/notifications/unsubscribe-auth/AZHW6vQD2YstzUislro6GFDKzm1AdszZks5seIC_gaJpZM4NwS64 .

-- Aparna Radhakrishnan Engility, NOAA/GFDL

Cell: 313-461-9948

aradhakrishnanGFDL commented 7 years ago

Pinging again.

Which anaconda version are you using?Also, wondering if this was tested on anaconda3?

-Is there a way to get genutil package any other way other than conda-forge? (e.g in a standard python distribution..is that recommended?

doutriaux1 commented 7 years ago

cdms for python3 is barely out, so I'm 100% it was not tested on anaconda3, not sure why you can't get cmor and cdms2/genutil together? They're comaptible since 3.2.7, right @dnadeau4 ?

dnadeau4 commented 7 years ago

There is no plan to get CMOR on python 3. You can get genutil just by doing a conda install -c conda-forge genutil command.

aradhakrishnanGFDL commented 7 years ago

Hi- With anaconda3, although genutil package installation seemed successful, the usage of it at runtime imposed a few different runtime symbol errors (because of other package dependency e.g. openssl)

We just installed anaconda2 here at GFDL now followed by genutil and then CMOR. It's working fine this way. Thanks

doutriaux1 commented 7 years ago

yes there shouldn't even exist a genutil for python3 out there, I wonder where you got this one from. I'll check.

aradhakrishnanGFDL commented 7 years ago

A note to the CMOR installation notes to the anaconda2 software requirements might help.