search

barronh / pseudonetcdf

PseudoNetCDF like NetCDF except for many scientific format backends
GNU Lesser General Public License v3.0
77 stars 35 forks source link

How to simply convert CAMx output to NetCDF #10

Closed tamer540 closed 8 years ago

tamer540 commented 8 years ago

Example Form Tutorial Below is Work Fine for me,And Netcdf File was Created fine.

Dump an icartt file in CDL: pncdump -f ffi1001 HOX_DC8_20040626_R0.ict Create a netcdf from an icartt file: pncgen -f ffi1001 HOX_DC8_20040626_R0.ict HOX_DC8_20040626_R0.nc

But, with camx outpout from my run and also test case outpout for v6.3 at CAMx web sit http://www.camx.com/getmedia/24c71e1b-051f-4ae9-9301-6cbc93486283/CAMx6-30-test_run-outputs-Intel_compiler-160408.tgz i have the following error :-

My Command : >> pncdump -f ffi1001 CAMx.v6.30.midwest.36.12.noMPI.20020603.grd

My Error :- too many of pncdump help file and Meesage "pncdump.py: error: too few arguments" in End

what can i do to convert CAMx output to Netcdf format

barronh commented 8 years ago

CAMx files are typically in the uamiv format. So you should use "-f uamiv" instead of "-f ffi1001".

On May 8, 2016, at 5:56 PM, tamer540 notifications@github.com wrote:

Example Form Tutorial Below is Work Fine for me,And Netcdf File was Created fine.

Dump an icartt file in CDL: pncdump -f ffi1001 HOX_DC8_20040626_R0.ict Create a netcdf from an icartt file: pncgen -f ffi1001 HOX_DC8_20040626_R0.ict HOX_DC8_20040626_R0.nc

But, with camx outpout from my run and also test case outpout for v6.3 at CAMx web sit http://www.camx.com/getmedia/24c71e1b-051f-4ae9-9301-6cbc93486283/CAMx6-30-test_run-outputs-Intel_compiler-160408.tgz i have the following error :-

My Command : >> pncdump -f ffi1001 CAMx.v6.30.midwest.36.12.noMPI.20020603.grd

My Error :- too many of pncdump help file and Meesage "pncdump.py: error: too few arguments" in End

what can i do to convert CAMx output to Netcdf format

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

tamer540 commented 8 years ago

CAMx files are typically in the uamiv format. So you should use "-f uamiv" instead of "-f ffi1001".

On May 8, 2016, at 5:56 PM, tamer540 notifications@github.com wrote:

Example Form Tutorial Below is Work Fine for me,And Netcdf File was Created fine.

Dump an icartt file in CDL: pncdump -f ffi1001 HOX_DC8_20040626_R0.ict Create a netcdf from an icartt file: pncgen -f ffi1001 HOX_DC8_20040626_R0.ict HOX_DC8_20040626_R0.nc

But, with camx outpout from my run and also test case outpout for v6.3 at CAMx web sit http://www.camx.com/getmedia/24c71e1b-051f-4ae9-9301-6cbc93486283/CAMx6-30-test_run-outputs-Intel_compiler-160408.tgz i have the following error :-

My Command : >> pncdump -f ffi1001 CAMx.v6.30.midwest.36.12.noMPI.20020603.grd

My Error :- too many of pncdump help file and Meesage "pncdump.py: error: too few arguments" in End

what can i do to convert CAMx output to Netcdf format

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub ########################################################### pncdump -f uamiv CAMx.CAMx.v6.20.noMPI.ifortomp.20020603.depn.grd01 usage: pncdump.py [-h] [-f {MetaNetCDF, add_derived, arlpackedbit, bcon_profile, bpch, bpch2, cloud_rain, cloud_rain_center_time, cloud_rain_center_time_plus, cloud_rain_plus, csv, ctb_reader, ffi1001, finst, flightlogs, geos, height_pressure, height_pressure_center_time, height_pressure_center_time_plus, height_pressure_plus, humidity, humidity_center_time, icon_profile, ipr, irr, jtable, landuse, lateral_boundary, mrgaloft, net_reader, netcdf, newresolution, one3d, osat, point_source, reader, sum_reader, temperature, temperature_center_time, time_avg_new_unit, uamiv, uamiv_new, vertical_diffusivity, vertical_diffusivity_center_time, wind, wind_center_time_cell, window, MetaNetCDF.MetaNetCDF, MetaNetCDF.add_derived, MetaNetCDF.newresolution, MetaNetCDF.time_avg_new_unit, MetaNetCDF.window, aermodfiles.reader, cmaqfiles.jtable, geoschemfiles._diag_group, geoschemfiles.bpch, geoschemfiles.bpch2, geoschemfiles.flightlogs, geoschemfiles.geos, net_balance.ctb_reader, net_balance.mrgaloft, net_balance.net_reader, net_balance.sum_reader, noaafiles.arlpackedbit, textfiles.csv, cmaqfiles.profile.bcon_profile, cmaqfiles.profile.icon_profile, icarttfiles.ffi1001.ffi1001, camxfiles.cloud_rain.Memmap.cloud_rain, camxfiles.cloud_rain.Transforms.cloud_rain_center_time, camxfiles.cloud_rain.Transforms.cloud_rain_center_time_plus, camxfiles.cloud_rain.Transforms.cloud_rain_plus, camxfiles.finst.Memmap.finst, camxfiles.height_pressure.Memmap.height_pressure, camxfiles.height_pressure.Read.height_pressure, camxfiles.height_pressure.Transforms.height_pressure_center_time, camxfiles.height_pressure.Transforms.height_pressure_center_time_plus, camxfiles.height_pressure.Transforms.height_pressure_plus, camxfiles.humidity.Memmap.humidity, camxfiles.humidity.Read.humidity, camxfiles.humidity.Transforms.humidity_center_time, camxfiles.ipr.Memmap.ipr, camxfiles.ipr.Read.ipr, camxfiles.irr.Memmap.irr, camxfiles.irr.Read.irr, camxfiles.landuse.Memmap.landuse, camxfiles.lateral_boundary.Memmap.lateral_boundary, camxfiles.one3d.Memmap.one3d, camxfiles.one3d.Read.one3d, camxfiles.point_source.Memmap.point_source, camxfiles.point_source.Read.point_source, camxfiles.temperature.Memmap.temperature, camxfiles.temperature.Read.temperature, camxfiles.temperature.Transforms.temperature_center_time, camxfiles.uamiv.Memmap.uamiv, camxfiles.uamiv.Read.uamiv, camxfiles.uamiv.Read.uamiv_new, camxfiles.uamiv.Transforms.osat, camxfiles.vertical_diffusivity.Memmap.vertical_diffusivity, camxfiles.vertical_diffusivity.Read.vertical_diffusivity, camxfiles.vertical_diffusivity.Transforms.vertical_diffusivity_center_time, camxfiles.wind.Memmap.wind, camxfiles.wind.Read.wind, camxfiles.wind.Transforms.wind_center_time_cell, _diag_group}] [--help-format HELPFORMAT] [--sep SEPARATOR] [--inherit] [--mangle] [--remove-singleton REMOVESINGLETON] [-O] [--out-format {NETCDF3_CLASSIC,NETCDF4_CLASSIC,NETCDF4,bpch,cloud_rain,csv,ffi1001,height_pressure,humidity,landuse,lateral_boundary,one3d,point_source,temperature,uamiv,vertical_diffusivity,wind,camxfiles.cloud_rain,camxfiles.height_pressure,camxfiles.humidity,camxfiles.landuse,camxfiles.lateral_boundary,camxfiles.one3d,camxfiles.point_source,camxfiles.temperature,camxfiles.uamiv,camxfiles.vertical_diffusivity,camxfiles.wind}] [--mode {w,a,r+,ws,as,r+s}] [--verbose] [--dump-name CDLNAME] [--coordkeys key1,key2] [-v varname1[,varname2[,...,varnameN]] [-a att_nm,var_nm,mode,att_typ,att_val] [-m MASKS] [--from-convention FROMCONV] [--to-convention TOCONV] [--stack STACK] [--merge] [-s dim,start[,stop[,step]]] [-r dim,function[,weight]] [--mesh dim,weight,function] [-c dim,mode,wgt1,wgt2,...wgtN] [-e EXTRACT] [--extractmethod {nn,linear,cubic,quintic,KDTree}] [--op-typ OPERATORS] [--expr EXPRESSIONS] [--exprscript EXPRESSIONSCRIPTS] ifile [ifile ...] outpath pncdump.py: error: too few arguments

You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/barronh/pseudonetcdf/issues/10#issuecomment-217754400

barronh commented 8 years ago

Try adding a dummy argument. Is this on Windows or Mac?

tamer540 commented 8 years ago

CAMx files are typically in the uamiv format. So you should use "-f uamiv" instead of "-f ffi1001".

On May 8, 2016, at 5:56 PM, tamer540 notifications@github.com wrote:

Example Form Tutorial Below is Work Fine for me,And Netcdf File was Created fine.

Dump an icartt file in CDL: pncdump -f ffi1001 HOX_DC8_20040626_R0.ict Create a netcdf from an icartt file: pncgen -f ffi1001 HOX_DC8_20040626_R0.ict HOX_DC8_20040626_R0.nc

But, with camx outpout from my run and also test case outpout for v6.3 at CAMx web sit http://www.camx.com/getmedia/24c71e1b-051f-4ae9-9301-6cbc93486283/CAMx6-30-test_run-outputs-Intel_compiler-160408.tgz i have the following error :-

My Command : >> pncdump -f ffi1001 CAMx.v6.30.midwest.36.12.noMPI.20020603.grd

My Error :- too many of pncdump help file and Meesage "pncdump.py: error: too few arguments" in End

what can i do to convert CAMx output to Netcdf format

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

############################################################ after i convert data i can`t open it with GrADS grads -lc "sdfopen CAMx.CAMx.v6.20.noMPI.ifortomp.20020603.avrg.grd02.nc" I have the flowing message :-

gadsdf: SDF file has no discernable X coordinate. To open this file with GrADS, use a descriptor file with an XDEF entry. Documentation is at http://iges.org/grads/gadoc/SDFdescriptorfile.html

You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/barronh/pseudonetcdf/issues/10#issuecomment-217754400

tamer540 commented 8 years ago

Try adding a dummy argument. Is this on Windows or Mac?

You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/barronh/pseudonetcdf/issues/10#issuecomment-217871056

what do you mean about "a dummy argument" i use Fedora 20 Linux x64-86

barronh commented 8 years ago

Add the --from-conv=ioapi --to-conv=cf

This will add CF conventions

it looks like you editted your comment. the dummy argument is no longer relevant.

tamer540 commented 8 years ago

Add the --from-conv=ioapi --to-conv=cf

This will add CF conventions

it looks like you editted your comment. the dummy argument is no longer relevant.

You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/barronh/pseudonetcdf/issues/10#issuecomment-217872485 I am using nco to change Coordinate name:- ncrename -d LAY,level CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02.nc ncrename -d TSTEP,time CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02.nc ncrename -d ROW,latitude CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02.nc ncrename -d COL,longitude CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02.nc

and still GrADS message displyed

also i use your suggestion add CF conventions as following :-

pncdump -f uamiv --from-conv=ioapi --to-conv=cf CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02 pncgen --from-conv=ioapi CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02 --to-conv=cf CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02.nc

also still GrADS message displyed.

Copyright (C) 1995-2013 - All Rights Reserved University Corporation for Atmospheric Research NCAR Command Language Version 6.1.2 The use of this software is governed by a License Agreement. See http://www.ncl.ucar.edu/ for more details.

Variable: f Type: file filename: CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02 path: CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02.nc file global attributes: XORIG : -48000 YORIG : -912000 XCELL : 12000 YCELL : 12000 PLON : -97 PLAT : 40 TLAT1 : 45 TLAT2 : 33 IUTM : 0 ISTAG : 0 CPROJ : 2 GDTYP : 2 XCENT : -97 YCENT : 40 P_ALP : 45 P_BET : 33 P_GAM : -97 NLAYS : 1 NROWS : 113 NCOLS : 92 NVARS : 21 NSTEPS : 24 VAR-LIST : NO NO2 O3 SO2 H2O2 HNO3 NH3 PNO3 PSO4 PNH4 POA PEC FPRM CPRM CCRS FCRS SOA1 SOA2 SOA3 SOA4 SOA5
NAME : AVERAGE
NOTE : CAMx 6.30 Test Problem -- Mech2 CF CB6R2 v6.30.midwest.36.12 ITZON : 0 FTYPE : 1 VGTYP : 2 VGTOP : 10000 VGLVLS : ( 1, 0 ) GDNAM : CAMx
UPNAM : CAMx
FILEDESC : CAMx
SDATE : 2002155 STIME : 0 TSTEP : 10000 Conventions : CF-1.6 history : /usr/lib/python2.7/site-packages/PseudoNetCDF-3.0-py2.7.egg/PseudoNetCDF/pncgen.py -f uamiv --from-conv=ioapi CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02 --to-conv=cf CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02.nc;/usr/lib/python2.7/site-packages/PseudoNetCDF-3.0-py2.7.egg/PseudoNetCDF/pncgen.py -f uamiv --from-conv=ioapi CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02 --to-conv=cf CAMx.v6.30.midwest.36.12.noMPI.20020604.avrg.grd02.nc; dimensions: ncl_scalar = 1 LAY = 1 TSTEP = 24 // unlimited ROW = 113 COL = 92 tnv = 2 VAR = 21 DATE-TIME = 2 nv = 4 variables: double layer ( LAY ) units : model layers standard_name : layer

  float PSO4 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : PSO4                                                                            
     coordinates :  time level latitude longitude
     long_name :    PSO4            
     units :    micrograms/m**3

  float NO2 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : NO2                                                                             
     coordinates :  time level latitude longitude
     long_name :    NO2             
     units :    ppm

  float SOA5 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : SOA5                                                                            
     coordinates :  time level latitude longitude
     long_name :    SOA5            
     units :    micrograms/m**3

  float SOA4 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : SOA4                                                                            
     coordinates :  time level latitude longitude
     long_name :    SOA4            
     units :    micrograms/m**3

  float SOA1 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : SOA1                                                                            
     coordinates :  time level latitude longitude
     long_name :    SOA1            
     units :    micrograms/m**3

  float SOA3 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : SOA3                                                                            
     coordinates :  time level latitude longitude
     long_name :    SOA3            
     units :    micrograms/m**3

  float SOA2 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : SOA2                                                                            
     coordinates :  time level latitude longitude
     long_name :    SOA2            
     units :    micrograms/m**3

  float H2O2 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : H2O2                                                                            
     coordinates :  time level latitude longitude
     long_name :    H2O2            
     units :    ppm

  float NO ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : NO                                                                              
     coordinates :  time level latitude longitude
     long_name :    NO              
     units :    ppm

  float PNO3 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : PNO3                                                                            
     coordinates :  time level latitude longitude
     long_name :    PNO3            
     units :    micrograms/m**3

  float O3 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : O3                                                                              
     coordinates :  time level latitude longitude
     long_name :    O3              
     units :    ppm

  double time_bounds ( TSTEP, tnv )
     units :    seconds since 1970-01-01 00:00:00
     long_name :    synthesized time coordinate from SDATE, STIME, STEP global attributes

  float FPRM ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : FPRM                                                                            
     coordinates :  time level latitude longitude
     long_name :    FPRM            
     units :    micrograms/m**3

  integer TFLAG ( TSTEP, VAR, DATE-TIME )
     units :    DATE-TIME
     long_name :    TFLAG           
     var_desc : TFLAG                                                                           

  double latitude ( ROW, COL )
     units :    degrees_north
     standard_name :    latitude
     bounds :   latitude_bounds
     coordinates :  latitude longitude

  float PEC ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : PEC                                                                             
     coordinates :  time level latitude longitude
     long_name :    PEC             
     units :    micrograms/m**3

  double latitude_bounds ( ROW, COL, nv )
     units :    degrees_north
     standard_name :    latitude_bounds
     grid_mapping : LambertConformalProjection
     coordinates :  latitude longitude nv

  float CPRM ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : CPRM                                                                            
     coordinates :  time level latitude longitude
     long_name :    CPRM            
     units :    micrograms/m**3

  integer ETFLAG ( TSTEP, VAR, DATE-TIME )
     units :    DATE-TIME
     long_name :    ETFLAG          
     var_desc : Ending TFLAG                                                                    

  float HNO3 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : HNO3                                                                            
     coordinates :  time level latitude longitude
     long_name :    HNO3            
     units :    ppm

  float SO2 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : SO2                                                                             
     coordinates :  time level latitude longitude
     long_name :    SO2             
     units :    ppm

  float NH3 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : NH3                                                                             
     coordinates :  time level latitude longitude
     long_name :    NH3             
     units :    ppm

  double longitude_bounds ( ROW, COL, nv )
     units :    degrees_east
     standard_name :    longitude_bounds
     grid_mapping : LambertConformalProjection
     coordinates :  latitude longitude nv

  float PNH4 ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : PNH4                                                                            
     coordinates :  time level latitude longitude
     long_name :    PNH4            
     units :    micrograms/m**3

  double level ( LAY )
     units :    sigma
     positive : down
     standard_name :    level

  integer LambertConformalProjection ( ncl_scalar )
     grid_mapping_name :    lambert_conformal_conic
     standard_parallel :    ( 45, 33 )
     semi_major_axis :  6370000
     semi_minor_axis :  6370000
     latitude_of_projection_origin :    40
     longitude_of_central_meridian :    -97

  double longitude ( ROW, COL )
     units :    degrees_east
     standard_name :    longitude
     bounds :   longitude_bounds
     coordinates :  latitude longitude

  float CCRS ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : CCRS                                                                            
     coordinates :  time level latitude longitude
     long_name :    CCRS            
     units :    micrograms/m**3

  float FCRS ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : FCRS                                                                            
     coordinates :  time level latitude longitude
     long_name :    FCRS            
     units :    micrograms/m**3

  double time ( TSTEP )
     units :    seconds since 1970-01-01 00:00:00
     long_name :    synthesized time coordinate from SDATE, STIME, STEP global attributes
     bounds :   time_bounds

  double y ( ROW )
     units :    km
     long_name :    synthesized coordinate from YORIG YCELL global attributes

  double x ( COL )
     units :    km
     long_name :    synthesized coordinate from XORIG XCELL global attributes

  float POA ( TSTEP, LAY, ROW, COL )
     grid_mapping : LambertConformalProjection
     var_desc : POA                                                                             
     coordinates :  time level latitude longitude
     long_name :    POA             
     units :    micrograms/m**3
barronh commented 8 years ago

The netcdf file is readable by many CF compliant packages. I did some searching and it looks like you need to switch from sdfopen to xdfopen with a defining file. Below is an example set file if you were using making a test.nc file from the CAMx tutorial. I have not been able to test this because my version of grads does not accept the pref keyword and I am not a grads user.

$ pncgen -f uamiv -v O3 --from-conv=ioapi CAMx.v6.00.midwest.36.12.noMPI.20020603.avrg.grd02 test.nc
$ echo "dset ^test.nc
title This is only a test
dtype netcdf
UNDEF -9999.0 missing_value
PDEF 92 113 lcc 40 -97 4 76 33 45 -97 12000 12000
xdef 92 LINEAR -97.6 0.15
ydef 113 LINEAR 31 0.15
zdef 1 LEVELS 1000
tdef 24 LINEAR 0Z03JUL2002 1HR
VARS 1
O3=>ozone 1 t,z,y,x Ozone
ENDVARS" > test.dset
$ grads -lc "open test.dset"