barronh / pseudonetcdf

PseudoNetCDF like NetCDF except for many scientific format backends
GNU Lesser General Public License v3.0
76 stars 35 forks source link

Errors reading WRFCAMx Kv #87

Closed tanksnrAQ closed 4 years ago

tanksnrAQ commented 4 years ago

Hi, I'm having trouble converting WRFCAMx (v4.7) Kv files to NetCDF using pncgen. The WRFCAMx output is for a domain nx=164,ny=143,nz=20. The output has 25 time steps in it.

I use the command: pncgen -f camxfiles.vertical_diffusivity.Memmap.vertical_diffusivity,rows=143,cols=164 ${infile} ${outfile}

I'm using Pseudonetcdf 3.1.0. The error I get is

Traceback (most recent call last): File "/home/mnaidoo/scratch/software/pythons/pseudonetcdf/PseudoNetCDF/pncparse.py", line 895, in getfiles f = pncopen(ipath, format=file_format, *format_options) File "/home/mnaidoo/scratch/software/pythons/pseudonetcdf/PseudoNetCDF/_getreader.py", line 130, in pncopen outfile = reader(args, **kwds) File "/home/mnaidoo/scratch/software/pythons/pseudonetcdf/PseudoNetCDF/camxfiles/one3d/Memmap.py", line 104, in init self.records, self.record_items)[:, 1:3]) ValueError: cannot reshape array of size 11732763 into shape (500,23456)

I checked and line 104 in the one3d Memmap code relates to time:

   time_date = array(self.__memmap.reshape(
        self.__records, self.__record_items)[:, 1:3])

The array size of 11732763 is close to what is expected, i.e. 11726000 (nx ny nz * 25hours). Not sure what's happening?

barronh commented 4 years ago

1) If you test the file without providing row/col commands, does it succeed? If so, what is the shape? 2) Can you post a file for me to test?

tanksnrAQ commented 4 years ago

Thanks for having a look Barron.

1) It does not succeed. The ValueError is the same. 2) Ok attached here. wrfcamx.kv.2K.20160601.bin.zip

I did notice a warning while running pncgen; not sure if it's important.

/apps/chpc/bio/anaconda3/lib/python3.6/runpy.py:125: RuntimeWarning: 'PseudoNetCDF.pncgen' found in sys.modules after import o f package 'PseudoNetCDF', but prior to execution of 'PseudoNetCDF.pncgen'; this may result in unpredictable behaviour warn(RuntimeWarning(msg))

I installed PseudoNetCDF using the system wide Anaconda, but installed to my local directories.

barronh commented 4 years ago

The file you uploaded is not an old-style kv file. In the newer versions of CAMx, the uamiv gridded format is used for all files. The command below should work.

pncgen -f uamiv ${infile} ${outfile}
tanksnrAQ commented 4 years ago

Great, that works. Sorry didn't know about the version differences.