sodri126 / netcdf4-python

Automatically exported from code.google.com/p/netcdf4-python
Other
0 stars 0 forks source link

A "--nstop" option for "nc3tonc4" would be super useful #148

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
It is often useful to save a few records of an OPeNDAP URL as a local NetCDF 
file.  If "nc3tonc4" had a "--nstop" option to indicated at which record to 
stop at, it could accomplish this task nicely.

Right now it writes the entire file (or DAP dataset), which can be several GB 
to 33TB for our ocean model data! 

I'm thinking that this could be called together with chunk, so if you wanted 
just the 1st two records, you could call:

nc3tonc4 --nstop=1 --chunk=1 
http://testbedapps-dev.sura.org/thredds/dodsC/auto/in.usf.fvcom.ike.ultralite.va
rdrag.nowave.2d  testout.nc

Jeff, I tried hacking this and failed.  Figure you could do it in 15 minutes.

-Rich

Original issue reported on code.google.com by rsignell on 20 Nov 2012 at 3:19

GoogleCodeExporter commented 9 years ago
Sorry, this should have been an enhancement request, not a defect.   I can't 
figure out how to change it.

Original comment by rsignell on 20 Nov 2012 at 3:21

GoogleCodeExporter commented 9 years ago
Rich, how would this work?  Wouldn't you have to specify the dimension and/or 
the variables you want the value of nstop to apply to?

Original comment by whitaker.jeffrey@gmail.com on 21 Nov 2012 at 3:53

GoogleCodeExporter commented 9 years ago
The way I was envisioning it, it would only work with unlimited dimension, and 
would apply to all variables. 

Original comment by rsignell on 27 Nov 2012 at 11:00

GoogleCodeExporter commented 9 years ago
I just came another use for this that would be super useful to have this.  I 
need to convert netcdf3 to netcdf4 for a bunch of monthly files that start at 
0000 and end at 2400, thus there are duplicate time values when aggregated.   
If I could specify --nstop=-2 (skip the last value) I could just use the 
existing nc3tonc4 with no additional processing!

Does this make sense to you Jeff?

-Rich

Original comment by rsignell on 30 Nov 2012 at 7:05

GoogleCodeExporter commented 9 years ago
I went ahead and made this change (attached), adding a --istart and --istop 
option to control which records get output.  It seems to work.  I'm attaching 
it here because I don't know of a better way to contribute the changes (can't 
send a pull request).

Original comment by rsignell on 12 Dec 2012 at 3:24

Attachments:

GoogleCodeExporter commented 9 years ago
I'm adding some more stuff (just added a "recdim" argument to specify a 
dimension to use as the record dimension when there is no unlimited dimension), 
and adding a "banlist" of variables *not* to copy.

Original comment by rsignell on 12 Dec 2012 at 5:45

GoogleCodeExporter commented 9 years ago
Thanks Rich - I've committed your new version.  

Maybe I should switch to github...

Original comment by whitaker.jeffrey@gmail.com on 13 Dec 2012 at 1:32

GoogleCodeExporter commented 9 years ago

Original comment by whitaker.jeffrey@gmail.com on 7 Mar 2013 at 11:42