search

ClimateImpactLab / DataFS

An abstraction layer for data storage systems
MIT License
6 stars 2 forks source link

Update netcdf4 to 1.4.0 #561

Closed pyup-bot closed 4 years ago

pyup-bot commented 6 years ago

This PR updates netCDF4 from 1.3.1 to 1.4.0.

Changelog ### 1.4.0 ``` ============================= * fixed bug in detection of CDF5 library support in setup.py (pull request 736, issue 713). * fixed reading of variables with zero-length dimensions in NETCDF3_CLASSIC files (issue 743). * allow integer-like objects in VLEN slices (not just python ints, issue 526, pull request 757). * treating _FillValue as a valid_min/valid_max was too surprising, despite the fact the thet netcdf docs 'attribute best practices' suggests that clients should to this. Revert this change from issue 576 (issue 761). * remove netcdftime, since it is now a separate package. date2num, num2date and date2index still importable from netCDF4. * fix 'Unreachable code' cython warning (issue 767). * Change behavior of string attributes so that nc.stringatt = ['foo','bar'] produces an vlen string array attribute in NETCDF4, instead of concatenating into a single string ('foobar'). In NETCDF3/NETCDF4_CLASSIC, an IOError is now raised, instead of writing 'foobar'. Issue 770. * fix loading of enum type names (issue 775). * make sure missing_value applies only to scaled short integers if auto-scaling is on (issue 777). * automatically create views of compound types with character arrays as numpy strings (issue 773). Can be disabled using 'set_auto_chartostring(False)'. Numpy structured array dtypes with 'SN' string subtypes can now be used to define netcdf compound types (they get converted to ('S1',N) character array types automatically). * always return masked array by default, even if there are no masked values (too surprising to get ndarray or MaskedArray depending on slice, issue 785). * treat valid_min/valid_max/_FillValue/missing_value as unsigned integers if _Unsigned is set (to mimic behaviour of netcdf-java). Conversion to unsigned type now occurs before masking and scale/offset operation. Issue 794. version 1.3.1 (tag v1.3.1rel) ============================= * add parallel IO capabilities. netcdf-c and hdf5 must be compiled with MPI support, and mpi4py must be installed. To open a file for parallel access, use `parallel=True` in `Dataset.__init__` and optionally pass the mpi4py Comm instance using the `comm` kwarg and the mpi4py Info instance using the `info` kwarg. IO can be toggled between collective and independent using `Variable.set_collective`. See `examples/mpi_example.py`. Issue 717, pull request 716. Minimum cython dependency bumped from 0.19 to 0.21. * Add optional `MFTime` calendar overload to use across all files, for example, `'standard'` or `'gregorian'`. If `None` (the default), check that the calendar attribute is present on each variable and values are unique across files raising a `ValueError` otherwise. * Allow _FillValue to be set for vlen string variables (issue 730). version 1.3.0 (tag v1.3.0rel) ============================== * always search for HDF5 headers when building, even when nc-config is used (since nc-config does not always include the path to the HDF5 headers). Also use H5get_libversion to obtain HDF5 version info instead of H5public.h. Fixes issue 677. * encoding kwarg added to Dataset.__init__ and Dataset.filepath (default is to use sys.getfilesystemencoding()) so that oddball encodings (such as cp1252 on windows) can be handled in Dataset filepaths (issue 686). * Calls to nc_get_vars are avoided, since nc_get_vars is very slow (issue 680). Strided slices are now converted to multiple calls to nc_get_vara. This speeds up strided slice reads by a factor of 10-100 (especially for NETCDF4/HDF5 files) in most cases. In some cases, strided reads using nc_get_vars are faster (e.g. strided reads over many dimensions such as var[:,::2,::2,::2])), so a variable method use_nc_get_vars was added. var.use_nc_get_vars(True) will tell the library to use nc_get_vars instead of multiple calls to nc_get_vara, which was the default behaviour previous to this change. * fix utc offset time zone conversion in netcdftime - it was being done exactly backwards (issue 685 - thanks to pgamez and mdecker). * Fix error message for illegal ellipsis slicing, add test (issue 701). * Improve timezone format parsing in netcdftime (https://github.com/Unidata/netcdftime/issues/17). * make sure numpy datatypes used to define CompoundTypes have isalignedstruct flag set to True (issue 705), otherwise. segfaults can occur. Fix required raising them minimum numpy requirement from 1.7.0 to 1.9.0. * ignore missing_value, _FillValue, valid_range, valid_min and valid_max when creating masked arrays if attribute cannot be safely cast to variable data type (and issue a warning). When setting these attributes don't cast to variable dtype unless it can be done safely and issue a warning. Issue 707. version 1.2.9 (tag v1.2.9rel) ============================== * Fix for auto scaling and masking when _Unsigned attribute set (create view as unsigned type after scaling and masking). Issue 671. * Always mask values outside valid_min, valid_max (not just when missing_value attribue present). Issue 672. * Fix setup.py so pip install doesn't fail if cython not installed. setuptools >= 18.0 now required for installation (Issue 666). version 1.2.8 (tag v1.2.8rel) ============================== * recognize _Unsigned attribute used by netcdf-java to designate unsigned integer data stored with a signed integer type in netcdf-3 (issue 656). * add Dataset init memory parameter to allow loading a file from memory (pull request 652, issues 406 and 295). * fix for negative times in num2date (issue 659). * fix for failing tests in numpy 1.13 due to changes in numpy.ma (issue 662). * Checking for _Encoding attribute for NC_STRING variables, otherwise use 'utf-8'. 'utf-8' is used everywhere else, 'default_encoding' global module variable is no longer used. getncattr method now takes optional kwarg 'encoding' (default 'utf-8') so encoding of attributes can be specified if desired. If _Encoding is specified for an NC_CHAR ('S1') variable, the chartostring utility function is used to convert the array of characters to an array of strings with one less dimension (the last dimension is interpreted as the length of each string) when reading the data. When writing the data, stringtochar is used to convert a numpy array of fixed length strings to an array of characters with one more dimension. chartostring and stringtochar now also have an 'encoding' kwarg. Automatic conversion to/from character to string arrays can be turned off via a new set_auto_chartostring Dataset and Variable method (default is True). Addresses issue 654. * Cython >= 0.19 now required, _netCDF4.c and _netcdftime.c removed from repository. version 1.2.7 (tag v1.2.7rel) ============================== * fix for issue 624 (error in conversion to masked array when variable slice returns a scalar). This is a regression introduced in 1.2.5 associated with support for vector missing_values. Test (tst_masked5.py) added for vector missing_values. * fix for python 3.6 compatibility (error retrieving character _FillValue attribute, issue 626). Test with python 3.6 using travis CI. version 1.2.6 (tag v1.2.6rel) ============================== * fix some test failures on big endian PPC64 that were due to errors in byte-swapping logic. Also fixed bug in enum code exposed on PPC64 (issue 608). * remove support for python 2.6 (it probably still will work for a while though). * Sometimes checking that data being assigned to a variable has an 'ndim' attribute is not sufficient, instead check to see that the object supports the buffer interface (issue 613). * make get_variables_by_attributes work in MFDataset (issue 610) The hack is also applied for set_auto_maskandscale, set_auto_scale, set_automask, so these don't have to be duplicated in MFDataset (pull request 571). version 1.2.5 (tag v1.2.5rel) ============================== * Add MFDataset.set_auto_maskandscale (plus set_auto_scale, set_auto_mask). Fixes issue 570. * Use valid_min/valid_max/valid_range attributes when defining mask (issue 576). Values outside the valid range are considered to be missing when defining the mask. * Fix for issue 584 (add support for dates before -4712-1-1 in 360_day and 365_day calendars to netcdftime.utime). * Fix for issue 593: add support for datetime.timedelta operations (adding and subtracting timedelta, subtracting two datetime instances to compute time duration between them), implement datetime.replace() and datetime.__str__(). datetime.__repr__() includes the full state of an instance. Add datetime.calendar. datetime comparison operators have full accuracy now. * Fix for issue 585 by increasing the size of the buffer used to store the filepath. * Fix for issue 592: Add support for string array attributes. (When reading, a vlen string array attribute is returned as a list of strings. To write, use var.setncattr_string("name", ["two", "strings"]).) * Fix for issue 596 - julian day calculations wrong for negative years, caused incorrect rountrip num2date(date2num(date)) roundtrip for dates with year < 0. * Make sure negative years work in utime.num2date (issue 596). * raise NotImplementedError when trying to pickle Dataset, Variable, CompoundType, VLType, EnumType and MFDataset (issue 602). * Fix for issue 527: initialize vldata[i].p in Variable._get(...). version 1.2.4 (tag v1.2.4rel) ============================== * Fix for issue 554. It is now ensured that data is in native endian byte order before passing to netcdf-c library. Data read from variable with non-native byte order is also byte-swapped, so that dtype remains consistent with netcdf variable. Behavior now consistent with h5py. * raise warning for HDF5 1.10.x (issue 549), since backwards incompatible files may be created. * raise AttributeError instead of RuntimeError when attribute operation fails. raise IOError instead of RuntimeError when nc_create or nc_open fails (issue 546). * Use NamedTemporaryFile instead of deprecated mktemp in tests (pull request 543). * add AppVeyor automated windows tests (pull request 540). version 1.2.3.1 (tag v1.2.3.1rel) ================================== * fix bug in setup.py (pull request 539, introduced in issue 518). version 1.2.3 (tag v1.2.3rel) ============================== * try to avoid writing NC_STRING attributes if possible, by trying to convert unicode strings to ascii and write as NC_CHAR (issue 529). This preserves compatibility with clients (like Matlab) that can't deal with NC_STRING attributes. A 'setncattr_string' method was added for Dataset and Variable to that users can force attributes to be written as NC_STRING if necessary. * fix failing tests with numpy 1.11 (issues 521 and 522). * fix indentation bug in nc4tonc3 utility (issue 519). * add the capability in setup.py to use pkg-config instead of nc-config (pull request 518). * make sure slices which return scalar masked arrays are consistent with numpy.ma (issue 515). * add test/tst_cdf5.py and test/tst_filepath.py (to test new NETCDF3_64BIT_DATA format and filepath Dataset method). * expose netcdftime.__version__ (issue 504). * fix potential memory leak in Dataset.filepath in attempt to fix mysterious segfaults on CentOS6 (issue 506). Segfaults can apparently still occur on systems like CentOS6 with old versions of glibc. version 1.2.2 (tag v1.2.2rel) ============================= * fix failing tests on python 2.6 (issue 497). Change minimum required python from 2.5 to 2.6. * Potential memory leaks fixed by freeing string pointers internally allocated in netcdf-c using nc_free_string. Also use nc_free_vlens to free space allocated for vlens inside netcdf-c (issue 495). * invoke str on filename argument to Dataset constructor, so pathlib instances can be used (issue 489). * don't use hardwired NC_MAX_DIMS or NC_MAX_VARS internally to allocate space for dimension or variable ids. Instead, find out the number of dims and vars and use malloc. NC_MAX_NAME is still used to allocate space for attribute and variable names, since there is no obvious way to determine the length of these names. * if trying to write a unicode attribute, check to see if it exists first and is NC_CHAR, and if so, delete it and recreate it. Workaround for C lib bug discovered in issue 485. * support for NETCDF3_64BIT_DATA format supported in netcdf-c 4.4.0. Similar to NETCDF3_64BIT (now NETCDF3_64BIT_OFFSET), but includes 64 bit dimensions and sizes, plus unsigned and 64 bit integer data types. * make sure chunksize does not exceed dimension size (for non-unlimited dimensions) on variable creation (issue 480). * add 'size' attribute to Dimension (same as len(d), where d is a Dimension instance, issue 477). * fix bug in nc3tonc4 with --unpackshort=1 (issue 474). * dates do not have to be contiguous, i.e. can be before and after the missing dates in Gregorian calendar (pull request 476). version 1.2.1 (tag v1.2.1rel) ============================= * add the capability to slice variables with unsorted integer sequences, or integer sequences with duplicates (issue 467). This was done by converting boolean array slices to integer array slices internally, instead of the other way around. * raise TypeError if masked array assigned to a VLEN str variable slice (issue 464). * Ellipsis now can be used with scalar VLEN str variables (issue 458). Slicing of scalar VLEN (non-str) variables now works. * Allow non-positive reference years in non-real-world calendars (issue 442). version 1.2.0 (tag v1.2.0rel) ============================= * Fixes to setup.py for building on windows (issue 460). * warnings now issued if file being read contains unsupported variables or data types (they were previously being silently skipped). * added 'get_variables_by_attributes' method (issue 454). * check for 'units' attribute in date2index (issue 453). * added support for enum types (issue 452). * added 'isopen' Dataset method (issue 450). * raise ValueError if year 0 or negative year used in time units string. The year 0 does not exist in the Julian and Gregorian calendars (issue 442). version 1.1.9 (tag v1.1.9rel) ============================= * fix for issue 391 (data is already byte-swapped to native endian format by the HDF4 library). * fix for issue 415 (copy.deepcopy does not work on netcdftime datetime object). * fix for issue 420 - len(v) where v is a scalar variable returned unexpected IndexError, now returns "TypeError: len() on unsized object" (same as numpy does for len() on a scalar array). * translate docstrings from epydoc markup to markdown, so pdoc can be used (epydoc is dead). * add small offset in conversion to Julian date for numerical stability (more accurate round trip calculations). This offset is removed in back conversion only from microseconds. Pull request 433. * add detection of unsigned integers to handling of automatic packing (set_auto_scale and set_auto_maskandscale) when writing. Pull request 435. * use USE_SETUPCFG env var to over-ride use of setup.cfg. If USE_SETUPCFG evaluates to false, setup.cfg will not be used and all configuration variables can be set from environment variables. Useful when using 'pip install' and nc-config is broken (issue 438). * fix for integer overflow in date2index (issue 444). version 1.1.8 (tag v1.1.8rel) ============================= * v[...] now returns a numpy scalar array (not just a scalar) when v is a scalar netcdf variable (issue 413). * unix-like paths can now be used in createVariable and createGroup. v = nc.createVariable('/path/to/var1',('xdim','ydim'),float) will create a Variable named 'var1', while also creating the Groups 'path' and 'path/to' if they do not already exist. Similarly, g = nc.createGroup('/path/to') acts like 'mkdir -p' in unix, creating the Groups 'path' and '/path/to', if they don't already exist. Users who relied on nc.createGroup(groupname) failing when the group already exists will have to modify their code, since nc.createGroup will now return the existing group instance. Dataset.__getitem__ also added. nc['/path/to'] returns a Group instance, and nc['/path/to/var1'] returns a Variable instance. * change minimum required numpy to 1.7.0, fix so all tests pass with 1.7.0. Added travis tests for minimum required cython, numpy (issue 404). * enable abbreviations to time units specification, as allowed in CF (issue 402). Now, instead of just 'seconds' and 'seconds', 'secs', 'sec' and 's' are also allowed (similar to minutes, days and hours). * install utility scripts in utils directory with setuptools entry points (pull request 392 from mindw). Code for utilities moved to netCDF4_utils.py - makes utilities more windows-friendly. * make sure booleans are treated correctly in setup.cfg. Add use_cython (default True) to setup.cfg. If set to False, then cython will not be used to compile netCDF4.pyx (existing netCDF4.c will be used instead). * use "from Cython.Build import cythonize" instead of "from Cython.Distutils import build_ext" in setup.py (issue 393) to conform to new cython build mechanism (CEP 201, described at https://github.com/cython/cython/wiki/enhancements-distutils_preprocessing). * unicode attributes now written as strings, not bytes (using nc_put_att_string instead of nc_put_att_text, issue 388). * add __orthogonal_indexing__ attribute to Variable, Dataset and Group (issue 385) to denote that Variable objects do not follow numpy indexing semantics for integer and boolean array indices. * make sure application of scale_factor and add_offset works correctly when scale_factor not given (issue 381). * add man pages for nc3tonc4, nc4tonc3, ncinfo in man directory. Not installed by setup.py (contributed by Ross Gammon, issue 383). * replace tabs with spaces by running reindent.py on all *.py and *.pyx files (issue 378). * refactor netCDF4_utils and netCDF4 module into netCDF4 package. Refactoring effectively removes netCDF4 utils private attributes from netCDF4 namespace, so has the potential to break code using private attributes (issue 409). version 1.1.7 (tag v1.1.7rel) ============================= * check to make sure cython >= 0.19 is available before trying to use it (otherwise compilation with fail). Issue 367. * add ipython notebooks from Unidata workshop in examples directory. * fix ellipsis variable slicing regression (issue 371). * release the Global Interpreter Lock (GIL) when calling the C library for read operations. Speeds up multi-threaded reads (issue 369). Caution - the HDF5 library may need to be compiled with the threadsafe option to ensure that global data structures are not corrupted by simultaneous manipulation by different threads. * Make sure USE_NCCONFIG environment variable takes precedence over value of use_ncconfig in setup.cfg. With this change, 'pip install netCDF4' with USE_NCCONFIG=1 will use environment variables to find paths to libraries and include files, instead of relying on nc-config (issue 341). version 1.1.6 (tag v1.1.6rel) ============================= * fix for issue 353 (num2date can no longer handle units like 'hours since 2000-01-01 0'). * fix for issue 354 (num2date no longer supports multi-dimensional arrays). * fix for spurious UserWarning about endian-ness mismatch (issue 364). * make calendar name keyword for num2date/date2num case insensitive (issue 362). * make sure units parser returns time-zone naive datetime instance that includes UTC offset (issue 357). UTC offset was applied incorrectly in netcdftime.date2num and num2date. No longer need to depend on python-dateutil. version 1.1.5 (tag v1.1.5rel) ============================= * add dependency on python-dateutil in setup.py and install docs. * use python datetime in num2date and date2num whenever possible. Remove duplicate num2date and date2num functions from netcdftime. Addresses issue 344. Add microsecond capability to netcdftime.datetime. Roundtrip accuracy of num2date/date2num now down to less than a millisecond. * use nc-config by default to find dependencies. setup.py modified to handle failure to find nc-config more gracefully (issue 340). If you wish to use env vars to point to the libs, you must first move the setup.cfg file out of the way (rename it to setup.cfg.save), or set USE_NCCONFIG to 0. * if endian-ness of variable is specified, adjust datatype to reflect this when opening a file (issue 346). * fix for issue 349 (seconds outside the range 0-59 in netcdftime.num2date). version 1.1.4 (tag v1.1.4rel) ============================= * speedup conversion of array indices to slices (issue 325). * fix for issue 330 (incorrect values for seconds returned by netcdftime). * fix reading of scalar vlen variables (issue 333). * setting fill_value=False in createVariable for vlen and compound variables now does nothing, instead of causing an error when the Dataset is closed (issue 331). * cython will regenerate netCDF4.c when install is run, not just build. Makes 'pip install' do the right thing when cython is installed (issue 263). version 1.1.3 (tag v1.1.3rel) ============================= * checked in _datetime.c to git (resolves issue 315). Note - _datetime.c was *not* included in the 1.1.2 release. * Changed __str__ to __repr__ in MFDataset, to be consistent with Dataset (issue 317). IPython uses __repr__ to make use-friendly human-readable summaries of objects in the terminal. version 1.1.2 (tag v1.1.2rel) ============================= * fix for issue 312 (allow slicing with objects that can be cast to ints). * indexing netCDF variables with integer sequences and boolean arrays now behave the same way (integer sequences are converted to boolean arrays internally). Addresses issue 300. Since indexing using integer sequences does not behave exactly as before, some client code may break. For example, previously when integer index arrays had the same length, and that length was equal to the number of dimensions of the array being indexed, netcdf4-python mirrored the numpy indexing behavior and treated the elements of the index arrays as individual sets of integer indices. This special case has been removed. An IndexError is now raised when the new behavior would produce a different result than the old, i.e. when the indices in an integer sequence are not sorted, or there are duplicate indices in the sequence. * fix for issue 310 (masked arrays not returned correctly when variable has non native endian-ness). * fix for issue 306 (slicing variable with "-1" when there is only one element along that dimension). * Improved speed of num2date and date2num for standard, julian, gregorian and proleptic gregorian calendars by vectorizing the functions. See Issue 296 * Fix for issue 301 ("Datestring parser chokes on years with extra space"). * Add name property for Dimension, Variable and Group instances (to access string name associated with instance). * Allow for null byte attributes (so _FillValue='\x00' can be set manually). Issue 273. * Added __repr__ (matching __str__) for all types (pull request 291). IPython uses __repr__ to make use-friendly human-readable summaries of objects in the terminal. version 1.1.1 (tag v1.1.1rel) ============================== * make sure _FillValue is a byte for character arrays in Python 3 (issue 271). * add numpy to install_requires in setup.py (issue 282, fixes issue 211). 'pip install netcdf4-python' will no longer fail if numpy not installed. * Fix for issue 278 (UnicodeDecodeError reading netcdf.h from setup.py with Python 3.4). * Make netcdftime.datetime immutable and hashable (issue 255). * Fix issue with slicing of scalar VLEN arrays (issue 270). * Add set_auto_mask and set_auto_scale methods to control auto scaling and auto masking separately. (issue 269). Also added set_auto_maskandscale, set_auto_scale, set_auto_mask Dataset/Group methods that recursively walk through all variables in the Dataset/Group. * Make sure file_format attribute propagated to Group instances (issue 265). * Fix for issue 259 ("Cannot use broadcasting to set all elements of a Variable to a given value"). version 1.1.0 (tag v1.1.0rel) ============================= * revert weakref change, so that previous behaviour (Dimensions and Variables keep strong references to parent Dataset) is the default. New keyword argument 'keepweakref' for Dataset.__init__ can be set to true to get weak references. version 1.0.9 (tag v1.0.9rel) ============================= * speed up the creation of new Group instances (issue 239). * fix logic errors in setup.py (issue 236). * it is now possible to create and set variable length string variables with numpy string datatypes (pull request 224). * add .travis.yml (for travis-ci testing on github), silence warnings from test output (issue 225). * modify __unicode__ for Variable and Dimension to return more useful error message when Dataset object has been garbage collected. * use weak references to group instances when creating Dimension and Variable objects. This prevents cyclic references messing up garbage collection (issue 218, pull request 219). * accessing values from a 0-dimensional Variable now returns a 0-dimensional numpy array, not a 1-dimensional array (issue 220). To write code compatible with both the old and new (fixed) behavior, wrap values accessed from a 0-dimensional Variable with numpy.asscalar. * add an __array__ method to Variable to make numpy ufuncs faster (issue 216). * change download_url in setup.py to point to pypi instead of googlecode. * fix for date2index error when time variable has only one entry (issue 215). * silence warnings ("Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line") with Cython 0.2. * reduced memory usage for Variable.__getitem__ under Python 2. version 1.0.8 (tag v1.0.8rel) ============================= * change file_format Dataset attribute to data_model (keeping file_format for backward compatibility). Add disk_format attribute (underlying disk format, one of NETCDF3, HDF4, HDF5, DAP2, DAP4, PNETCDF or UNDEFINED). Uses nc_inq_format_extended, added in version 4.3.1 of library. If using earlier version of lib, disk_format will be set to UNDEFINED. * default _FillValue now ignored for byte data types (int8 and uint8) as per http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-c/Fill-Values.htmlFill-Values "If you need a fill value for a byte variable, it is recommended that you explicitly define an appropriate _FillValue attribute, as generic utilities such as ncdump will not assume a default fill value for byte variables". ncinfo now returns fill mode information (issue 209). * check to see if filling was disabled before masking data equal to default fill value (issue 209). * add variable type information to Dataset.__repr__ (output of ncinfo). version 1.0.7 (tag v1.0.7rel) ============================= * add the ability to specify the locations of hdf4,jpeg and curl libs, in case netCDF4 was built statically with HDF4 and/or OpenDAP support (issue 207). * add 'ncinfo' utility (like 'ncdump -h' but less verbose). * more information displayed when Dataset or Group instance is printed. * fix for issue 194 (versions after 1.0.5 fail for netcdf 4.1.1, due to call to nc_inq_path, which was added in netcdf 4.1.2). Fixed by adding compile time API check similar to what was done for nc_rename_grp. If filepath Dataset method is called an exception will be raised at runtime if the module was built with netcdf < 4.1.2, or cython was not installed at build time. * fix for issues 202 and 206 (exception raised by numpy.isnan for character data types). * if dateutils not installed and time unit accuracy < 1 second requested, have netcdftime raise an ImportError. version 1.0.6 (svn revision 1312) ================================ * issue warning of endian-ness of dtype argument does not match endian kwarg in createVariable. * make sure netcdf type NC_CHAR always returned in numpy array dtype 'S1' (sometimes arrays of type 'U1' were being returned). Fixes intermittently failing test tst_compoundatt.py on python 3.3. * fix for issue 201 (if data associated with numpy array not the same endian-ness as dtype, data was written incorrectly). Now bytes are swapped if necessary. Variable.endian() now returns 'native' instead of None for NETCDF3 formatted files. createVariable now enforces endian='native' for NETCDF3 files. Added tst_endian.py test case. * fix for issue 200 (library version detection failed on cygwin). * fix for issue 199 (nc4tonc3 utility not copying global attributes). * fix for issue 198 (setup.py chokes when no arguments given). * fix for issue 197 (slicing of netCDF variables using lists of integers). * create 'path' attribute for group instance using posixpath, instead of os.path (to ensure the unix path is used on all platforms). Issue 196. * fix for issue 196 (test failures on win32 due to files being deleted before they are closed). version 1.0.5 (svn revision 1278) ================================ * change setup.py to compile the Cython sources directly, if cython is available. This allows for "ifdef" like capability to modify source at compile time to account for changes in netcdf API (e.g. the forthcoming addition of the nc_rename_grp in version 4.3.1). * added a "renameGroup" method, which raises an exception if the netcdf lib version linked does not support it. Requires netcdf >= 4.3.1. * support for more than one missing value (missing_value attribute is a vector) when converting to masked array. * add 'renameAttribute' method to Dataset, Group and Variable. * fix so that var[:] = x works if x is a scalar, and var is a netcdf variable with an unlimited dimension that has shape () - i.e. no data has been written to it yet. Before this change, var[:] = x did not write any data. Now the scalar x will be written as the first entry in var along the unlimited dimension. * remove dos line feeds from nc3tonc4 (issue 181). * add datatype property for Variable that returns numpy dtype for primitive datatypes (same as dtype attribute) but returns CompoundType or VLType instance for compound or vlen variables (issue 178). * fix logic for deciding where to look for nc-config in setup.py (issue 177). * issue a warning and don't try to apply scale_factor or add_offset if these attributes are not convertible to floats (issue 176). * add filepath method to Dataset instance to return file path (or opendap URL) used to create Dataset (issue 172). * fix for issue 170 (opening a remote DAP dataset fails after creating a NETCDF4 formatted file). * fix for issue 169 (error in chartostring function on 64-bit windows). * add support for missing_value or _FillValue == NaN (issue 168). * added a Dimension.group() method (issue 165). version 1.0.4 (svn revision 1229) ================================= * fixed alignment bug that could cause memory corruption when reading compound type variables. All users of compound types should upgrade. version 1.0.3 (svn revision 1219) ================================= * don't try to write empty data array to netcdf file (fixed failing test with netcdf 4.3.0rc2). * date2num, num2date and date2index now can handle units of microseconds and milliseconds (for proleptic_gregorian calendar, or gregorian and standard calendars as long as the time origin is after 1582-10-15). Issue 159. * Added a _grp attribute to Dimension (issue 165). * don't bundle ordereddict (issue 164). * support reading of vlen string attributes (issue 156). * add --vars option to nc3tonc4 (issue 154). * Don't try to set fletcher32 checksum on scalar variables (it causes HDF5 to crash). Fixes issue 150. * Add --istart/--istop options to nc3tonc4 (issue 148, courtesy of Rich Signell). * fix for proleptic_gregorian in netcdftime.py (courtesy of Matthias Cuntz). version 1.0.2 (svn revision 1196) ================================= * disable version check for HDF5, which is broken by hdf5 1.8.10. * make sure all files have a calendar attribute in MFTime (issue 144). * more robust fix to issue 90 (array shape modified by assignment to a netCDF variable with one more dimension), including test case. version 1.0.1 (svn revision 1190) ================================= * fix error that occurred when retrieving data from a variable that has a missing_value attribute specified as a string (issue 142). * automatically close netcdf files when there are no references left to Dataset object (using __dealloc__ method). Fixes issue 137. * fix for slicing of scalar vlen string variables (issue 140). * fix to allow writing of unicode data to a NC_CHAR variable. * allow for writing of large variables (> 2**32 elements). Fixes issue 130. version 1.0fix1 =============== * fix python 3 incompatibility in setup.py (issue 125). version 1.0 (svn revision 1164) =============================== * add 'aggdim' keyword to MFDataset, so the name of the dimension to aggregate over can be specified (instead of using the unlimited dimension). aggdim=None by default, which results in the previous behavior. aggdim must be the leftmost dimension of all the variables to be aggregated. * raise IndexError when indexing a netcdf variable out of range so iterating over a variable in a for loop behaves as expected (as described in http://effbot.org/zone/python-for-statement.htm). Fixes issue 121. * added MacPorts portfile (so it can be installed via MacPorts on macosx using a "local Portfile repository"). Installs from svn HEAD using 'port install netcdf4-python'. * added experimental 'diskless' file capability (only added to the C lib after the 4.2 release). Controlled by kwarg 'diskless' to netCDF4.Dataset (default False). diskless=True when creating a file results in a file that exists only in memory, closing the file makes the data disapper, except if persist=True keyword given in which case it is persisted to a disk file on close. diskless=True when opening a file creates an in-memory copy of the file for faster access. * add the ability to specify the location of the required libs (and whether to use nc-config) with setup.cfg, instead of using environment variables. * fix ISO9601 date parser so it recognizes time zone offsets in time unit strings (contributed by David Hassel, issue 114, r1117). * add setncatts Dataset,Group and Variable method to add a bunch of attributes (given in a python dictionary) at once. Speeds things up for NETCDF3 and NETCDF4_CLASSIC files a lot, since nc_redef/nc_enddef not need to be called for each attribute (issue 85, r1113). Adding 1000 attributes is about 35 times faster using setncatts to add them all at once. Makes no difference for NETCDF4 formatted files, since nc_redef/nc_enddef is not called. * only round after apply scale_factor and add_offset if variable type is integer (issue 111, r1109). * Fixed bug with all False Boolean index (r1107). * added support for after, before and nearest selection method to date2index fast "first guess" indexing (r1106). * Remove white space in time units string (netcdftime._parse_date). An extra space in the time units of one CMIP3 model caused an error (r1105). * based on results with examples/bench_compress2.py, change default complevel for zlib compression from 6 to 4. If complevel=0, turn compression off entirely (set zlib=False) (r1102). version 0.9.9 (svn revision 1099) ================================ * changed default unicode encoding from "latin-1" to "utf-8", since this is the python 3 default, and the only encoding that appears to work for dimension and variable names. * added test case for unicode attributes, variable and dimension names. * fixes for unicode variable, dimension and group names. * fix for unicode attributes in python3 (ncdump did not intrepret them as text strings). Issue 107. * add --format option to nc4tonc3 utility (can be either NETCDF3_CLASSIC or NETCDF3_64BIT). Fixes issue 104. version 0.9.8 (svn revision 1080) ================================ * use numpy.ma.isMA to check for masked array (instead of checking for presence of 'mask' attribute). * fixes for AIX with ibm xlc compiler. * make sure unicode attributes don't get converted to ascii strings (issue 98). version 0.9.7 (svn revision 1073) ================================ * Added __str__ methods to Dataset, Variable, Dimension, CompoundType, VLType and MFDataset, so useful human-readable information is provided when these objects are printed in an interactive session. * don't try to apply scale_factor and offset if scale_factor=1 and add_offset=0 (to avoid making copies of large arrays). * changed netCDF4._default_fillvals to netCDF4.default_fillvals (to make part of public API). Added to docs (issue 94). version 0.9.6 (svn revision 1043) ================================= * changed default unicode encoding from "ascii" to "latin-1" (iso-8859-1). * add "unicode_error" module variable to control what happens when characters cannot be decoded by the encoding specified by the "default_encoding" module variable (which is "ascii" by default). unicode_error = "replace" by default which means bad characters are replace by "?". Previously an error was raised, the old behavior can be obtained by setting unicode_error = 'strict'. Fixes issue 92. * add __enter__ and __exit__ methods so you can do "with Dataset(url) as f:" (issue 89). * don't add extra singleton dimensions to rhs numpy arrays when assigning to a netcdf variable. Fixes issue 90. * coerce missing_value attribute to same type as variable (for primitive types). Fixes issue 91. version 0.9.5 (svn revision 1031) ================================ * fix for compound variables on python 3.2. * fix slicing of masked MFDataset variables (issue 83). * round to nearest integer after packing with scale_factor and add_offset (instead of truncation) (issue 84). * if add_offset missing, but scale_factor present, assume add_offset zero. if scale_factor missing, but add_offset present, assume scale_factor one. (this is consistent with unidata recommendations - issue 86). * only try to convert strings to bytes for python 3 so Dataset can be subclassed (issue 87). version 0.9.4 (svn revision 1018) ================================ * tested with python 2.7.1/3.1.3 using netcdf 4.1.2 and hdf5 1.8.6. * Added a 'default_encoding' module variable that controls how unicode strings are encoded into bytes. Default is 'ascii'. * now works on Python 3. * netCDF3 module removed. If you still need it, get it from netCDF4 0.9.3. * regenerated C source with Cython 0.14.1. * Added a MFTime class. Provide a unified interface to MFDataset time variable using different time units. * Fixed bug in netcdftime (issue 75) that occurs when time specified is within one second of the end of the month. * on unix-like systems, the environment variable USE_NCCONFIG can be set to tell setup.py to use the nc-config script installed by netcdf to figure out where all the libs and headers are (without having to specify NETCDF_DIR, HDF5_DIR, etc). Only works with netcdf 4.1.2. version 0.9.3 (svn revision 930) ================================ * fix chunk sizes bug (chunk sizes pointer should be size_t, not int). Fixes issue 66. Added test in tst_compression.py * fixed writing of data with missing values with scale/offset packing. Added test (tst_masked2.py). * fix iso8601 regex in netcdftime date parser so it can parse 'hours since 1-1-1 ...' (year had to be 4 digits previously) version 0.9.2 (svn revision 907) ================================ * fix netcdftime bug with '360_day' calendar. Fixes issue 59. * make sure scalar slice of 1d variable returns array scalar (not array of shape (1,)). Fixes issue 57. * updated date parser in netcdftime. Can now handle units like "seconds since 1970-01-01T00:00:00Z". * added support in setup.py for specifying the locations of the HDF5/netcdf-4 headers and libs separately with environment variables (HDF5_INCDIR, HDF5_LIBDIR).i Patch contributed by Patrice Dumas. * add masked array support to num2date (dates for missing times set to None). * add chunk_cache keyword to createVariable. HDF5 default is 1mb, which can cause problems when creating 1000's of variables. In such cases, chunk_cache can be reduced, or set to zero. * add set_var_chunk_cache and get_var_chunk_cache Variable methods. * raise AttributeError when trying to set _FillValue attribute (it can only be reliably set on variable creation, using the fill_value keyword to createVariable). ```
Links - PyPI: https://pypi.org/project/netcdf4 - Changelog: https://pyup.io/changelogs/netcdf4/ - Repo: http://github.com/Unidata/netcdf4-python
coveralls commented 6 years ago

Coverage Status

Coverage remained the same at 96.327% when pulling 831aded2b7279069a5c0c166e557225da307ca5f on pyup-update-netcdf4-1.3.1-to-1.4.0 into 0d32c2b4e18d300a11b748a552f6adbc3dd8f59d on master.