Parallel-NetCDF / PnetCDF

Source code repository of PnetCDF library and utilities
https://parallel-netcdf.github.io
Other
80 stars 22 forks source link

change checking for size of dim/attr/header limit #130

Closed wkliao closed 5 months ago

wkliao commented 5 months ago

The maximal header extent for CDF 1 and 2 should be NC_MAX_INT, i,e, the maximum value of a signed 4-byte integer, 2^31-1. Same for the dimension size. This is because NetCDF file format defines them as non-negative signed integers, OFFSET and NON_NEG respectively.

NetCDF file format specification:

netcdf_file  = header data
header       = magic numrecs dim_list gatt_list var_list
dim_list     = ABSENT | NC_DIMENSION nelems [dim ...]
dim          = name dim_length
var          = name nelems [dimid ...] vatt_list nc_type vsize begin
vsize        = NON_NEG
dim_length   = NON_NEG 
NON_NEG      = <non-negative INT>
begin        = OFFSET
OFFSET       = <non-negative INT> | <non-negative INT64>
INT          = <32-bit signed integer, Bigendian, two's complement>

Note for variable size, the maximal is 2^31-3 for CDF-1 2^32-3 for CDF-2 2^63-3 for CDF-5 Variable size is calculated internally in PnetCDF. Therefore, the variable size is limited to the space available in bytes. -3 is due to the 4-byte upward alignment. Although vsize is defined and stored in the file header, it is marked as a redundant item and is not used in PnetCDF.