CF compliant output and time handling

[CyprienBosserelle]

CF compliancy

• [x] Add long name and units to all output variables
• [ ] Add correct long name and unitsfor xx and yy axis (easting/northing/longitude/lat)
• [x] Add correct time units for time axis (see below)
• [x] Add CRS info
• [x] Add shuffle deflate level

time handling

• [x] Add ref time in param file as well as user
• [x] Handle time string reading for txt forcing
• [x] read time from netcdf properly
• [x] using stringtime for totaltime and endtime
• [ ] Add reftime/starttime strings to log output

[CyprienBosserelle]

Time reading from text files

Below std lib should be enough for reading and interpreting time from txt files

#include <ctime>
#include <iomanip>
#include <sstream>
#include <iostream>
#include <stdio.h>

[CyprienBosserelle]

How to add deflate and compression level for stored variables:

nc_create("test.nc", NC_CLOBBER | NC_NETCDF4, &ncid);

    nc_def_dim(ncid, "x", NX, &x_dimid);

    nc_def_var(ncid, "data", NC_DOUBLE, NDIMS, &x_dimid, &varid);
    int shuffle = 1;
    int deflate = 1;        // This switches compression on (1) or off (0).
    int deflate_level = 8;  // This is the compression level in range 1 (less) - 9 (more).
    nc_def_var_deflate(ncid, varid, shuffle, deflate, deflate_level);
    nc_enddef(ncid);

    nc_put_var_double(ncid, varid, data_out);
    nc_close(ncid);

[CyprienBosserelle]

Deflate level is amazing!

in a small test using the valley topography available in the test. I save 7 variables at 3 different levels (0, 1, 2) saving approx. every 3rd step.

Deflate level	Output file size (KB)	% of original size	runtime
None	11,851	100%	34s
1	722	6.1%	35s
2	700	5.9%	35s
4	758	6.3%	34s
6	555	4.7%	34s
8	522	4.4%	34s
9	519	4.3%	33s

[CyprienBosserelle]

Using datetime string for start/end time as well as forcing time series.

You can now use datetime string as input for TXT files as yyyy-mm-ddTHH:MM:SS as in year-month-day"T"hours:minutes:seconds also accepted yyyy/mm/ddTHH:MM:SS .

Time can still be in the "old" float format.

[!NOTE]
Note the T is the key in BGFlood for identifying if whether the input is a float (old format) or a datetime string.

The way that BG_Flood is working internally doesn't change so dealing with time string requires a reference time.

While the model doesn't need a reference time, it is preferable/good practice to set one up if using. That reference time can be specified in the parameter file. There are several options:

• Specify reftime = as a date string. Automatically totaltime will be set to 0 so startime == reftime
• Specify totaltime=; or endtime but no reftime. Automatically reftime is set to totaltime or endtime whichever comes first in the param file.
• Do not specify totaltime or endtime or reftime in the param file. Then reftime will be obtained from the first specified string in boundaries, river, wind file. If none have datetime string forcing reftime will be set to 2000-01-01T00:00:00

[!IMPORTANT]
Set starttime to avoid surprises

Example

Example of flow file or boundary file:

2020-01-01T00:00:00,1.000000
2020/01/02T01:00:00,1.000000

BG_param.txt

reftime = 2020-01-01T00:00:00

[CyprienBosserelle]

Nan Handling improved for bathymetry

Instaed of transforming NAN to 0 systematically for any input we are now using explicit "denan" function for each input. Effectivelly that doesn't affect much appart from atmospheric pressure. But it allows bathymetry input to include NANs in a useful way.

Multi DEM including some with NAN mask

When using multiple input DEM. The model removes any NAN in the first DEM of teh list. But any NaN in subsequent DEM means that the model cells falling on that zone will ignore thethis DEM and keep whatever value is was already assigned.

...Give example of usage here...

[CyprienBosserelle]

Adding CRS information to output

BGFlood does not require a Coordinate Reference System. The model can run on projected or spherical coordinate system regardless of the projection as long as the CRS of the input DEM and other forcing are consistent. Hence BGFlood can run without information on the CRS. It is however useful for CRS info to be displayed in BGFlood output file for use in GIS software and for plotting.

Option 1: Inherited from principal DEM file (NetCDF input only)

If CRS information is stored in the principal DEM file in a Well Known Text format (WKT) (first one listed in parameter file) it will automatically inherited. BGFlood first looks for the value grid_mapping attribute in the topography variable of teh NetCDF file. The BGFLood looks for the variable named in the attributed. This variable should contain CRS in WKT format in either the crs_wkt or spatial_ref attribute.

Option 2: Enforce the WKT in parameter file

If the CRS data is not available in WKT in the input DEM you can specify it as: crs = wktstring where wktstring is the CRS WKT string. You can find/copy the crs wkt string on https://epsg.io/ For example the WGS 84 / UTM zone 2S string should be: crs = PROJCS["WGS 84 / UTM zone 2S",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-171],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",10000000],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","32702"]]