search

CHLNDDEV / OceanMesh2D

A two-dimensional triangular mesh generator with pre- and post-processing utilities written in pure MATLAB (no toolboxes required) designed specifically to build models that solve shallow-water equations or wave equations in a coastal environment (ADCIRC, FVCOM, WaveWatch3, SWAN, SCHISM, Telemac, etc.).
https://github.com/sponsors/krober10nd
GNU General Public License v3.0
179 stars 65 forks source link

Support for writing Self Attraction and Loading (SAL) files in NetCDF for the ADCIRC model #231

Closed krober10nd closed 2 years ago

krober10nd commented 3 years ago

m = msh('dummy.14');

TPXO9 = '/Volumes/KeithJaredR/MESHING_DATASETS/TIDAL_CONDITIONS/h_tpxo9.v1.nc';

TS = '2012 Aug 01 00:00';

TE = '2012 Nov 01 00:00';

SALDATA = 'FES2004';

DT = 1.0;

m = Make_f15(m, TS, TE, DT, 'tidal database', TPXO9, 'const',{'major8'}) ;

m = Make_f24(m, SALDATA, 1, 1);

write(m,'testing','f24');


The following codes creates a fort.24 file called `testing.24.nc` with the following header containing 8 major tidal constituents. Note the constituents are named with a unique alphanumeric identifier corresponding to FES2004. Any thoughts on a better way to do this are suggested. Note the *_omega values are scalars.

(base) keiths-MacBook-Pro:private Keith$ ncdump -h testing.24.nc netcdf testing.24 { dimensions: valspernode = 3 ; nnodes = 191600 ; variables: double Q1_omega ; double Q1_vals(nnodes, valspernode) ; double O1_omega ; double O1_vals(nnodes, valspernode) ; double P1_omega ; double P1_vals(nnodes, valspernode) ; double K1_omega ; double K1_vals(nnodes, valspernode) ; double N2_omega ; double N2_vals(nnodes, valspernode) ; double M2_omega ; double M2_vals(nnodes, valspernode) ; double S2_omega ; double S2_vals(nnodes, valspernode) ; double K2_omega ; double K2_vals(nnodes, valspernode) ; }

WPringle commented 3 years ago

I think fort.24 netcdf format could be improved by adding number of frequency dimensions and metadata information. I can work on this a bit later today.

WPringle commented 3 years ago

@zcobell @krober10nd see what you think of the format..

Example:

image
zcobell commented 3 years ago

Are there CF conventions for tide parameters? The current adcirc format is loosely CF-ish in naming and variable fields and attributes

krober10nd commented 3 years ago

These appear to be the CF conventions https://www.bodc.ac.uk/resources/delivery_formats/cfnetcdf_format/

Also Zach brought this up offline but the __FillValue should be 0.0 so that NetCDF (at least the compressed versions) won't write that data to the file and save a lot of memory potentially for meshes with many dry nodes.

So in each variable probably should have 

units 
standard_name
long_name 
_FillValue

Probably should also have this at the end: 

title — a succinct description of the data set
insititute — the organisation where the data were produced
source — how the data were produced, e.g. model type, run number and circumstances
history — an audit trail of data set processing
references — a list of references that describe the data or the methodology used
comment — other useful information not covered elsewhere that adds value
author — the person(s) who generated the data
zcobell commented 3 years ago

There is a standard_name table worth consulting too

https://cfconventions.org/standard-names.html

krober10nd commented 3 years ago

A couple suggestions:

WPringle commented 3 years ago

The salnodes is different to other output files for the nodes, because it is (potentially) a subset of nodes where SAL is defined on.

The fill value of zero is redundant because actually SAL is defined non-zero everywhere (even overland, same as the potential).

krober10nd commented 3 years ago

Oh okay, that part wasn't clear. Wouldn't this then be different then from how all the other output files are defined?

Is it really more efficient to store indices of where the values are defined vs. use a _FillValue?

WPringle commented 3 years ago

Oh okay, that part wasn't clear. Wouldn't this then be different then from how all the other output files are defined?

Is it really more efficient to store indices of where the values are defined vs. use a _FillValue?

It's just that ADCIRC requires knowing the indices that each SAL point is defined on. Although as a standalone file its worth knowing the geographic locations this is for an input file for ADCIRC.

krober10nd commented 3 years ago

Okay this is good to go by me. I added some global attributes to the created file. I also added a warning telling users about the potential to use the NetCDF format.

>> ncdisp testing.24.nc
Source:
           /Volumes/KeithJaredR/testing.24.nc
Format:
           netcdf4_classic
Global Attributes:
           title         = 'The self-attraction and loading terms for an ADCIRC simulation'
           creation_date = '26-Jun-2021 16:17:49'
           source        = 'Made by OceanMesh2D writefort24'
           references    = 'https://github.com/CHLNDDEV/OceanMesh2D/'
Dimensions:
           sal_nodes = 191600
           num_const = 8
           char_len  = 2
Variables:
    mesh_indices
           Size:       191600x1
           Dimensions: sal_nodes
           Datatype:   int32
           Attributes:
                       long_name = 'index of SAL values into mesh coordinates'
    const       
           Size:       8x2
           Dimensions: num_const,char_len
           Datatype:   char
           Attributes:
                       long_name = 'names of the tidal harmonic constituents'
    frequency   
           Size:       8x1
           Dimensions: num_const
           Datatype:   double
           Attributes:
                       long_name = 'frequency of harmonic constituents'
                       units     = 'rad/s'
    salamp      
           Size:       8x191600
           Dimensions: num_const,sal_nodes
           Datatype:   single
           Attributes:
                       long_name = 'amplitude of self-attraction and loading tide elevation'
                       units     = 'm'
    salphs      
           Size:       8x191600
           Dimensions: num_const,sal_nodes
           Datatype:   single
           Attributes:
                       long_name = 'phase-lag of self-attraction and loading tide elevation'
                       units     = 'degrees with respect to GMT/UTC'
WPringle commented 3 years ago

Cool, you didn't want to change to SAL_amplitude and SAL_phase etc? I think that is probably a good idea.

zcobell commented 3 years ago

I think this format should probably mirror the fort.63 format as closely as we can for consistency across ADCIRC files and without the lookup table. I also would not abbreviate things so they aren't clear. I'd recommend something like this:

dimensions:
        node = 1578383 ;
        num_constituents = 8 ;
        char_len  = 2 ;
variables:
        double x(node) ;
                x:long_name = "longitude" ;
                x:standard_name = "longitude" ;
                x:units = "degrees_east" ;
                x:positive = "east" ;
        double y(node) ;
                y:long_name = "latitude" ;
                y:standard_name = "latitude" ;
                y:units = "degrees_north" ;
                y:positive = "north" ;
         constituents(num_constituents,char_len);
                constituents:long_name = 'names of the tidal harmonic constituents'
                constituents:standard_name = 'name_of_tidal_harmonic_constituents' // is there a standard name? if not we should follow the CF standard name construction format
         frequency(num_constituents);
                frequency:long_name = 'frequency of tidal harmonic constieuents' ;
                frequency:standard_name = 'frequency_of_tidal_harmonic_constituents' ;
                frequency:units = 'radians/second';
         sal_amplitude(num_constituents,node);
                sal_amplitude:long_name = 'amplitude of self-attraction and loading tide elevation' ;
                sal_amplitude:standard_name = 'amplitude_of_self_attraction_and_loading_tide_elevation' ;
                sal_amplitude:units = 'm' ;
                sal_amplitude:_FillValue = 0.0 ;
         sal_phase(num_constituents,node);
                sal_phase:long_name = 'phase-lag of self-attraction and loading tide elevation' ;
                sal_phase:standard_name = 'phase_lag_of_self_attraction_and_loading_tide_elevation' ;
                sal_phase:units = 'degrees with respect to GMT/UTC' ;
                sal_phase:_FillValue = 0.0 ;
krober10nd commented 3 years ago 
>> ncdisp testing.24.nc
Source:
           /Users/Keith/junk/OceanMesh2D/testing.24.nc
Format:
           netcdf4_classic
Global Attributes:
           title         = 'The self-attraction and loading terms for an ADCIRC simulation'
           creation_date = '28-Jun-2021 12:35:29'
           source        = 'Made by OceanMesh2D writefort24'
           references    = 'https://github.com/CHLNDDEV/OceanMesh2D/'
Dimensions:
           node             = 191600
           num_constituents = 8
           char_len         = 2
Variables:
    x            
           Size:       191600x1
           Dimensions: node
           Datatype:   int32
           Attributes:
                       standard_name = 'longitude'
                       units         = 'degrees_east'
                       positive      = 'east'
    y            
           Size:       191600x1
           Dimensions: node
           Datatype:   int32
           Attributes:
                       standard_name = 'latitude'
                       units         = 'degrees_north'
                       positive      = 'north'
    constituents 
           Size:       8x2
           Dimensions: num_constituents,char_len
           Datatype:   char
           Attributes:
                       standard_name = 'name_of_tidal_harmonic_constituents'
                       long_name     = 'name of tidal harmonic constituents'
    frequency    
           Size:       8x1
           Dimensions: num_constituents
           Datatype:   double
           Attributes:
                       standard_name = 'frequency_of_tidal_harmonic_constituents'
                       long_name     = 'frequency of tidal harmonic constituents'
                       units         = 'radians/second'
    sal_amplitude
           Size:       8x191600
           Dimensions: num_constituents,node
           Datatype:   single
           Attributes:
                       _FillValue    = 0
                       standard_name = 'amplitude_of_self_attraction_and_loading_tide_elevation'
                       long_name     = 'amplitude of self attraction and loading tide elevation'
                       units         = 'm'
    sal_phase    
           Size:       8x191600
           Dimensions: num_constituents,node
           Datatype:   single
           Attributes:
                       _FillValue    = 0
                       long_name     = 'phase-lag of self-attraction and loading tide elevation'
                       standard_name = 'phase_lag_of_self_attraction_and_loading_tide_elevation'
                       units         = 'degrees with respect to GMT/UTC'
WPringle commented 3 years ago

Cool this is fine by me, since it is more standalone.

@zcobell so, is your proposal to do the mapping of the nodes inside adcirc? Or, since practically speaking we generally always apply sal nodes everywhere you could enforce that number of nodes between this file and the fort.14 are the same?

zcobell commented 3 years ago

Right, I think this file becomes tied to the mesh numbering similar to the fort.13 and apply the values inside of adcirc as appropriate. Having the lookup table also complicates the decomposed fort.24

WPringle commented 3 years ago

I'm fine to merge it if we are good btw.

krober10nd commented 3 years ago

In regard to the CF standard names for name_of_tidal_harmonic_constituents this appears fine and in agreement with the guidelines (no standard name already exists upon searching their database).

http://cfconventions.org/Data/cf-standard-names/docs/guidelines.html

zcobell commented 3 years ago

For merging, I'd say let's throw the format example up on the adcirc issue for feedback and probably no use merging this here until ADCIRC/ADCPREP/PADCIRC actually support it too.

krober10nd commented 3 years ago

Yep, agreed it has to wait for changes in ADCIRC anyway.

krober10nd commented 2 years ago

Let's update this with the current branch and merge it in. Having SAL files in a simple NetCDF format may be useful for other models besides ADCIRC at some point too.