search

nextsimhub / nextsimdg

neXtSIM_DG : next generation sea-ice model with DG
https://nextsim-dg.readthedocs.io/en/latest/?badge=latest
Apache License 2.0
10 stars 13 forks source link

XIOS: Determine Axes and Domains from NetCDF file #732

Open jwallwork23 opened 1 week ago

jwallwork23 commented 1 week ago

Thinking more about what's required for #552, I realised there's a key missing piece for reading files.

Usually, XIOS reads in an XML configuration file along with the NetCDF file that contains the data. The XML file says how to set up the various attributes, e.g., Axes, Domains, Fields. If we want to go ahead with using the XIOS API instead of XML files then we need to be able to read such information from NetCDF files directly. This could be done by borrowing from the existing I/O implementation, for example.

On the other hand, am I right in thinking that the only use case for having XIOS read from file is for restarts? In the case of restarts, you presumably already know how the dimensions and fields are laid out so could reproduce the XIOS attributes without needing to read them.

Any thoughts on this @timspainNERSC @einola?

timspainNERSC commented 1 week ago

The ultimate plan is of course to replace my ad-hoc IO with XIOS. So, below is how file reading currently works.

  1. The restart file is briefly opened to read the contents of structure/type. This is used to select with implementation of StructureModule will be used to read the file. The file is then closed.
  2. (Assuming ParaGrid from here on). The restart file is opened and the dimension data is read. Up until this point, the model has no defined size or land map. This is all determined from the restart file at this point in the start-up phase.
  3. The contents of the file are then read, using the sizes just obtained, and every variable contained in the restart file is added to a name->data map contained in a ModelState object.
  4. The restart file is closed and the ModelState object passed to the cascade of classes that make up the model.

Additionally there is reading forcing data from forcing files, which contain many time samples. Currently the forcing files are fixed to the same grid as the model fields, but the hope is that XIOS or OASIS will allow us to use the files provided in the reanalysis datasets.

The summary is that the current method is to read the dimensions and fields as part of the input process, and not to have them predefined in the configuration file, if possible. And definitely not to set them at compile time.

jwallwork23 commented 6 days ago

Thanks for the description here @timspainNERSC, it's really useful! I'll implement file writing first and see what comes up there before coming back to this to implement file reading.