Ofx/sftof varies with time

[uwefladrich]

The sftof variable (sea area percentage) in the Ofx table varies over time, which should not happen for any variable in the *fx tables. This means that the content of the file will depend on what year was actually used to cmorise the data and that leads to inconsistent data across different realisations with the same model configuration.

Beside being inconsistent, the values are probably plain wrong as well. It appears that the variable is somehow derived from the sea ice (hence the time dependency).

This problem was already mentioned in https://github.com/EC-Earth/ece2cmor3/issues/249#issuecomment-457176610 as one of the issues with Ofx variables. However, it was never followed up and not solved when #249 was closed.

[uwefladrich]

Questions for now:

• How does ece2cmor3 derive this variable?
• Can we identify a grid/mask related variable in the NEMO output that fits the purpose better?

[uwefladrich]

The definition in file_def_nemo-opa.xml is

<field enabled="True" field_ref="iceconc_pct" freq_op="1y" 
          grid_ref="grid_T_2D" id="id_1y_sftof" 
          name="sftof" operation="once" unit="%"> 
                100 - this
</field>

which seems wrong from the start.

[uwefladrich]

... and for reference:

vid: [var] sftof [20e7d22ad09b324af00f41f6060701a7]
frequency: fx
title: Sea Area Percentage
2.3 Dimensions and related information: Fixed, Global field (single level)
mipTable: Ofx
label: sftof
mtid: [miptable] Ofx [MIPtable::Ofx]
type: real
description: This is the area fraction at the ocean surface.
processing: Should this be recorded as a function of depth? Report on the same grid
that ocean fields are reported (i.e., the ocean native grid, or the grid that ocean data
has been provided to CMIP. For completeness, provide this even if the ocean grid is
the same as the atmospheric grid.
modeling_realm: ocean

[klauswyser]

My suggestion would be to skip all the processing of fx variables with ece2cmor, and instead "somebody" creates these files manually (from e.g. the areas and masks file from Oasis). These files need to be generated only once and can then be copied to all experiments (once per experiment and configuration, not once per member!). Of course, the metadata and file names will have to be adjusted, but that shouldn't be a problem (and certainly possible with help of a short script).

[uwefladrich]

Given that NEMO is not having any partial cells, it seems that the tmask (converted to 0/100 per cent) could be a candidate for sftof. Can any ocean person confirm?

[oloapinivad]

My suggestion would be to skip all the processing of fx variables with ece2cmor, and instead "somebody" creates these files manually (from e.g. the areas and masks file from Oasis). These files need to be generated only once and can then be copied to all experiments (once per experiment and configuration, not once per member!). Of course, the metadata and file names will have to be adjusted, but that shouldn't be a problem (and certainly possible with help of a short script).

My two cents on this would to keep everything into ece2cmor. I am aware that this may be not so easy to implement and perhaps redundant since each Ofx will be repeated for each experiment/ensemble member. However, I personally like the idea of a single tool that handles everything without further adjustment later on: this helps the workflow and possibly minimise the source of errors for users that are not into the ece2cmor development. Furthermore, such files are cmorized only once by the tool, so that their volume and computational cost it is not really relevant.
Of course both the hypothesis will work for me, these are simply my feelings on this.

[klauswyser]

Furthermore, such files are cmorized only once by the tool, so that their volume and computational cost it is not really relevant.

This is currently not the case, Ofx files are created for each year and for each member of the simulation, leading to the infamous *.nc.copy files that then have to be removed manually in a 2nd step. This can certainly be improved upon, of course.

Another problem is that many of the variables needed to create fx, Ofx, Lfx and efx files are not in the model output, for example areacell. This information is easily available from the Oasis input files, but these are not part of the output and therefore need a hack anyway. In my eyes the simplest solution by far is therefore to create a set of template fx files that then only need to be adjusted for each experiment and model configuration. Ideally, this adjustment could become part of ece2cmor3, however it may take a while to 1) create the templates, 2) write a script that manipulates the headers , and 3) integrate all this into ece2cmor3.

[goord]

Perhaps a middle ground would be to have a separate script or option in ece2cmor3 that does the fx variables. In this way one can be sure the output format, metadata etc. is correct because it is embedded in the ece2cmor3 archirtecture, but it is not enabled by default and its inner working will be quite different than ece2cmor3, with various static input files, etc.

[klauswyser]

Here is a reason for not having the fx variables as part of ece2cmor: let's assume you create fx files on your local system. You can then try to publish these files, but what if somebody else (BSC, KNMI, DMI,...) already has publsihed a set of fx files for that configuration and that experiment? I don't know how the index server will react if you try to publish a file that already exists elsewhere, but I'd rather not try. fx-files are special, and we have to make sure that there is one and only one set of fx files for each configuration and experiment in the CMIP6 archive.

[oloapinivad]

Here is a reason for not having the fx variables as part of ece2cmor: let's assume you create fx files on your local system. You can then try to publish these files, but what if somebody else (BSC, KNMI, DMI,...) already has publsihed a set of fx files for that configuration and that experiment?

Uhm, forgive but I am not following here, fx files are specific for each model-experiment-ensemble member set? Or they should be unique for each experiment? Currently mine reads as this one: areacello_Ofx_EC-Earth3_historical_r4i1p1f1_gn.nc

[klauswyser]

Sorry, @oloapinivad is correct, the identifier is part of the filename and each member of an experiement comes with its own set of fx files (which is stupid in my eyes because they are just copies of each other, fortunately they are not big). In that case my argument falls, and the fx files could be part of the ordinary workflow.

[treerink]

Indeed I overlooked this issue when closing #249.

Questions for now:

• How does ece2cmor3 derive this variable?

Genecec takes this from the ping_ocean_DR1.00.27.xml file in branch-r6874-control-output-files/runtime/classic/ctrl/

   <field id="CMIP6_sftof"         field_ref="iceconc_pct" > 100 - this </field> <!-- P1 (%) sea_area_fraction : This is the area fraction at the ocean surface. **** NEMO-RD: variable already provided in ping_seaIce under name siconc - we decide to keep it here as well -->

Its comment referes to siconc given by ping_seaIce_DR1.00.27.xml (and which I believe is produced/cmorised correctly):

   <field id="CMIP6_siconc"        field_ref="iceconc_pct"           /> <!-- P1 (%) sea_ice_area_fraction : Area fraction of grid cell covered by sea ice -->

In principle this seems correct to me. Also when I check the field values of siconc and `sftof`` in the raw NEMO output this looks fine and consistent.

The remaining thing is that our NEMO output doesn't have a time fixed sftof, it is evidently changing in time. So I would suggest that we can't provide with EC-earth3 Ofx sftof, so we should drop it. And for the rest for anyone who want this sftof daily or monthly (time dependent), it is easy to derive from siconc.

Any opinions?

Otherwise I suggest to drop Ofx sftof(only) from the json data request file.

[uwefladrich]

Well, yes, sftof is derived from the sea ice fraction in NEMO/ShaCoNEMO. I did actually check the latest ShaCoNEMO repository state, it's still what they do. The mystery is why they do it. Given the fact that sftof is in Ofx and that it is the "area fraction of ocean surface", we came to the conclusion that is is a variable describing the ocean grid cells, not the ice free part. After all, it says ocean surface, not water surface. If that is correct, then sftof is just 100 for all ocean (i.e. non-land) grid cells of the ocean grid. As such, it would be easy to produce.

[treerink]

You mean just the ocean mask? But Sea Area Percentage, then the percentage is misleading me as well at least if they really mean actually a mask.

http://clipc-services.ceda.ac.uk/dreq/u/20e7d22ad09b324af00f41f6060701a7.html

[uwefladrich]

Yes, in the case of NEMO it is just the mask, if we're correct. I was assuming there could be models that have cells with partial ocean surface (the other parts being land or maybe ice shelf or whatever). I think IFS has partial land/sea cells, not sure it makes any sense for an ocean model, but I assume it could exist.

The main argument is still that it is in Ofx and any sea-ice related variable can't be there, unless I'm misunderstanding the meaning of fx or one assumes fixed sea-ice. A secondary argument is that the definition of sftof doesn't mention any sea ice.

[zklaus]

I just discussed this with Torben a bit and we also concluded that it should be the land sea mask. Apart from the arguments mentioned already, also consider that there is no other variable in Ofx that reports the mask. Also note the usage of the other fields sft*f used for various components and in various tables.

[treerink]

The recipe of producing Ofx sftof is given here by Uwe.

[treerink]

This issue and link is added to the wiki.

Closing.

[etiennesky]

I would like to understand why this known problem was not fixed in ece2cmor3, given that it is a simple fix, instead of done in a post-processing script which is not included in ece2cmor3

[goord]

I guess this is an example of 'minimization of effort' from our side. We have plenty of static data on b2share, this one could be added easily.

[etiennesky]

any news on this issue? what are we supposed to due for optimesm cmorization ?