Cloud variable modifications required to align with OSCAR requirements

[ejwelton]

Branch

https://github.com/wmo-im/wmds/blob/issue265/tables_en/1-01-01.csv

Summary and Purpose

Improvement of the vocabulary for clouds: Variables are missing definitions or the definitions are unclear and should be modified.

Stakeholder(s)

Proposal

Add the following variable:	notation	path	name	description
new id	\Atmosphere\Clouds\Cloud Phase	Cloud Phase	Aggregate state of cloud particles, either water droplets/drops, snowflakes, ice/snow crystals, or mixed.

Add descriptions:	notation	path	name	description (old)	description (new)
179	\Atmosphere\Clouds\Cloud amount	Cloud amount		Fraction of the sky covered by clouds of a particular type or combination. It can refer to a genus, species, variety, layer, or a certain combination of clouds. Also, see 'cloud cover'.
180	\Atmosphere\Clouds\Cloud cover	Cloud cover		The amount of sky estimated to be covered by a specified cloud type (partial cloud amount), or by all cloud types (total cloud amount). In either case, the estimate is made to the nearest okta (eighth) and is reported on a scale which is essentially one of the nearest eighth, except that figures 0 and 8 on the scale signify a completely clear and cloudy sky, respectively, with consequent adjustment to the adjacent 1 and 7 okta intervals.
181	\Atmosphere\Clouds\Cloud drop effective radius	Cloud drop effective radius	Size distribution of liquid water drops, assimilated to spheres of the same volume. Considered as both a 3D field throughout the troposphere and a 2D field at the top of cloud surface.	The area weighted mean radius of liquid water drops, assimilated to spheres of the same volume
329	\Atmosphere\Clouds\Ice\Cloud ice effective radius	Cloud ice effective radius	Size distribution of ice particles, assimilated to spheres of the same volume. Considered as both a 3D field throughout the troposphere and a 2D field at the top of cloud surface	The area weighted mean radius of ice particles, assimilated to spheres of the same volume.
506	\Atmosphere\Clouds\Optical properties\Cloud optical depth	Cloud optical depth	Effective depth of a cloud from the viewpoint of radiation propagation. OD = exp(-K.Δz) where K is the extinction coefficient [km-1] and Δz the vertical path [km] between the base and the top of the cloud	Effective depth (thickness) of a cloud from the viewpoint of radiation propagation. Specifically, OD = exp(-K.Δz), where K is the extinction coefficient [km-1] and Δz the vertical path [km] between the base and the top of the cloud. The WMO Cloud Atlas (https://cloudatlas.wmo.int/en/optical-thickness.html) defines cloud optical thickness as 'The optical thickness of a cloud is the degree to which the cloud prevents light from passing through it.'

Rename the following variables to align with OSCAR/Requirements and add a definitions	notation	path	name (old)	name (new)	description (new)
531	\Atmosphere\Clouds\Position\ Cloud Base Height	Height of cloud base	Cloud Base Height	For surface observations, height of the cloud base above ground level; for aircraft observations, altitude of the cloud base above mean sea level.
532	\Atmosphere\Clouds\Position\ Cloud Top Height	Height of cloud top	Cloud TopHeight	Height of the top of the cloud (highest cloud in case of multi-layer clouds) above mean sea level.

Rename the following variable to provide context	notation	path	name (old)	name (new)	description
508	\Atmosphere\Clouds\Optical properties\Optical depth within each cloud layer	Optical depth within each layer	Optical depth within each cloud layer

Reason

This proposal will better align these cloud variables with the OSCAR requirements.

---

Original comment:

Branch https://github.com/wmo-im/wmds/blob/issue265/tables_en/1-01-01.csv (edited)

Summary and Purpose Some cloud variables should be deprecated because they do not map to OSCAR requirements. Other variables are missing definitions or the definitions are unclear and should be modified.

Stakeholder(s)

Proposal

1. Variable: Cloud Amount. This variable is not in the OSCAR requirements and has no definition. It should be deprecated.
2. Variable: Cloud Cover. This variable has no definition. Suggest using the definition from OSCAR requirement: 3D field of fraction of sky filled by clouds
3. Variable: Cloud drop effective radius: The definition is confusing and inaccurate. The current definition says it is a size distribution which is not the same thing as effective radius. Suggest new definition based on existing one: The area weighted mean radius of liquid water drops, assimilated to spheres of the same volume. Considered as both a 3D field throughout the troposphere and a 2D field at the top of cloud surface. 
4. Variable: Cloud ice effective radius: The definition is confusing and inaccurate. The current definition says it is a size distribution which is not the same thing as effective radius. Suggest new definition based on existing one: The area weighted mean radius of ice particles, assimilated to spheres of the same volume. Considered as both a 3D field throughout the troposphere and a 2D field at the top of cloud surface. 
5. Variable: Optical depth within each layer. This variable is not in the OSCAR requirements and has no definition. Also Cloud optical depth variable already exists. Optical depth within each layer should be deprecated.
6. Variable: Cloud Base Height. The definition does not specify if this is height above ground or height above sea level. This should be resolved to avoid confusion and misuse of data in future. Also, the sub-category name on the OSCAR surface website (Height of cloud base) does not match the code list variable name. The category name should be changed to match code list.
7. Variable: Cloud Top Height. The definition does not specify if this is height above ground or height above sea level. This should be resolved to avoid confusion and misuse of data in future. Also, the sub-category name on the OSCAR surface website (Height of cloud top) does not match the code list variable name. The category name should be changed to match code list.

Reason This proposal will better align these cloud variables with the OSCAR requirements.

[fstuerzl]

This issue is related to #190. Regarding 1. and 5.: I discussed this with @joergklausen and the WIGOS code lists for observed variables can be seen as a set of variables with overlapping subsets for OSCAR/Requirements variables and OSCAR/Surface variables. That means the entries "Cloud amount" and "Optical depth within each layer" do not need to be deprecated from the code list.

Changes:	notation	path	name	description
180	\Atmosphere\Clouds\Cloud cover	Cloud cover	3D field of fraction of sky filled by clouds.
181	\Atmosphere\Clouds\Cloud drop effective radius	Cloud drop effective radius	The area weighted mean radius of liquid water drops, assimilated to spheres of the same volume. Considered as both a 3D field throughout the troposphere and a 2D field at the top of cloud surface.
329	\Atmosphere\Clouds\Ice\Cloud ice effective radius	Cloud ice effective radius	The area weighted mean radius of ice particles, assimilated to spheres of the same volume. Considered as both a 3D field throughout the troposphere and a 2D field at the top of cloud surface.
531	\Atmosphere\Clouds\Position\ Cloud Base Height	Cloud Base Height	Height (reference level?) of the bottom surface of the cloud
532	\Atmosphere\Clouds\Position\ Cloud Top Height	Cloud TopHeight	Height (reference level?) of the top of the cloud (highest cloud in case of multi-layer clouds)

[ejwelton]

If "cloud amount" is kept in the code list, it requires a definition. This should be unique, and not the same as "cloud cover" (which is cloud fraction from OSCAR requirements). If "cloud amount" meaning is not defined and unique, then users could select either variable which is opposite to standardization goals. Same comment for "optical depth within each layer". This needs a definition if it will be kept in the list. If it is contained under \Atmosphere\Clouds\ then realize the definition of the existing variable "cloud optical depth" already encompasses "each layer".

[joergklausen]

@fstuerzl Can you please verify if 'cloud amount' originates from the GOS Manual, also what other cloud variables are listed there? I don't know where the cloud variables in OSCAR/Requirements come from, but these terms need to be consolidated and properly defined. @RMaerz and @nuneslf might be able to provide more insight also.

[fstuerzl]

I found the term "Cloud amount" listed in the Guide to the Global Observing System a cloud variable. The table below contains all GOS cloud variables: (see p. 117)

The document does not provide definitions of these terms, but refers to the Guide to Meteorological Instruments and Methods of Observation (WMO-No. 8), in which "cloud amount" is defined as follows:

The amount of sky estimated to be covered by a specified cloud type (partial cloud amount), or by all cloud types (total cloud amount). In either case, the estimate is made to the nearest okta (eighth) and is reported on a scale which is essentially one of the nearest eighth, except that figures 0 and 8 on the scale signify a completely clear and cloudy sky, respectively, with consequent adjustment to the adjacent 1 and 7 okta intervals (see 15.1.4.1).

Definitions in the International Cloud Atlas:

Total cloud cover is the fraction of the sky covered by all the visible clouds. Cloud amount refers to the fraction of the sky covered by clouds of a particular type or combination. It can refer to a genus, species, variety, layer, or a certain combination of clouds.

[joergklausen]

@fstuerzl Please include these definitions in the branch. Please refer to the respective other variable explicitly, i.e.: "Also, see 'cloud amount'"; "Also, see 'cloud cover'" or something similar.

[ejwelton]

Since some variables are to be included that are not in the OSCAR requirements, I would also add to my original proposal to add "cloud phase" as a variable in the WMDR code registry for observedVariableAtmosphere. This is a useful indication of water, ice, or mixed phase of observed clouds. This variable is documented in BUFR: http://codes.wmo.int/bufr4/codeflag/0-20-056 I propose copying this variable and definition/values to WMDR in addition to changes requested above.

[fstuerzl]

Updates:

• According to the Guide to Instruments and Methods of Observation, cloud base and top height are measured above ground.
• @ejwelton, could you include a definition of the variable "Cloud phase" in your proposal?

notation	path	name	description
179	\Atmosphere\Clouds\Cloud amount	Cloud amount	Fraction of the sky covered by clouds of a particular type or combination. It can refer to a genus, species, variety, layer, or a certain combination of clouds. Also, see 'cloud cover'
180	\Atmosphere\Clouds\Cloud cover	Cloud cover	3D field of fraction of sky filled by clouds. Also, see 'cloud amount.
181	\Atmosphere\Clouds\Cloud drop effective radius	Cloud drop effective radius	The area weighted mean radius of liquid water drops, assimilated to spheres of the same volume. Considered as both a 3D field throughout the troposphere and a 2D field at the top of cloud surface.
329	\Atmosphere\Clouds\Ice\Cloud ice effective radius	Cloud ice effective radius	The area weighted mean radius of ice particles, assimilated to spheres of the same volume. Considered as both a 3D field throughout the troposphere and a 2D field at the top of cloud surface.
531	\Atmosphere\Clouds\Position\ Cloud Base Height	Cloud Base Height	Height of the bottom surface of the cloud above ground
532	\Atmosphere\Clouds\Position\ Cloud Top Height	Cloud TopHeight	Height of the top of the cloud (highest cloud in case of multi-layer clouds) above ground
new id	\Atmosphere\Clouds\Cloud Phase	Cloud Phase

View differences in branch: https://github.com/wmo-im/wmds/commit/f5ddee96fd41f4d74d51d0d7a037ed9dd3bc4f6b#diff-160fbee712206a9a655a755c35b0b78b48ab1514bf8937984c6069c5fdf6ba6d

[joergklausen]

Descriptions of 180, 181, 329 above include attributes '3D field of' and '2D field', which should be dropped, and the descriptions updated.

The description for 'cloud phase' could be: Aggregate state of cloud particles, either liquid, ice, or mixed.

[fstuerzl]

Updates:

notation	path	name	description
179	\Atmosphere\Clouds\Cloud amount	Cloud amount	Fraction of the sky covered by clouds of a particular type or combination. It can refer to a genus, species, variety, layer, or a certain combination of clouds. Also, see 'cloud cover'
180	\Atmosphere\Clouds\Cloud cover	Cloud cover	Fraction of sky filled by clouds. Also, see 'cloud amount.
181	\Atmosphere\Clouds\Cloud drop effective radius	Cloud drop effective radius	The area weighted mean radius of liquid water drops, assimilated to spheres of the same volume
329	\Atmosphere\Clouds\Ice\Cloud ice effective radius	Cloud ice effective radius	The area weighted mean radius of ice particles, assimilated to spheres of the same volume.
531	\Atmosphere\Clouds\Position\ Cloud Base Height	Cloud Base Height	Height of the bottom surface of the cloud above ground
532	\Atmosphere\Clouds\Position\ Cloud Top Height	Cloud TopHeight	Height of the top of the cloud (highest cloud in case of multi-layer clouds) above ground
new id	\Atmosphere\Clouds\Cloud Phase	Cloud Phase	Aggregate state of cloud particles, either liquid, ice, or mixed.

View differences in branch: https://github.com/wmo-im/wmds/commit/af8bcce4acbadc91f5e000b896a5bd405ef6eb8d#diff-160fbee712206a9a655a755c35b0b78b48ab1514bf8937984c6069c5fdf6ba6d

[ejwelton]

I concur with the definition for cloud phase.

[IgorZahumensky]

Aggregate state of cloud particles, either liquid, solid (instead of ice), or mixed. OR Aggregate state of cloud particles, either water droplets/drops, snowflakes, ice/snow crystals, or mixed.

[gaochen-larc]

A minor suggestion: should we change effective radius to effective size as I have seen some in-situ measurement investigators reported radius while others reported diameter. Effective size will be more inclusive.

[IgorZahumensky]

• Variable: Cloud Base Height. The definition does not specify if this is height above ground or height above sea level. This should be resolved to avoid confusion and misuse of data in future. Also, the sub-category name on the OSCAR surface website (Height of cloud base) does not match the code list variable name. The category name should be changed to match code list. My comment: The definition from WMO-No. 8, Chapter 15, should be used: "The height of the cloud base is defined as the height above ground level".

• Variable: Cloud Top Height. The definition does not specify if this is height above ground or height above sea level. This should be resolved to avoid confusion and misuse of data in future. Also, the sub-category name on the OSCAR surface website (Height of cloud top) does not match the code list variable name. The category name should be changed to match code list. Only ASL or AMSL has the meaning, not above ground.

[IgorZahumensky]

Or more accurate, according to: https://cloudatlas.wmo.int/en/height-and-altitude.html: Height/Altitude of cloud base: For surface observations, height of the cloud base above ground level; for aircraft observations, altitude of the cloud base above mean sea level.

[joergklausen]

@IgorZahumensky Thanks for your comments that will be taken into account and are greatly appreciated. But you are not supposed to close the issue until the amendment is approved, so I have re-opened it.

[IgorZahumensky]

sorry for closing; done incidentally clicking on the wrong button; I am aware of my task to be done here

[ejwelton]

I think there is a larger issue to consider here, based on Gao's suggestion above on effective radius. I was going to bring this up during the ACV group meeting yesterday but we never made it past agenda item 1 ... 8)

Gao's suggestion to rename effective radius to effective size, and his justification, is at odds with the original approach: namely the variable name and its definition were very specific. I was attempting to follow the original approach where specific definitions were used for many atmospheric comp variables. Same for cloud base and top, whereby we provide a specific definition of height as above ground or altitude. Following Gao's approach, those other variables would just be cloud base and top, and defined as something like "height above ground OR altitude above mean sea level" because various projects report this parameter using both. The more I thought about it, there are many instances of variables in the code lists where some are very specific and others are more vague. The vague definitions capturing a true "concept" rather than an actual, specific variable definition. So I guess this is for @joergklausen, should we be defining these variables as "concepts" as the "type" column implies? or actual, specific variable definitions that might be different from project to project. If the former, then we don't want to be too specific about units and definitions in the code list. This may require a larger scrubbing of atmospheric composition variables than we've even touched on yet in the ACV group.

[IgorZahumensky]

If "cloud amount" is kept in the code list, it requires a definition. This should be unique, and not the same as "cloud cover" (which is cloud fraction from OSCAR requirements). If "cloud amount" meaning is not defined and unique, then users could select either variable which is opposite to standardization goals. Same comment for "optical depth within each layer". This needs a definition if it will be kept in the list. If it is contained under \Atmosphere\Clouds\ then realize the definition of the existing variable "cloud optical depth" already encompasses "each layer".

Definition of the cloud amount: I believe that the definition from the "WMO-No. 8, Vol. I, 15.1.1 Definitions, p. 487) should be used: "The amount of sky estimated to be covered by a specified cloud type (partial cloud amount), or by all cloud types (total cloud amount). In either case, the estimate is made to the nearest okta (eighth) and is reported on a scale which is essentially one of the nearest eighth, except that figures 0 and 8 on the scale signify a completely clear and cloudy sky, respectively, with consequent adjustment to the adjacent 1 and 7 okta intervals (see 15.1.4.1)."

[IgorZahumensky]

@fstuerzl Can you please verify if 'cloud amount' originates from the GOS Manual, also what other cloud variables are listed there? I don't know where the cloud variables in OSCAR/Requirements come from, but these terms need to be consolidated and properly defined. @RMaerz and @nuneslf might be able to provide more insight also.

The "GOS Manual" (WMO-No. 544) was terminated in 2019; Cloud amount originates from The International Cloud Atlas: Manual on the Observation of Clouds and Other Meteors (WMO-No. 407), now an electronic version; the definition of the amount at: https://cloudatlas.wmo.int/en/total-cloud-cover-and-cloud-amount.html

[fstuerzl]

Updates:

notation	path	name	description
179	\Atmosphere\Clouds\Cloud amount	Cloud amount	The amount of sky estimated to be covered by a specified cloud type (partial cloud amount), or by all cloud types (total cloud amount). In either case, the estimate is made to the nearest okta (eighth) and is reported on a scale which is essentially one of the nearest eighth, except that figures 0 and 8 on the scale signify a completely clear and cloudy sky, respectively, with consequent adjustment to the adjacent 1 and 7 okta intervals.
180	\Atmosphere\Clouds\Cloud cover	Cloud cover	Fraction of sky filled by clouds. Also, see 'cloud amount'.
181	\Atmosphere\Clouds\Cloud drop effective radius	Cloud drop effective radius	The area weighted mean radius of liquid water drops, assimilated to spheres of the same volume
329	\Atmosphere\Clouds\Ice\Cloud ice effective radius	Cloud ice effective radius	The area weighted mean radius of ice particles, assimilated to spheres of the same volume.
531	\Atmosphere\Clouds\Position\ Cloud Base Height	Cloud Base Height	For surface observations, height of the cloud base above ground level; for aircraft observations, altitude of the cloud base above mean sea level.
532	\Atmosphere\Clouds\Position\ Cloud Top Height	Cloud TopHeight	Height of the top of the cloud (highest cloud in case of multi-layer clouds) above mean sea level.
new id	\Atmosphere\Clouds\Cloud Phase	Cloud Phase	Aggregate state of cloud particles, either water droplets/drops, snowflakes, ice/snow crystals, or mixed.
508	\Atmosphere\Clouds\Optical properties\Optical depth within each cloud layer	Optical depth within each cloud layer

[fstuerzl]

Branch updated: https://github.com/wmo-im/wmds/blob/issue265/tables_en/1-01-01.csv Please review the proposal in the table above.

[IgorZahumensky]

If "cloud amount" is kept in the code list, it requires a definition. This should be unique, and not the same as "cloud cover" (which is cloud fraction from OSCAR requirements). If "cloud amount" meaning is not defined and unique, then users could select either variable which is opposite to standardization goals. Same comment for "optical depth within each layer". This needs a definition if it will be kept in the list. If it is contained under \Atmosphere\Clouds\ then realize the definition of the existing variable "cloud optical depth" already encompasses "each layer".

Definition of the cloud amount: I believe that the definition from the "WMO-No. 8, Vol. I, 15.1.1 Definitions, p. 487) should be used: "The amount of sky estimated to be covered by a specified cloud type (partial cloud amount), or by all cloud types (total cloud amount). In either case, the estimate is made to the nearest okta (eighth) and is reported on a scale which is essentially one of the nearest eighth, except that figures 0 and 8 on the scale signify a completely clear and cloudy sky, respectively, with consequent adjustment to the adjacent 1 and 7 okta intervals (see 15.1.4.1)."

Two definitions of optical depth/thickness within each cloud layer are offered here: 1) The optical thickness of a cloud is the degree to which the cloud prevents light from passing through it. (https://cloudatlas.wmo.int/en/optical-thickness.html) or 2) The degree to which a cloud modifies the light passing through it (https://glossary.ametsoc.org/wiki/Optical_thickness)

[joergklausen]

@RMaerz @ferrighi Could you please comment on your prefered definition for optical thickness; and the other changes as well in the branch?

[ejwelton]

Optical depth has a specific definition within remote sensing, the one used in the code list already for cloud optical depth is sufficient:

Effective depth of a cloud from the viewpoint of radiation propagation. OD = exp(-K.Δz) where K is the extinction coefficient [km-1] and Δz the vertical path [km] between the base and the top of the cloud.

This is also in agreement with optical depth definition for aerosol. The issue is that the existing definition for cloud optical depth specifies that it is for a cloud layer, not total column cloud optical depth. Thus the other existing variable called "optical depth of a layer" is redundant and should be retired.

I would recommend adding a new variable called total cloud optical depth if this is needed to distinguish layer from total.

[ferrighi]

I have got a review from remote sensing expert (Steinar Eastwood) at METNO. A general comment about the latest list in the table: 180 - since it refers to 179, maybe specify that 180 is in percentage, which is different to 179, which is in octas.

For the optical depth/thickness we would support the first definition: The optical thickness of a cloud is the degree to which the cloud prevents light from passing through it. (https://cloudatlas.wmo.int/en/optical-thickness.html)

We would also be in favor of adding a total cloud optical depth entry to distinguish from cloud layer.

[joergklausen]

@IgorZahumensky @RMaerz Your input is required: 2 different variables for 'cloud optical depth' and 'cloud optical thickness'? If not, which term to use? Description? The WMO Cloud Atlas uses only optical thickness.

What about @ferrighi's request to add 'total cloud optical depth'?

[IgorZahumensky]

My proposal would be to follow the WMO Cloud Atlas, i.e. only optical thickness

[RMaerz]

I would agree that optical thickness is more likely to describe it. The other (depth) might confuse with another traditional used term of something like 'vertical visibility' for measurements i.e. in fog. To have a 'total cloud optical depth' (like @ferrighi requested) would avoid calculating a virtual value , for instance if it needs to be validated against radiation measurements on the surface. So it would be surely useful.

[joergklausen]

@ejwelton Could you weigh in again on depth vs thickness?

[ejwelton]

Optical depth and thickness are used interchangeably in remote sensing, at least for aerosol it is common. However, the exact definition of this term is the integrated extinction coefficient over a defined distance (ie, the definition I have provided above). Using a definition such as "the degree to which a cloud prevents light from passing through it" is something one would expect to see in an elementary school science book. I do not understand the argument here. If a data user is providing optical depth (or call it thickness), then they provide a value (ie number) that is in fact the integrated extinction coefficient over a distance thru the cloud or aerosol layer. Its the same as the more vague description being proposed, so why not use the more specific definition that is already in the WIGOS code list? If there is a desire to try to align with a cloud atlas, then perhaps just add the more vague description to the existing one (ie plain language).

[joergklausen]

I have decided to include both definitions for sake of mathematical rigor and to reflect the official WMO definition.

[amilan17]

@fstuerzl Please update issue summary with final decision. The branch does not currently match the last table in the comments: https://github.com/wmo-im/wmds/issues/265#issuecomment-834169573.

[fstuerzl]

@amilan17, I have included the final proposal in the first comment of this issue.