search

wmo-im / wmds

WIGOS Metadata Standard: Semantic standard and code tables
16 stars 22 forks source link

5-02-01 Lidar in Measurement/Observing Method (atmosphere) #298

Closed fstuerzl closed 2 years ago

fstuerzl commented 3 years ago

Branch

https://github.com/wmo-im/wmds/tree/issue298

Summary and Purpose

Follow-up on #266

Proposal

Deprecate

notation name description Path
243 Lidar for aerosol observations Light detection and ranging (Lidar) profiler for aerosol observations \Atmosphere\Aerosol\Remote-sensing, active\Optical properties\Lidar for aerosol observations
324 Lidar for visibility and optical range Light detection and ranging (Lidar) for observations of visibility and optical range \Atmosphere\Visual range\Remote-sensing, active\Laser-based methods\Lidar for visibility and optical range

Supersede

notation name description Path supersede by
269 Lidar for aerosol observations Light detection and ranging (Lidar) profiler for aerosol observations \Atmosphere\Aerosol\Remote-sensing, active\Optical properties\Lidar for aerosol observations 143 (see table below)

Modify

  notation name description tags
Current ~142~ ~Lidar for wind~ ~Light detection and ranging (Lidar) profiler for wind observations~ ~\Atmosphere\Wind\Remote-sensing, active\Lidar for wind~
New 142 Doppler wind lidar Light detection and ranging (Lidar) doppler wind capability (direct and coherent mode), typically used for measurements of winds. Atmosphere, 'Remote-sensing, active', wind
Current ~143~ ~Lidar for humidity~ ~Light detection and ranging (Lidar) for humidity measurements~ ~\Atmosphere\Humidity\Remote-sensing, active\Lidar for humidity~
New 143 Raman lidar Light detection and ranging (lidar) with Raman capability, typically used for measurements of aerosol particle phase and cloud measurements as well as water vapor and temperature. Atmosphere, 'Remote-sensing, active', water vapor, aerosol, particle phase
Current ~320~ ~DIAL for trace gas observations~ ~Differential absorption Lidar (DIAL) for trace gas observations~ ~\Atmosphere\Trace gas\Remote-sensing, active\Laser-based methods\DIAL for trace gas observations~
New 320 Integrated path differential absorption (IPDA) lidar Light detection and ranging (lidar) with IPDA capacity, typically used for column measurements of CH4 and CO2. Atmosphere, 'Remote-sensing, active', greenhouse gas
Current ~335~ ~DIAL for ozone observations~ ~Differential absorption Lidar (DIAL) for ozone measurements~ ~\Atmosphere\Ozone\Remote-sensing, active\Laser-based methods\DIAL for ozone observations~
New 335 Differential absorption lidar (DIAL) Light detection and ranging (lidar) with DIAL capability, typically used for trace gas measurements, most notably ozone and water vapor. This technique can be extended to molecules which have suitably defined differential absorption characteristics. Atmosphere, 'Remote-sensing, active', ozone, water vapor

New

notation name description tags
341 Backscatter lidar Elastic backscatter light detection and ranging (lidar) typically used for measurements of cloud and aerosol particle phase properties, altimetry, ocean properties, and mixed layer height. Atmosphere, 'Remote-sensing, active', cloud, aerosol, particle phase, altimetry, Ocean, mixed-layer height
342 High spectral resolution (HSR) lidar Light detection and ranging (lidar) with high spectral resolution capability, typically used for measurements of cloud and aerosol particle phase properties, ocean properties, and mixed layer height. Atmosphere, 'Remote-sensing, active', cloud, aerosol, particle phase, mixed-layer height, Ocean
343 Polarized lidar Light detection and ranging (lidar) capable of determining linear or circular depolarization ratio, typically used for measurements of particle shape and orientation. Atmosphere, 'Remote-sensing, active', aerosol, particle phase

Stakeholder(s)

@gaochen-larc, @ewelton, lidar community 

Consultations

[include names and emails or handles of individuals or groups that reviewed the proposal]

Judd Welton @ejwelton.@ejwelton, John Sullivan (john.t.sullivan@nasa.gov), John Hair (johnathan.w.hair@nasa.gov); Amin Nehrir (amin.r.nehrir@nasa.gov)

Context

[include references to manuals or guides that are reviewed to ensure alignment, if proposal differs then document how and why]

Expected Impact of Change

HIGH

To implement this change in OSCAR/Surface a migration plan is needed. The deprecated and superseded codes can only be removed from the method list in OSCAR, if they are not included in station records. This requires help from the lidar community.

Moreover the new or modified codes introduce a tag approach to the method table instead of using paths. This cannot implemented in OSCAR at the moment. It does not prevent the inclusion of the changes completely, but results in a different placement in the "method tree" than the tags suggest.

gaochen-larc commented 3 years ago

Agree with the general approach, but have two questions: 1) should we add HSRL to the Measurement Principle? This was mentioned in issue #266. I also consulted with a few lidar researchers. 2) not sure if the retrieval information would benefit the general research users? We may need further discussion.

joergklausen commented 3 years ago

I think ´high spectral resolution lidar (HSRL)´ can and should be added. Can we see the current branch, please to decide what's still missing? I suspect, the bigger issue to resolve is the path (which we still need at present). I believe we/I made an error at the time to preceed '/Remote-Sensing, active' with a variable (wind, temperature, etc). Let's explore if this can be dropped, and establish a clean branch based on a path \Atmosphere\Remote-sensing, active\Optical properties\Light detection and ranging (Lidar)\

Would this work?

amilan17 commented 3 years ago

Gao discussed options for categorizing lidar based on measurements or techniques. Reach out to Judd Welton @ejwelton.

ejwelton commented 3 years ago

Can someone provide me with an update on the status of the original issue on this topic (#266 ). It appears to have made it thru the last fast track, but the actual code list does not show those changes. I will have to review some of this material before I can respond to Jorg's latest comment. We took so long off over the summer that I need to refresh my memory.

ejwelton commented 3 years ago

Here are the current lidar entries in the ObservingMethodAtmosphere code list: DIAL, 320 DIAL, 335 LIDAR, 143 LIDAR, 324 LIDAR, 142 LIDAR, 243 LIDAR, 269 My original issue (#266 ) attempted to clarify the meanings of the multiple LIDAR entries, and also the two DIAL entries. I do not know the status of the original issue, but based on plans with this new issue it seems that the older lidar entries will be removed and replaced? is that correct?

If so, then the proposed changes from @joergklausen above will be helpful. I like the new path suggestion as well, elevating Remote Sensing, active to just under Atmosphere. I would recommend removing the retrieval information from the lidar descriptions. It is not needed here and adds confusion (as one can use the Fernald retrieval with HSRL data, in fact most any lidar type). Also, the DIAL entries still need to be addressed.

I suggest:

It would be good to have others review above, especially the proposed DIAL, ozone, and wind lidar descriptions. Those are more outside my area of expertise. Ozone lidars utilize the DIAL technique, which is also used for temperature and water vapor profiling. So I'm not sure it makes sense to specify ozone lidar separately, but they are so prevalent it seems it should be specified.

amilan17 commented 3 years ago

Can someone provide me with an update on the status of the original issue on this topic (#266 ). It appears to have made it thru the last fast track, but the actual code list does not show those changes.

@ejwelton - codes in #266 were adopted by WMO, but https://codes.wmo.int is not updated yet. It will be soon. In the meantime, you can see the adopted codes in the Amendment doc available from https://community.wmo.int/activity-areas/wis/wis-manuals

gaochen-larc commented 3 years ago

@ejwelton, I followed up with what you proposed and made a few minor changes after talking to John H and Armin N:

• Backscatter Lidar, Elastic backscatter light detection and ranging (lidar) typically used for measurements of cloud and particulate properties, altimetry, ocean properties, and mixed layer height
• Raman Lidar, Light detection and ranging (lidar) with Raman capability, typically used for measurements of aerosol and cloud measurements as well as water vapor and temperature • HSR Lidar, Light detection and ranging (lidar) with high spectral resolution capability, typically used for measurements of cloud and aerosol properties, ocean properties, and mixed layer height. • Polarized Lidar, Light detection and ranging (lidar) capable of determining linear or circular depolarization ratio, typically used for measurements of particle shape and orientation • Differential Absorption Lidar, Light detection and ranging (lidar) with DIAL capability, typically used for measurements of O3, water vapor. This technique can be extended to molecules which have appropriate strength absorption lines that overlap laser transmitter and detector technologies needed to employ these measurements. • Integrated Path Differential Absorption lidar, light detection and ranging (lidar) with IPDA capacity, typically used for column measurements of CH4 and CO2. • Doppler Wind Lidar, Light detection and ranging (lidar) with doppler wind capability (direct and coherent mode), typically used for measurements of winds

Will this work? please feel free to make edits

ejwelton commented 3 years ago

@gaochen-larc I think we have iterated to a final solution. Looks good to me, however we need to use consistent language in the instrument descriptions with aerosol, particulate, etc... Especially given the changes we are making with aerosol terminology. I propose using the term "aerosol particle phase" in above definitions to replace "particulate" and "aerosol" terms.

gaochen-larc commented 3 years ago

@gaochen-larc I think we have iterated to a final solution. Looks good to me, however we need to use consistent language in the instrument descriptions with aerosol, particulate, etc... Especially given the changes we are making with aerosol terminology. I propose using the term "aerosol particle phase" in above definitions to replace "particulate" and "aerosol" terms.

Agree with you @ejwelton. We should make changes and use "aerosol particle phase"

gaochen-larc commented 3 years ago

• Backscatter Lidar, Elastic backscatter light detection and ranging (lidar) typically used for measurements of cloud and aerosol particle phase properties, altimetry, ocean properties, and mixed layer height • Raman Lidar, Light detection and ranging (lidar) with Raman capability, typically used for measurements of aerosol particle phase and cloud property measurements as well as water vapor and temperature • HSR Lidar, Light detection and ranging (lidar) with high spectral resolution capability, typically used for measurements of cloud and aerosol particle phase properties, ocean properties, and mixed layer height. • Polarized Lidar, Light detection and ranging (lidar) capable of determining linear or circular depolarization ratio, typically used for measurements of aerosol particle phase shape and orientation • Differential Absorption Lidar, Light detection and ranging (lidar) with DIAL capability, typically used for measurements of O3, water vapor. This technique can be extended to molecules which have appropriate strength absorption lines that overlap laser transmitter and detector technologies needed to employ these measurements. • Integrated Path Differential Absorption lidar, light detection and ranging (lidar) with IPDA capacity, typically used for column measurements of CH4 and CO2. • Doppler Wind Lidar, Light detection and ranging (lidar) with doppler wind capability (direct and coherent mode), typically used for measurements of winds

This is the final version. I hope I made all the changes in the right places... need additional pairs of eyes to make sure @ejwelton @fstuerzl @joergklausen

ejwelton commented 3 years ago

@fstuerzl we have a solution, see below: • Backscatter Lidar, Elastic backscatter light detection and ranging (lidar) typically used for measurements of cloud and aerosol particle phase properties, altimetry, ocean properties, and mixed layer height • Raman Lidar, Light detection and ranging (lidar) with Raman capability, typically used for measurements of aerosol particle phase and cloud measurements as well as water vapor and temperature • HSR Lidar, Light detection and ranging (lidar) with high spectral resolution capability, typically used for measurements of cloud and aerosol particle phase properties, ocean properties, and mixed layer height. • Polarized Lidar, Light detection and ranging (lidar) capable of determining linear or circular depolarization ratio, typically used for measurements of particle shape and orientation • Differential Absorption Lidar, Light detection and ranging (lidar) with DIAL capability, typically used for measurements of O3, water vapor. This technique can be extended to molecules which have appropriate strength absorption lines that overlap laser transmitter and detector technologies needed to employ these measurements. • Integrated Path Differential Absorption lidar, light detection and ranging (lidar) with IPDA capacity, typically used for column measurements of CH4 and CO2. • Doppler Wind Lidar, Light detection and ranging (lidar) with doppler wind capability (direct and coherent mode), typically used for measurements of winds

ejwelton commented 3 years ago

Ha we posted the same time. They are the same so can use either.

joergklausen commented 3 years ago

Congrats, happy to see that this is approaching the finish line :-)

amilan17 commented 3 years ago

confirm with ET on Upper Air Measurements

joergklausen commented 3 years ago

Proposal updated as per above. @amilan17 Please create branch. @gaochen-larc, @ewelton Please confirm branch once established. Also, please obtain 2 additional confirmations from lidar experts, e.g., from ET on Upper Air Measurements.

ejwelton commented 3 years ago

@gaochen-larc, lets each get one external lidar expert confirmation. Sound good?

gaochen-larc commented 3 years ago

@gaochen-larc, lets each get one external lidar expert confirmation. Sound good?

Sure, I will get either John Hair or Amin N

amilan17 commented 3 years ago

Meeting #16 notes:

ejwelton commented 3 years ago

I have comments back from Lucia Mona, my fellow GALION co-chair, head of EARLINET, and she is also on the Aerosol SAG. She said that entries are ok, but we are missing fluorescence lidar. I recommend that be added, she said EARLINET is working on this. Can we still add additional lidar type? If so, I will ask her to draft a sample entry and include here, I am not an expert on this type of lidar so not best to do that.

ejwelton commented 3 years ago

Also, I have a question on the use of these new lidar entries. When I first proposed changing the code list for lidar, I envisioned a listing like what we are working on here. My thought was to have users select multiple lidar types to describe more complicated instruments. For instance, most Raman lidars provide both backscatter lidar capability and Raman capability. The Raman techinque is not useful all the time or for some altitudes, and instead the backscatter technique would be used. I want to check that is acceptable to include more than one observingMethod per observation? Also how? Multiple deployed equipment elements, each with a different observing method? Or one deployed equipment element, with multiple observing methods?

joergklausen commented 3 years ago

One can use only one method for an observation. I would argue, if you observe a quantity using different methods, these are different observations. Addition of fluorescence Lidar would better be deferred to the next FT, unless there is an obvious urgency.

gaochen-larc commented 3 years ago

Comments from John Sullivan (john.t.sullivan@nasa.gov):

Gao, I think the DIAL lidar text could be improved. The hardware portion of this text just reads a bit awkward.

Current text: Light detection and ranging (lidar) with DIAL capability, typically used for measurements of O3, water vapor. This technique can be extended to molecules which have appropriate strength absorption lines that overlap laser transmitter and detector technologies needed to employ these measurements.

Proposed text: Light detection and ranging (lidar) with DIAL capability, typically used for trace gas measurements, most notably ozone and water vapor. This technique can be extended to molecules which have suitably defined differential absorption characteristics.

gaochen-larc commented 3 years ago

Comments from John Hair (johnathan.w.hair@nasa.gov):

  1. The proposed general lidar types and descriptions are reasonable. Also agree with John Sullivan’s proposed DIAL description change
  2. While these general lidar types probably can be used to described the prominent capability of most of the current lidar systems, many systems have multiple capabilities. For example, we deployed a system with both DIAL and HSRL capabilities to make profile measurements of ozone and aerosol properties. Our HSRL systems also have Polarized lidar capability to measure aerosol depolarization ratio
  3. It will require more than one general lidar type to properly describe the measurements of certain variables, e.g., aerosol depolarization ratio.
gaochen-larc commented 3 years ago

One can use only one method for an observation. I would argue, if you observe a quantity using different methods, these are different observations. Addition of fluorescence Lidar would better be deferred to the next FT, unless there is an obvious urgency.

I agree to defer the addition of fluorescence Lidar to the next FT.

gaochen-larc commented 3 years ago

Comments from Amin Nehrir (amin.r.nehrir@nasa.gov):

I agree with John Hair and John Sullivan’s comments on DIAL and HSRL. I’ll note that resonance Fluorescence lidar is a prominent technique that is used to understand upper atmospheric dynamics. This technique has been used for many decades. That said, deferring this technique to a later time is ok.

amilan17 commented 3 years ago

@gaochen-larc Can you please update the branch with latest proposal? And are you able to update the initial proposal as well? 

Have you or anyone reviewed the [Guide to Instrument and Methods of Observations] ?

gaochen-larc commented 3 years ago

figuring out how to update the branch. I checked [Guide to Instrument and Methods of Observations] and did not find significant discrepancies. One difference I found is "depolarization lidar" in the guide and "Polarized lidar" proposed here. However, they are not quite the same thing. However, I am not a lidar expert. @ejwelton comments?

gaochen-larc commented 3 years ago

branch updated

ejwelton commented 2 years ago

@gaochen-larc Your comments above are a nomenclature problem. Below is my attempt to clarify confusion.

A Polarized lidar is an instrument that emits laser pulses with distinct polarization (linear or elliptical). The returned attenuated backscatter signals are themselves polarized to some degree based on the original polarization of the emitted laser pulse and the shape of the scattering particle. Some degree of depolarization occurs from the scattering event (based on particle shape) and polarized lidar receivers are designed to measure this quantity from the returned signals. Thus a "Polarized" lidar will provide measurements of particle and molecular "depolarization" ratio.

By contrast, a depolarized lidar is one that first depolarizes the laser pulses prior to emission from the transmitter. Thus the returned attenuated backscatter signals contain no information on particle shape because the signals themselves also come back depolarized. So a "depolarized" lidar will not actually be able to measure particle and molecular "depolarization" ratio.

I've some people refer to a depolarization lidar before, and without further clarification its unclear to me what they might mean. One could interpret that as a lidar capable of measuring depolarization or not. I'm assuming that the guide you examined is using depolarization lidar to indicate a lidar that measured depolarization ratio (which here I proposed calling a polarized lidar).

gaochen-larc commented 2 years ago

@ejwelton thank you! I think we can close this issue for now. Any objections?

amilan17 commented 2 years ago

@gaochen-larc - I requested that a member from the Expert Team on Upper Air Measurements to also review. I'd like to wait for their response.

joergklausen commented 2 years ago

@amilan17 At this point, we need a name of a specific individual from this team (or the chair) to respond quickly. Who is this? The consulted experts so far are all NASA, but they are outstanding experts in the field. If we don't get a formal confirmation from the ET Upper Air, I think, we should still proceed. This proposal is very mature.

ejwelton commented 2 years ago

@joergklausen Your statement is not true. I provided feedback from Lucia Mona (EARLINET head, my GALION co-chair, and on the Aerosol SAG). See post from 21 days ago. She said that the proposed lidar descriptions were ok, but we are missing fluorescence lidar.

I can get comments from other GALION reps if needed.

joergklausen commented 2 years ago

@ejwelton Oops, yes, and that‘s great!

amilan17 commented 2 years ago

We received feedback from the ET-UAM 

https://wmoomm.sharepoint.com/:w:/s/wmocpdb/Ede963uIBk5PiyQgl5v9BmQBtvWVArkzmzZmrfjwaivsTQ?e=seixer

amilan17 commented 2 years ago

@gaochen-larc, per the last meeting (https://github.com/wmo-im/wmds/wiki/2021-12-16-TT-WIGOSMD-17) you were going to "review the table changes from ET-UAM and come back with a report including what is to be accepted or rejected." Have you had a chance to do so? If not, will you be able to do so this week or early next week?

joergklausen commented 2 years ago

Comments to feedback from ET-UAM received from @gaochen-larc and Jonathan Hair (Nasa). Based on this, following slight adaptation to the proposal:

notation name description tags
341 Backscatter lidar Elastic backscatter light detection and ranging (lidar) typically used for measurements of cloud and aerosol particle phase optical properties (e.g., extinction and backscatter coefficient), altimetry, ocean optical properties, and mixed layer height. Atmosphere, 'Remote-sensing, active', cloud, aerosol, particle phase, altimetry, Ocean, mixed-layer height
342 High spectral resolution (HSR) lidar Light detection and ranging (lidar) with high spectral resolution capability, typically used for measurements of cloud and aerosol particle phase properties, ocean properties, and mixed layer height. Atmosphere, 'Remote-sensing, active', cloud, aerosol, particle phase, mixed-layer height, Ocean
343 Polarized lidar Light detection and ranging (lidar) capable of determining the linear or circular depolarization ratio, typically used to discriminate spherical and non-spherical particles. Atmosphere, 'Remote-sensing, active', aerosol, particle phase, particle shape

Based on this consolidation, the branch and the section 'New' of the proposal can be updated (@fstuerzl ?) and confirmed by ET-UAM (@amilan17 ?) and @gaochen-larc, please.

gaochen-larc commented 2 years ago

Could we move the right parenthesis ")" from the end of the "Backscatter lidar" description to after the word "coefficient"? It should read like:

Elastic backscatter light detection and ranging (lidar) typically used for measurements of cloud and aerosol particle phase optical properties (e.g., extinction and backscatter coefficient), altimetry, ocean optical properties, and mixed layer height.

Thanks!

joergklausen commented 2 years ago

Yes, sorry, my bad! I have fixed this.