search

PollyNET / Pollynet_Processing_Chain

NRT lidar data processing program for multiwavelength polarization Raman lidar network (PollyNET)
https://polly.tropos.de/
GNU General Public License v3.0
20 stars 8 forks source link

WV Calibration - AERONET #59

Closed HolgerPollyNet closed 4 years ago

HolgerPollyNet commented 4 years ago

i realized, that for PollyXT TROPOS no watervapour calbration was performed since months even though Aeronet WV is availale. Is it easy to check, wether the connection to the aeronet website works or if an errror occured?

ZPYin commented 4 years ago

I checked this issue, but found out there were five calibration results on March and May. So, this at least says the water vapor calibration module works.

id,cali_start_time,cali_stop_time,standard_instrument,standard_instrument_meas_time,wv_const,uncertainty_wv_const,nc_zip_file,polly_type 1,2020-03-23 04:00:00,2020-03-23 04:59:30,AERONET,2020-03-23 06:26:42,44.529595,0,2020_03_23_Mon_TROPOS_00_00_01.nc,PollyXT_TROPOS 2,2020-03-24 04:00:00,2020-03-24 04:59:30,AERONET,2020-03-24 06:23:04,30.965185,0,2020_03_24_Tue_TROPOS_00_00_01.nc,PollyXT_TROPOS 3,2020-03-25 04:00:00,2020-03-25 04:59:30,AERONET,2020-03-25 06:21:04,56.396246,0,2020_03_25_Wed_TROPOS_00_00_01.nc,PollyXT_TROPOS 4,2020-04-04 18:00:00,2020-04-04 18:59:30,AERONET,2020-04-04 16:30:59,36.943473,0,2020_04_04_Sat_TROPOS_18_00_01.nc,PollyXT_TROPOS 5,2020-04-05 18:00:00,2020-04-05 18:59:30,AERONET,2020-04-05 16:30:21,43.137646,0,2020_04_05_Sun_TROPOS_18_00_01.nc,PollyXT_TROPOS

Another issue is why there were so few calibration results. This is partly caused by my last change to the water vapor calibration module. I added a stricter mask for filtering daytime water vapor measurement, namely filtering out profiles between 50 min before sunrise and 50 min after sunset (compared with just after sunrise and before sunset for the previous version)

https://github.com/PollyNET/Pollynet_Processing_Chain/blob/4d25962a2fa587376bdf3b13e278c60f9139ba2e/lib/polly_general_func_lib/pollyxt_wv_calibration.m#L102-L103

I did this change with the thinking of removing the influence of solar background on the calibration. At twilight, the signal at water vapor channel already starts to be covered up by solar background (see figure below, sunrise: 03:23 UTC, civil twilight: 02:42-03:23 UTC):

image

The large background will deteriorate the SNR and bring large uncertainty to the calibration results. That's why I found out some very deviated resuts for the previous version of algorithm.

However, after adopt this change, the calibration condition are harder to be met, taking into account there was another constraint of the maximun lag time between AERONET and PollyXT measurement of 2 hours. But anyhow, there always exists the trade-off between the stability of calibration results and the frequency of successful calibration periods.

At present version, I prefer choosing the stability. But it's easy to be switched back if you want more calibration results.

HolgerPollyNet commented 4 years ago 
hey Zhenping,

thanks for the feedback.

So in case I want to change something I can modify "maxIWVTLag" to
change the two hour prerequisite?
But is there a setting where I can change the 50 minutes?

Am 12.05.2020 um 15:04 schrieb ZPYin:

  I checked this issue, but found out there were five calibration
    results on March and May. So, this at least says the water vapor
    calibration module works.

    id,cali_start_time,cali_stop_time,standard_instrument,standard_instrument_meas_time,wv_const,uncertainty_wv_const,nc_zip_file,polly_type
      1,2020-03-23 04:00:00,2020-03-23 04:59:30,AERONET,2020-03-23
      06:26:42,44.529595,0,2020_03_23_Mon_TROPOS_00_00_01.nc,PollyXT_TROPOS
      2,2020-03-24 04:00:00,2020-03-24 04:59:30,AERONET,2020-03-24
      06:23:04,30.965185,0,2020_03_24_Tue_TROPOS_00_00_01.nc,PollyXT_TROPOS
      3,2020-03-25 04:00:00,2020-03-25 04:59:30,AERONET,2020-03-25
      06:21:04,56.396246,0,2020_03_25_Wed_TROPOS_00_00_01.nc,PollyXT_TROPOS
      4,2020-04-04 18:00:00,2020-04-04 18:59:30,AERONET,2020-04-04
      16:30:59,36.943473,0,2020_04_04_Sat_TROPOS_18_00_01.nc,PollyXT_TROPOS
      5,2020-04-05 18:00:00,2020-04-05 18:59:30,AERONET,2020-04-05
      16:30:21,43.137646,0,2020_04_05_Sun_TROPOS_18_00_01.nc,PollyXT_TROPOS

  Another issue is why there were so few calibration results.
    This is partly caused by my last change to the water vapor
    calibration module. I added a stricter mask for filtering
    daytime water vapor measurement, namely filtering out profiles
    between 50 min before sunrise and 50
      min after sunset (compared with just after sunrise
    and before sunset for the previous version)
  https://github.com/PollyNET/Pollynet_Processing_Chain/blob/4d25962a2fa587376bdf3b13e278c60f9139ba2e/lib/polly_general_func_lib/pollyxt_wv_calibration.m#L102-L103
  I did this change with the thinking of removing the influence
    of solar background on the calibration. At twilight, the signal
    at water vapor channel already starts to be covered up by solar
    background (see figure below, sunrise: 03:23 UTC, civil
    twilight: 02:42-03:23 UTC):

  The large background will deteriorate the SNR and bring large
    uncertainty to the calibration results. That's why I found out
    some very deviated resuts for the previous version of algorithm.
  However, after adopt this change, the calibration condition are
    harder to be met, taking into account there was another
    constraint of the maximun lag time between AERONET and PollyXT
    measurement of 2 hours. But anyhow, there always exists the
    trade-off between the stability of calibration results and the
    frequency of successful calibration periods.
  At present version, I prefer choosing the stability. But it's
    easy to be switched back if you want more calibration results.
  —
    You are receiving this because you authored the thread.
    Reply to this email directly, view it on GitHub, or unsubscribe.
  [

{ "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/PollyNET/Pollynet_Processing_Chain/issues/59#issuecomment-627329981", "url": "https://github.com/PollyNET/Pollynet_Processing_Chain/issues/59#issuecomment-627329981", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

--

Dr. Holger Baars (Dipl.-Met.) Leibniz Institute for Tropospheric Research (TROPOS) Permoserstr. 15 04318 Leipzig, Germany phone: ++49 341 2717 7314 fax: ++49 341 2717 99 7314 email: baars@tropos.de internet: http://www.tropos.de

ZPYin commented 4 years ago

Solved in PR #60