search

PollyNET / Pollynet_Processing_Chain

NRT lidar data processing program for multiwavelength polarization Raman lidar network (PollyNET)
https://polly.tropos.de/
GNU General Public License v3.0
19 stars 8 forks source link

added option to read IWV values from HATPRO-MWR cloudnet products, now including corrected location subdir #216

Closed ulysses78 closed 6 months ago

ulysses78 commented 1 year ago

This new function adds the option to use IWV cloudnet products from HATPRO-MWR. To use this new function, one has to change the following key/values in the polly_global_config.json file:

"IWV_instrument": "MWR_cloudnet", "MWRFolder": "/data/level1b/cloudnetpy/products/{location}",

This pull request refers to issue https://github.com/PollyNET/Pollynet_Processing_Chain/issues/152

p.s.: now including corrected location subdir

HolgerPollyNet commented 1 year ago

The branch is working without conflict, but no calibration is done by Picasso

ZPYin commented 1 year ago

The branch is working without conflict, but no calibration is done by Picasso

It could be an issue of latency of cloudnet processing. You could test it by running some old PollyXT data with available IWV from cloudnet.

ulysses78 commented 1 year ago

I guess it is not a latency issue. The amount of iwv-data should be sufficient. The error occurs in this line of code in the readIWV.m file: if (tLagStart > p.Results.maxIWVTLag) || (tLagEnd > p.Results.maxIWVTLag)

@ZPYin ... can you tell us the units of the key maxIWVTLag in the polly-config file?

HolgerPollyNet commented 1 year ago

I increased already "maxIWVTLag": 2400,

but still: grafik

ulysses78 commented 1 year ago

So we should printout the tLagStartand the tLagEndvalues to see where we are at the moment.

ulysses78 commented 1 year ago

I just added the disp-fucntion to this variables. so let's see...

ZPYin commented 1 year ago

@ZPYin ... can you tell us the units of the key maxIWVTLag in the polly-config file?

The unit is in days. Default value: 0.0833, equals 2 hours.

HolgerPollyNet commented 6 months ago

@ulysses78 @ZPYin I lost a bit track sorry. Can we merge this still?

ulysses78 commented 6 months ago

This branch is working fine with the new mwr_cloudnet option. But be aware to include the key/value-pair in the correct polly-config file as follows:

"IWV_instrument": "MWR_cloudnet",
"MWRFolder": "/data/level1b/cloudnetpy/products/{location}",

p.s.: of course this works only if a hatpro measurement and the resulting cloudnet-product is available.

p.p.s.: e.g.:


[2024-01-08 10:12:58] Start water vapor calibration

mwrResFilename =

/data/level1b/cloudnetpy/products/leipzig/2023/09/11/20230911_*_hatpro*.nc

mwrResFileSearch =

       name: '20230911_leipzig_hatpro_a217d3ec.nc'
       date: '12-Sep-2023 20:40:17'
      bytes: 127377
      isdir: 0
    datenum: 7.3914e+05

If more than one mwr product was found for one day. E.g. different cloudnet-versions. Only choose the last one for the calibration. This should be the latest cloudnet-versio
n
/data/level1b/cloudnetpy/products/leipzig/2023/09/11/20230911_leipzig_hatpro_a217d3ec.nc
IWV measurement time: 00:30
Closest lidar measurement time: 00:31
IWV measurement time: 01:45
Closest lidar measurement time: 01:46
IWV measurement time: 03:21
Closest lidar measurement time: 03:21
IWV measurement time: 04:21
Closest lidar measurement time: 04:21
Integration top is less than 8000m for water calibration during 20230911 03:51 to 04:50.
IWV measurement time: 05:25
Closest lidar measurement time: 05:26
Integration top is less than 8000m for water calibration during 20230911 04:51 to 05:59.