search

PollyNET / Pollynet_Processing_Chain

NRT lidar data processing program for multiwavelength polarization Raman lidar network (PollyNET)
https://polly.tropos.de/
GNU General Public License v3.0
20 stars 8 forks source link

Near-real time processing of quasi-products corrupted #135

Closed HolgerPollyNet closed 2 years ago

HolgerPollyNet commented 2 years ago

During the operational processing, it seems that the latest quasi products are empty, and the target categorization is false, while the input,- the att. bsc's and volume depol- is ok!

For example: grafik grafik

grafik grafik grafik

Any idea what could be the problem? Is it due to the transfer of data within the chain?

ZPYin commented 2 years ago

It's not a bug with Picasso, but something wrong with the latest GDAS1 profiles. Below is the GDAS1 profile for leipzig:

image

I don't know why the temperature (TEMP) and height (HGTS) are both zeros. Anyhow, this will feed in invalid meteorological data without triggering any warnings from Picasso. Therefore, all products associated with Rayleigh scattering properties will be wrong, to name a few, particle backscatter, quasi-backscatter/depol.

I enahnced the check for meteorological data to filter such invalid gdas1 profiles (see commit https://github.com/PollyNET/Pollynet_Processing_Chain/commit/3f02c5b7d62cfb62ee74eadd0b16c82b7bd33c57). But maybe someone still needs to dive into the GDAS1 profile issue.

HolgerPollyNet commented 2 years ago

Many thanks! I will inform our GDAS folks! Best

HolgerPollyNet commented 2 years ago

Hi @ZPYin ,

since you enhanced your check for invalid GDAS files, only standard atmosphere data is used even though the GDAS profiles are available. Can you check?

ZPYin commented 2 years ago

Hi @ZPYin ,

since you enhanced your check for invalid GDAS files, only standard atmosphere data is used even though the GDAS profiles are available. Can you check?

My last implementation was incorrect. The condition check that I used to resolve the original issue, leads to this new bug. The details in terms of the bug and the following-up fix can be found in commit https://github.com/PollyNET/Pollynet_Processing_Chain/commit/c8af058fa867c887f9e4d649b616363eed01d9df.

Ps, Fixing issues or bugs can get tricky as the increasing of the project volume.

HolgerPollyNet commented 2 years ago

Many thanks @ZPYin , I really appreciate the efforts you are still doing for us. In future we will of course try to more and more fix the bugs by ourselves. But your advise is always welcome, as it might be much easier for you to get an idea were the bug us located!