Finnish VRAD data not dealiased

[nwhpt]

Finnish radar data from the years 2019 and 2020 are not dealiased (i.e. dealiasVRAD=0, but it should be 1), which results in faulty wind (bird) speeds and directions in all Finnish radars. In older data, the dealiasVRAD parameter is 1, as it should be, and winds seem fine. This error concerns only data on the Github repository, not data from the BALTRAD archive which FMI receives back. So the issue does not seem to originate from the Finnish data per se as it is sent to BALTRAD originally, but is probably due to a wrong configuration of the respective parameter when data is processed for Github.

[peterdesmet]

1. The data on GitHub are processed by generic (all countries the same) processing on BALTRAD. Unfortunately that means no exception is made for Finland (to use dealiasVRAD=1 instead of 0). I'll look into this with Gunther.

2. For the processing of the historical data on BALTRAD (2012-2017) we can have more country specific settings, and we will take the dealiasVRAD=1 into account. Would that improve the quality for these data?

[nwhpt]

Thank you for the update. 1) is okay. 2) Historical data is fine, i.e. the problem concerns just data from 2019/2020, so I think older data does not require any changes, but it may not hurt to check. In any case, dealiasVRAD should be 1 in any kind of data.

[adokter]

A fix on baltrad would be to run the latest vol2bird version without an options.conf configuration file, in that case the algorithm figures out for itself whether it should dealias or not

[peterdesmet]

It looks like the dealiased=0 problem was introduced with version 0.3.20:

vp <- read_vpfiles("fianj_vp_20190925T044500Z_0xb.h5")
vp$attributes$how$dealiased
vp$attributes$how$task_version

fianj_vp_20170826224500.h5: 1, 0.3.6
fianj_vp_20180206T1645Z_0x7.h5: 1, 0.3.18
fianj_vp_20190925T044500Z_0xb.h5: 0, 0.3.20
fianj_vp_20200204T144500Z_0xb.h5: 0, 0.3.20
fianj_vp_20210118T145500Z_0xb.h5: 0, 0.3.20

[adokter]

These are the differences between 0.3.18 and 0.3.20: https://github.com/adokter/vol2bird/compare/0.3.18...0.3.20

There has been one commit that changed a Nyquist velocity threshold for dealiasing (https://github.com/adokter/vol2bird/commit/35a05966460cbdd9ac365c43fca75f0e81230339), however it can't explain the Finnish change as it made dealising more frequent.

Specifically, MAX_NYQUIST_DEALIAS was changed from 20 to 25 m/s. This means that in version 0.3.20 and up by default scans will be dealiased if the Nyquist velocity is < 25 m/s, while it used to be < 20 m/s.

@nwhpt what are the Nyquist velocities of the Finnish data typically?

It might be that when vol2bird was updated to 0.3.20 on Baltrad also the options.conf was changed and that it now specifies DEALIAS_VRAD=FALSE, which will suppress dealiasing in all cases.

[nwhpt]

Nyquist for FIN is 7.61 m/s for LPRF and 48.06 m/s for HPRF measurements, but I don't know if bioRad uses the latter at all, these measurements are not executed as often as LPRF.

[peterdesmet]

I have uploaded the current options.conf at https://github.com/enram/globam-dmp/blob/master/data/options.conf

[adokter]

Dealiasing is disabled in the options.conf at this line: https://github.com/enram/globam-dmp/blob/aea56bfd21462ba299e70b46aec7ad8bb864db9b/data/options.conf#L95-L96

[peterdesmet]

So, should that line be remove or be set to TRUE?

[adokter]

It should be removed - in fact I would remove the entire options.conf which seems to be a copy of the example options.conf in the /etc directory of the repo.

e.g. also the wavelength of the radar is hard set in this options.conf which is not necessary.

[peterdesmet]

So advice to Günther to not use an option.conf at all?

[adokter]

Yes I think that is the preferred way to go as long as we keep working with 'one size fits all' settings.

These are the other thresholds involved:

https://github.com/enram/globam-dmp/blob/aea56bfd21462ba299e70b46aec7ad8bb864db9b/data/options.conf#L71-L75

• Scans with a Nyquist velocity below MIN_NYQUIST_DEALIAS will be discarded altogether. This is because countries (like the Netherlands) often include one or two scans with collected with a single pulse repetition frequency (PRF) and therefore very low nyquist velocity that are used to detect precipitation as far as possible, those scans are not really relevant for us because of the poor radial velocity data.
• Any scan with a Nyquist velocity above MAX_NYQUIST_DEALIAS will not be considered for dealiasing.

So if a polar volume has only a single scan with a Nyquist velocity between MIN_NYQUIST_DEALIAS and MAX_NYQUIST_DEALIAS it will trigger the dealiasing algorithm, which will affect the resulting speeds and directions in all height layers

One risk of removing options.conf altogether is that we may trigger dealiasing also in other radars that may not really need it. This would be good to assess after the change, and to double-check that especially the default MIN_NYQUIST_DEALIAS to throw out single-PRF scans is adequate for as many radars as possible

[peterdesmet]

vol2bird has been updated on BALTRAD since 25 January 2021 16 UTC. The new files are dealiased:

> vp <- read_vpfiles("fivim_vp_20210126T005500Z_0xb.h5")
> vp$attributes$how$task_version
[1] "0.5.0.9169"
> vp$attributes$how$dealiased
[1] 1

Closing this issue.

[peterdesmet]

@nwhpt I'm reopening this issue based on the meeting we had. If I understand this issue, Finnish radar data should be dealiased (1):

• [ ] 2012-2017 archive data (currently being processed, to be uploaded in repo): I'll look if data are dealiased
• [ ] 2018-2020 existing data: apparently not ok (https://github.com/enram/data-repository/issues/98). Difficult to fix, because we don't have the original data (but could request it)
• [x] 2021+ data: should be ok, since we requested Günther to not use options.conf (https://github.com/enram/data-repository/issues/98)

[peterdesmet]

The data from 2012-2016 look ok. I currently only have empty data from 2017:

library(bioRad)
#> Welcome to bioRad version 0.5.2.9419
#> Warning: Docker daemon is not running
#> Warning: bioRad functionality requiring Docker has been disabled
#> 
#> To enable Docker functionality, start Docker and run 'check_docker()'
#> in R
vp_2012 <- read_vpfiles("~/Downloads/finland_vp_20120126/fianj_vp_20120126T1535Z.h5")
vp_2012$attributes$how$task_version
#> [1] "0.5.0"
vp_2012$attributes$how$dealiased
#> [1] 1

vp_2013 <- read_vpfiles("~/Downloads/finland_vp_20130924/fikuo_vp_20130924T1645Z.h5")
vp_2013$attributes$how$task_version
#> [1] "0.5.0"
vp_2013$attributes$how$dealiased
#> [1] 1

vp_2014 <- read_vpfiles("~/Downloads/finland_vp_20140113/fivim_vp_20140113T2215Z.h5")
vp_2014$attributes$how$task_version
#> [1] "0.5.0"
vp_2014$attributes$how$dealiased
#> [1] 1

vp_2015 <- read_vpfiles("~/Downloads/finland_vp_20150317/fiuta_vp_20150317T0145Z.h5")
vp_2015$attributes$how$task_version
#> [1] "0.5.0"
vp_2015$attributes$how$dealiased
#> [1] 1

vp_2016 <- read_vpfiles("~/Downloads/finland_vp_20160806/fipet_vp_20160806T2030Z.h5")
vp_2016$attributes$how$task_version
#> [1] "0.5.0"
vp_2016$attributes$how$dealiased
#> [1] 1

^{Created on 2022-03-17 by the reprex package (v2.0.1)}