Fitacf 3.0 as the default?

[mts299]

As we mentioned in the teleconference about reviewing and changing default values. What is the thought on moving make_fit default version to 3.0?

I know SuperDARN Canada is no producing Fitacf 3.0 data for our researchers and people are asking for it.

If this is something we want, I think it will be a simple PR to do.

[ecbland]

I think it's time that FITACF3.0 was made the default, but not before we have addressed #257. Of course FITACF2.5 would still be available using -fitacf-version 2.5. Comments from other users are welcome!

[billetd]

To add to this, we've been looking at how new Borealis data looks under 2.5 and 3.0. I can say that 2.5 consistently produces noisier looking data, for Borealis at least:

[pasha-ponomarenko]

@billetd, this is a known issue with the old noise calculation algorithm. The algorithm is based on averaging 10 lowest values of lag 0 power for a given integration time. If there are considerably more than 10 range gates with the "pure" noise then this approach leads to underestimating the average noise power. This happens because the lowest power values would represent the low-power tail of the noise distribution rather than the whole distribution. In FITACF3 we introduced a correction procedure that also accounts for the number of averages (width of the noise distribution).

[ksterne]

Are we comfortable with this still? Is this something to take to the PIs? I know I've seen examples where there are still some noisy data that fitacf 3.0 isn't filtering out. One of these was thought to be from lightning strikes that we sometimes see in mid-latitude data. I've seen this in some regular radar data as well though. I think Hokkaido West is a prime example as seen in the plots below:

Fitacf 3.0

Fitacf 2.5

It seems as though the noise here is from high spectral width returns that are often over 150 m/s (I expanded the scale on a different plot to subjectively figure this). I think this happens often at Hokkaido West (and East some as well), but I'm not sure about other radars to determine the impact this might have.

I wanted to present what I've seen so we can make an informed choice that we'd be accepting this look to the data. Not knowing much about it, I think this high of spectral width seems like something that wouldn't occur in the ionosphere and is more self-generated noise. If we go with making fitacf 3.0 the default, we'll need to know how to explain some of these strange looking data patterns. Likely something to talk about for an upcoming meeting.

[pasha-ponomarenko]

@ksterne, this is a known issuewith HOK: the excessive "noise" started to occur right after HKW has been commissioned. The reason for that is that the "noise" distribution has a heavy tail which is hard to account for. FITACF2.5 "cleans off" the interference based on empyric criteria which also "kill" lots of valid data.

We discussed this issue with Nozomu in detail, but so far they were unable to fix the problem. Below is our communication with Nozomu during the last SD meeting. The plots show HOK data with HKW being intially on and and then switched off:

---

Hi, Nozomu!

Indeed, there is a dramatic change in the "grainy" noise occurrence for FITACF3 data:

FITACF2.5 data does not show much difference but also shows much less data:

Cheers,

Pasha On 04/06/2019 5:43 a.m., Nozomu Nishitani wrote:

Dear Pasha,

Here is the period when the HKW was stopped (HOK was operating by itself). Start: 04 UT on Nov 21, 2018 End: 02 UT on Jan 10, 2019

Best regards,

Nozomu Nishitani

--

Dr Pavlo (Pasha) Ponomarenko ポノマレンコ パブロ

Senior Analyst, Research Computing Academic and Research Technologies

University of Saskatchewan

Mailing address: Information and Communication Technology 111 Peterson Building 54 Innovation Blvd. Saskatoon, SK, S7N 2V3 CANADA Phone +1 306 966-6458 Email pasha.ponomarenko@usask.ca WWW www.usask.ca/~pasha.ponomarenko ORCID ID: 0000-0001-8407-0193

---

[mtwalach]

Hi everyone,

I have some concerns regarding the noise and I think perhaps we're not quite ready for it becoming "the default" without some caveats. If you are looking at some fitacf data plots (i.e. like the beam plots above) in isolation, fair enough - you get more noise, you get more data and you can see what's what. If you want to process the data further (i.e. gridded & convection maps), then some of that noise is presumable going to carry (has anyone tested this?) and you may or may not get something that looks very different. In addition, all the baseline models (RG96, CS10, TS18 etc.) use FITACF<3, so they may or may not be valid. From a development side, I am not at all against improvements and changes but as a user I do worry about weird results that can't be explained, especially when a lot of work has gone into cross-validating existing SuperDARN data with other datasets. Has anyone done any systematic, large-scale tests comparing the two versions of FITACF for all the radars?

[ecbland]

@ksterne and I have just had a telecon with Kathryn. We have concluded that the Data Standards Working Group would be responsible for making a recommendation to change the default fitting algorithm. Many of us are members of the DSWG so I hope we can continue this conversation in that space!

@mtwalach has raised a very good point about the impacts on the higher-level data products (grid/map) for different fitting algorithms. There have been some large-scale comparisons of FITACF2.5/3.0 (presented at 2018 SD Workshop), but these did not include any comparison of grid/map-level data.

All that being said, I encourage everyone to test the different fitting algorithms on their data, and to share examples where they produce very different results. I'm actually trying to collect some interesting examples so feel free to email me directly if you prefer.

Have a nice weekend!