Open egthomas opened 1 year ago
@egthomas , from years ago my recollection is that there was some serious technical problem with removing the existing fields, but I never knew the exact details. I think then we were talking about removing fields related to XFC power fitting.
@pasha-ponomarenko I removed all the unused XCF fields for fitacf3
files in #472, so there shouldn't be any technical issue here. I'm all in favour of @egthomas's suggestion to remove the unused fields.
Gee, I am getting senile! 🤔 Of course, please go ahead with removing these fields.
On a somewhat related note, I just realized that grid (and map) files use the noise.search
parameter to calculate the noise_mean
and noise_sd
values. Not all radars seem to record noise.search
values, so maybe this should be changed to something else like sky noise?
Yep, this is probably a better solution. I was always uneasy with using the search noise level in conjunction with FITACF data as its magnitude is established using a very different algorithm and much shorter integration time. I believe that originally there was some magic scaling coefficient applied to make the search noise compatible with that estimated from low-power ACFs.
@pasha-ponomarenko thanks for the response - I've just opened a pull request to change the underlying library to use the sky noise value rather than the search noise.
(I haven't yet had the time to examine removing the noise.lag0
and noise.vel
fields to see if that causes any issues elsewhere in the RST)
@egthomas, do you know how the noise level is utilised in gridding/mapping routines? I can only see that it is copied into the respective file formats.
@pasha-ponomarenko as far as I can tell, the noise values recorded in the grid files are only for informative purposes and not used in any subsequent processing algorithms (eg Map Potential, or even plotting).
Thanks, @egthomas , this was my impression too: just to maintain consistency between the formats.
I have been trying to better educate myself on the different noise parameters in the
rawacf
andfitacf
format files, and was wondering about thenoise.lag0
andnoise.vel
fields infitacf
files. As far as I can tell, these fields were populated by older versions of FITACF via thenoise_acf
function, which was removed from FITACF following discussions at the 2006 SD workshop, eg:and
(and likely influenced by the work of Ponomarenko and Waters [2006])
The "modern" fitting algorithms in the RST (eg,
fitacf2
,fitacf3
,lmfit2
) all seem to setnoise.lag0
andnoise.vel
to zero (thenoise.sky
value, however, is of course still used).So, here is a philosophical question - we have added several new fields to
fitacf
,map
, etc files over the years, but would we ever remove a field that is no longer relevant? Is there value in reading/storing thenoise.lag0
andnoise.vel
values fromfitacf
-format files (or more likelyfit
-format files) processed with pre-2007 versions of the software?Note there are several fields from the old parameter block which were dropped with the conversion from
dat
/fit
torawacf
/fitacf
(e.g.,nbaud
and the varioususr_*
-defined parameters).