adjust_rec_clocks_by_spp resetting the receiver clock

[ourairquality]

Describe the bug

Noted that without adjust_rec_clocks_by_spp disabled the REC_CLOCK appears to be reset to the adjustment time on some occasions breaking the solution.

To Reproduce

Using a config based on the ppp example script, without adding "adjust_rec_clocks_by_spp: false", and noted large steps in the receiver clock that mess up the solution. These did not appear to occur near a receiver clock step; could not identify any particular cause on first inspection.

Here is an example, and note that the clock jumps from +87796.0292 to -550.3730 which also happens to match the clock adjustment of 550.373, as if the clock has been reset to zero and then adjusted, or as if the adjustment has been applied as the new clock offset.

Processing G02 L1W L: PPP Phase count: LOG_ G02 L1W 0

Measurement for PHASMEAS G02 LOG 3 L1W

OBSERVED +21441327.2972 ~ 1.997e-05 -> 21441327.2972 RANGE -21425030.7594 ~ 0.000e+00 -> 16296.5378 REC_CLOCK +87796.0292 ~ Estimated -> 104092.5670 SAT_CLOCK -104082.9868 ~ 0.000e+00 -> 9.5801 ...

Adjusting LOG_ clock by 550.373 --------------------- Performing PPP -------------------

---

Processing G02 L1W L: PPP Phase count: LOG_ G02 L1W 0

Measurement for PHASMEAS G02 LOG 3 L1W

OBSERVED +21427665.1925 ~ 1.982e-05 -> 21427665.1925 RANGE -21410820.2280 ~ 0.000e+00 -> 16844.9645 REC_CLOCK -550.3730 ~ Estimated -> 16294.5914 SAT_CLOCK -104082.9400 ~ 0.000e+00 -> -87788.3485 SAGNAC -14.9238 ~ 0.000e+00 -> -87803.2724

grep NET_RESIDUAL test.trace .... NET_RESIDUAL -0.0000 ~ 0.000e+00 -> -1.2000 NET_RESIDUAL -0.0000 ~ 0.000e+00 -> -1.8182 NET_RESIDUAL -0.0000 ~ 0.000e+00 -> -1.1929 NET_RESIDUAL -0.0000 ~ 0.000e+00 -> -0.9612 NET_RESIDUAL -0.0000 ~ 0.000e+00 -> -87798.5610 NET_RESIDUAL -0.0000 ~ 0.000e+00 -> -87801.4923 NET_RESIDUAL -0.0000 ~ 0.000e+00 -> -87797.8910 NET_RESIDUAL -0.0000 ~ 0.000e+00 -> -87804.1346

Expected behavior

The step of the clock seems a little unexpected. Reporting in case this is unexpected.

With "adjust_rec_clocks_by_spp: false" the REC_CLOCK settled to the following for a period, until a receiver clock jump.

Processing G02 L1W L: PPP Phase count: LOG_ G02 L1W 0

Measurement for PHASMEAS G02 LOG 3 L1W

OBSERVED +20629053.9827 ~ 1.235e-05 -> 20629053.9827 RANGE -20502935.4508 ~ 0.000e+00 -> 126118.5319 REC_CLOCK -20853.9477 ~ Estimated -> 105264.5842 SAT_CLOCK -104074.8682 ~ 0.000e+00 -> 1189.7159 SAGNAC -13.0351 ~ 0.000e+00 -> 1176.6808

Configuration files

Based on the ppp example script.

Operating environment (please complete the following information):

Using the docker v3.1.0 version on Linux.

[polytopes-design]

It does seem that the state is being removed and then reinitialising, but its not clear why.

If you run the pea with --very-verbose at the command line, it should print Removing... to the console whenever it removes a state - state removal is usually limited to just ambiguities and ionosphere states though. It wont tell us why its being removed, but would confirm that in fact that function has been called. (I think the output is relatively recent so ginan versions more than a few weeks old probably wont print it)

There is a sigma_limit configuration that is capable of resetting receiver clocks, but its not used in the clocks part of the ppp_example, so it shouldn't be causing this issue.

[ourairquality]

Adding --very-verbose did not produce any "Removing" measages.

Given the hint about sigma, tried some option variations, and turning off this sigma check alone improved it a lot, ppp_filter: outlier_screening:postfit:sigma_check: false It would then handle the clock jump and the receiver clock estimate was much more stable and the residuals were much lower, and the solution appears to be converging.

[polytopes-design]

Your version of the software may be from before the Removing message was added. Under normal operation I'd expect to see many states being removes as they become stale, as satellites set. eg.

Removing ' AMBIGUITY     G03    ALIC        1'
Removing ' AMBIGUITY     G11    ALIC        1'
Removing ' AMBIGUITY     G14    ALIC        1'
Removing ' AMBIGUITY     G14    ALIC       20'
Removing ' IONO_STEC     G14    ALIC        0'
Removing ' AMBIGUITY     G31    ALIC        1'
Removing ' AMBIGUITY     G31    ALIC       20'
Removing ' IONO_STEC     G31    ALIC        0'

I've not seen the receiver clock removed though. I could try to run the debugger on your configuration+input+eph files if you send them through.

[ourairquality]

Traced this a little in a local build.

The REC_CLOCK is removed by a call to kfState.removeState() in removeBadReceivers(). The call graph is minOncePerEpoch :ppp:updateFilter:removeBadReceive. Changing the default ppp example config to "error_accumulation: enable: false" blocks this removal.

Also pppNet.kfState.measRejectCallbacks .push_back(incrementReceiverError); and pppNet.kfState.stateRejectCallbacks .push_back(rejectByState); and rejectByState() calling doMeasRejectCallbacks(), and perhaps also via filterKalman() calling doStateRejectCallbacks().

If the removal of the receiver clock state was unexpected then perhaps this heuristics are not working as intended.

The data was not clean, in an urban environment, but did have clear view of some satellites, and the receivers rtk solution was able to get a good fix in far worse. So this might not occur at a reference station. Shall leave it with you.

[polytopes-design]

A fix for the clock adjustment being too small and appearing to be absolute after the clock (all) states are removed is to add the second line below to the removeBadReceivers function:

kfState.statisticsMap["Rec error resets"]++;

rec.sol.dtRec_m_pppp_old[E_Sys::GPS] = 0;

The fix should be available in this week's develop update.