[observationtime] 'Caught' values and times bleed into next archive record

[bellrichm]

This seems to be a 'timing' issue along with the fact that the loop packets know nothing about archive periods and intervals. This appears to be what is happening.

• A loop packets is emitted by the driver.

• The engine dispatches a NEW_LOOP event.

• The engine dispatches a CHECK_LOOP event.

• StdArchive process the NEW_LOOP and creates a new Accumulator to process the next set of loop packets in a new archive record.

• StdArchive processes the CHECK_LOOP and raises a BreakLoop exception.

• The engine catches the BreakLoop exception and raises a POST_LOOP event.

• The engine raises the PRE_LOOP event.

• Between the time the POST_LOOP (or PRE_LOOP) event has been raised, n more loop packets are emitted.

• These loop packets are currently augmented with min/last values and times.

• But the new Accumulator has been created (which is correct), so these packets are in the next archive record.

The result is the min/last values 'bleed' into the next archive record.

[bellrichm]

I see no good and clean way to manage this. A couple of possibilities are

• Instead of relying on the PRE and POST LOOP events, give the service knowledge about the archive period/interval.
• Possibly add a 'primary' field (in this case lightning_strike_count) and use to determine if the loop packets should be augmented.

Ultimately, for now at least, the loop packet will only be augmented if the observation's whose value is to be captured is in the record and has a value. Effectively this removes 'state' from the processing. If downstream process, either in WeeWX or outside (for example, a MQTT Subscriber), wants to know the captured values/times; it either needs to keep track (which might be possible....) or it should use the archive record.

The question is, if the packet's observation being tracked does not have a value or is None, is the packet augmented with None. I believe the WeeWX way would augment with None, but I think missing values might be more useful and less misleading (and might be considered a partial packet). I will not put in None values in this case.

[bellrichm]

I went with checking that the observation has a value. Even this won't work all the time. If a packet comes in before the pre loop event is processed. And if that packet has a value that is bigger than currently stored packet, the currently stored packet will be used. When, really the one in the event packet should be used. This is better, and will roll with it for now. But needs more work. See commit, https://github.com/bellrichm/WeeWX-Extras/blob/41b578d026cae5f7105669ae6464ae4ce9fe11b6/bin/user/observationtime.py

[bellrichm]

Additional information is in this thread, https://groups.google.com/g/weewx-development/c/1cJBMAX3Wsg