simple approach for 'after -at' functionality

[d-hoke]

"Hey look! My first tcl baby(ies)!" An approach with no core changes required to obtain in any local (Tcl 8.6.9) application a form of requested 'after -at' style functionality in (subjectively) 99+% of situations, subject to the ambiguities and/or bugs of clock scan/format. I'll speculate the other 1-% of situations (daylight savings time periods) need serious consideration by application architects as to how they really want to handle time periods that don't exist (spring forward) and times that occur twice in the same day (fall back).
(This approach assumes that I didn't totally mis-understand the functionality sought.) (developed on local build of tcl 8.6.9 under debian x64) (Aside: Do hourly workers get paid for 8 hours when only 7 have actually passed? Do hourly workers get paid for an hour of overtime when 9 hours have 'elapsed' on the clock but they only worked 8?)

Burning question(s): Why didn't the feature requesters do/try something like this, as it seems pretty straightforward? Did they do/try something like this and abandon, and if so what was wrong with the approach?

[edit: allow 'after' id to be returned - may need to be differentiated from 'ok' somehow/way.]

# free form scan
proc at {time args} {
  # https://wiki.tcl-lang.org/page/Free%2Dformat+clock+scan
  # "...deprecate it if you like, but don't ever take the old code out."
  # free form scan does seem *really* useful
  set t1 [clock scan $time]
  set curtime [clock seconds]
  if { $t1 < $curtime } {
    error "time already past" "$t1 < $curtime" -1
  }
  if { $t1 == $curtime } {
    # time is now, execute in global context and return
    # TBD: Is this the correct way to accomplish that?
    uplevel #0 {*}$args
    return ok
  }

  set ival [expr $t1 - [clock seconds]]
  set cmd {after [expr $ival * 1000] $args}
  if {[string match  $args ""]} {
    #just a delay
    set cmd [concat after [expr $ival * 1000]]
  } else {
    set cmd [concat after [expr $ival * 1000] $args ]
    #give us something to vwait on with vwait...
    #set cmd [concat after [expr $ival * 1000] $args \; set lastatcmd ""]
  }
  # puts $cmd
  eval $cmd
  #allow 'after' id to be returned instead - return ok
}

# clock scan with options
proc atscanopts {time scanopts args} {
  set t1 [clock scan $time {*}$scanopts]
  set curtime [clock seconds]
  if { $t1 < $curtime } {
    error "time already past" "$t1 < $curtime" -1
  }
  if { $t1 == $curtime } {
    # time is now, execute in global context and return
    # TBD: Is this the correct way to accomplish that?
    uplevel #0 {*}$args
    return ok
  }

  set ival [expr $t1 - [clock seconds]]
  set cmd {after [expr $ival * 1000] $args}
  if {[string match  $args ""]} {
    #just a delay
    set cmd [concat after [expr $ival * 1000]]
  } else {
    set cmd [concat after [expr $ival * 1000] $args ]
    #give us something to vwait on... maybe not needed with tclreadline active?
    #global atdone
    #set atdone ""
    #set cmd [concat after [expr $ival * 1000] $args \; set atdone "done"]
  }
  # puts $cmd
  eval $cmd
  #allow 'after' id to be returned instead - return ok
}

can try with something similar to: at [clock format [clock add [clock seconds] 10 seconds]] {puts stdout [concat now [clock format [clock seconds]]]} at [clock format [clock add [clock seconds] -10 seconds]] {puts stdout [concat now [clock format [clock seconds]]]} at [clock format [clock add [clock seconds] 0 seconds]] {puts stdout [concat now [clock format [clock seconds]]]}

[sebres]

The issue is - after expects at the moment a relative time. The enhancement after at must operate with absolute time! Just imagine the relative time you calculated there is not reached (or even too late considering time-jumps) or occurred earlier (because of the time synchronization services like NTP, etc). More about you could find in @kennykb's TIP#302 and others (see my RFE below).

Additionally Tcl had missed monotonic time at all, so it was never guaranteed the moment where exactly relative time (of the timer-event) is expired/reached (by the way I saw already several bugs where the event was not triggered at all due to signed int-overflow).

Anyway, it is already implemented in RFE # fdfbd5e10. And it looks like accepted from TCT for the most part, and I started already with the TIP-processes for this, so hopefully it would be merged in next release.

[d-hoke]

@sebres - the description suggests you've done some fairly impressive work.

regarding my routine(s) (and considering time they obtain as 'current' time)... a)times found in the past 'error()', b)'current' time executes immediately (hopefully similar to 'after' executes, should prob. just have set ival to 0-or-1 and followed 'later' path instead of what I did) c)anything beyond current time will still get executed as its being executed on a delay (unless client 'after cancel's it before it can be initiated)

[sebres]

a) times found in the past 'error()',

considering current after strategy (and for common consistency reasons) - it should be rather the same as case b.

c)anything beyond current time will still get executed as its being executed on a delay (unless client 'after cancel's it before it can be initiated)

And this is exactly what I tried to describe above as potentially "dangerous" or unreachable using pure relative time (delay), which is just an offset from "now" to some target time point in the future. Just think about how "now" could change several times before the target time will be reached (due to time corrections your initially calculated delay may be invalid). after at must simply operate targeting absolute real-time instead.

The only one possibility to do it correctly on tcl-side were to check (even with after) in regular intervals this target time has been reached (and execute a command immediately if [clock seconds] >= $time), but it may be to "aggressive" for a lot of timer-events like this (missing update in-between, etc).

[d-hoke]

Perhaps food for thought, perhaps not...

I haven't looked at your code, but, I'm pretty sure that at best you can merely reduce the 'window of opportunity' for error, not eliminate it. I'm pretty sure even 'real-time' has to be subject to some caveats.

Take an NTP update arriving that will result in the local clock changing, BUT, wherever in the process it is, the local clock does not yet reflect that change - BUT, the local clock is actually out-of-sync with regard to 'real absolute' time (according to NTP atomic based time servers***), but you have a)either failed to execute the event on time (the 'real absolute' time already passed but the local clock was slow) -or- b)you executed it too early, because local clock was ahead of 'real absolute' time.

I suspect that either of those can happen even with your changes (even without looking at them), just the window or elapsed period of time within which it can occur may be much smaller.

I think it just depends on where one is willing to 'draw the line' in the sand as to what is acceptable.

And, if it can happen, it is likely really necessary for the application event to actually check and see where it is in 'time' vs what's acceptable for it to proceed, if the 'absolute' time of execution is that important to it - and even that may not be sufficient, unless the 'event' is actually running essentially not being subject to interrupted swaps, which I think is only feasible in driver interrupt/kernel land. And, even then, if that NTP update is in-process, but not yet reflected by local APIs when interrupts are disabled, there will still be a difference from 'real absolute'.

*** [atomic based time clocks reportedly can also show differences if you fly one around the world while the other remains 'stationary' [earth rotation speed] on surface - suggesting perhaps, that all time really is 'relative'... ? https://en.wikipedia.org/wiki/Hafele%E2%80%93Keating_experiment]

[sebres]

Surely. Just to explain my intention to rewrite this or why it is growing to "impressive work" like this. The relative time (delays by after command) should normally use rather monotonic time instead of the real-time (and this is "fixed" in my RFE), because it is mostly used as timeouts, regular intervals etc, which are all real interval from start/declaration. The after at command uses real-time, because it is mostly used as real-time target to execute an event.

Enclosed you'll find two artificial examples showing how it differences...

We are in 15:59:58 at the moment and different scenarios during vwait:

1. in result A you would see a normal flow (continuous time, no time drifts);
2. in result B the NTP switches the time backwards to 40 minutes (we were extremely forward in time);
3. in result C the NTP switches the time forwards to 40 minutes (time was extremely late compared to real time).

Additionally we'll simply assume the timeout will be occurring there (connecting host is unavailable).

Here is the example as diff between current tcl-core (red) and new handling in RFE fdfbd5e10 (green):

-after [expr {([clock scan "16:00"] - [clock seconds]) * 1000}] _do_something_in_16_00;
+after at [clock scan "16:00"] _do_something_in_16_00;
 ...
-set to [after 30000 {set done Timeout}]
 set chn [_connect -async ...]
 chan event $chn readable [list _read_data $chn done]
-vwait done
-if {$done eq "Timeout"} {error ...}
+if {![vwait 30000 done]} {error ...}
-after cancel $to

And the results (times given in real-time after NTP switch):

1. result A (continuous time, no time drifts, so the same for both):

   16:00:00 - _do_something_in_16_00 is executed
   16:00:28 - timeout occurred (service seems to be unavailable)

2. result B, time is 40 minutes backwards:

   -16:39:59 - _do_something_in_16_00 is executed
   -16:40:27 - timeout occurred (service seems to be unavailable)
   +16:00:00 - _do_something_in_16_00 is executed
   +16:00:28 - timeout occurred (service seems to be unavailable)

3. result C, time is 40 minutes forwards:

   -15:59:59 - timeout occurred (service seems to be unavailable)
   -15:59:59 - _do_something_in_16_00 is executed
   +16:00:00 - _do_something_in_16_00 is executed
   +16:00:28 - timeout occurred (service seems to be unavailable)

   As you can see only the new engine executed all this correctly in all 3 scenarios and even in scenario B the tcl-engine would generate the timeout ca. 40 minutes later than it should have (what is not acceptable at all) as well as missing the 16:00 time point.

[d-hoke]

alternate version, proc at2, that provides a 'filter'ing executor, proc atxwfilt, that could be used to decide what to do if 'waking' time not within some acceptable range of intended time.

This approach could be used to write an entire, sort of, replacement after, with the wake-up routine checking for next 'scheduled' item from list, and executing or not, according to whether it's target time was reached.

proc atxwfilt {time args} {
  puts stdout [concat atxwfilt audit beginning, $ time is $time]
  set t1 [clock scan $time]
  set curtime [clock seconds]
  # waiting increment
  set waitincr 30
  while { $curtime < $t1 } {
    set rem [expr $t1 - $curtime]
    puts stdout [concat atxwfilt waiting, rem $rem ]
    if { $rem > $waitincr } {
      set ival [expr $waitincr * 1000]
    } else {
      set ival [expr $rem * 1000]
    }
    after $ival
    set curtime [clock seconds]
  }
  # requested time should have been reached/passed
  set diff [expr $curtime - $t1]
  puts stdout [concat  $ t1 is $t1, $ curtime is $curtime ]
  if { $diff > 5 } { #select acceptable overrun in seconds
    puts stdout { diff is $diff }
    error "at: time > 5 seconds beyond requested, aborting" " diff is $diff " -1
    return
  }
  puts stdout { atxwfilt audit completed, executing }
  # let it execute...
  # if { $t1 == $curtime } {
    # time is now, execute in global context and return
    # TBD: Is this the correct way to accomplish that?
    uplevel #0 {*}$args
    # return ok
  # }
}

proc at2 {time args} {
  # https://wiki.tcl-lang.org/page/Free%2Dformat+clock+scan
  # "...deprecate it if you like, but don't ever take the old code out."
  # free form scan does seem *really* useful
  set t1 [clock scan $time]
  set curtime [clock seconds]
  if { $t1 < $curtime } {
    error "time already past" "$t1 < $curtime" -1
  }
  if { $t1 == $curtime } {
    # time is now, execute in global context and return
    # TBD: Is this the correct way to accomplish that?
    uplevel #0 {*}$args
    return ok
  }

  set ival [expr $t1 - [clock seconds]]
  set cmd {after [expr $ival * 1000] $args}
  if {[string match  $args ""]} {
    #just a delay
    set cmd [concat after [expr $ival * 1000] atxwfilt \\\{ $time \\\} ]
  } else {
    set cmd [concat after [expr $ival * 1000] atxwfilt \\\{ $time \\\} $args ]
    #give us something to vwait on with vwait...
    #set cmd [concat after [expr $ival * 1000] $args \; set lastatcmd ""]
  }
  # puts $cmd
  eval $cmd
  # return ok
}

to test the filter routines a bit:

atxwfilt [concat [clock format [clock add [clock seconds] -4 seconds]]]  puts stdout \[concat time is now \[clock format \[clock seconds\]\] \]
atxwfilt [concat [clock format [clock add [clock seconds] -6 seconds]]]  puts stdout \[concat time is now \[clock format \[clock seconds\]\] \]
atxwfilt [concat [clock format [clock add [clock seconds] 25 seconds]]]  puts stdout \[concat time is now \[clock format \[clock seconds\]\] \]

for at2, similar to original trials:

at2 [clock format [clock add [clock seconds] 10 seconds]] puts stdout \\\{hello there \\\}
at2 [clock format [clock add [clock seconds] 10 seconds]] puts stdout \\\{ hello there, time is now \[clock format \[clock seconds\]\] \\\}

[sebres]

No idea why you are trying to do this further. Just for the record: the proper solution implies strictly differentiation of monotonic (for relative timers) and real-time (for absolute timers), so it is simply impossible as long as Tcl does not provide both timing variants (which my RFE basically does).

[bovine]

Our Tcl bounties have the goal of improving Tcl core with new native functionality, not providing workarounds

[d-hoke]

Your comment indicates the "and/or Tcl extensions" phrase on your bounties page is not intended to be taken as broadly as it might from a straight-forward reading by anyone lacking 'sufficient' knowledge.

You may wish to clarify on that page exactly what you mean by 'Tcl extensions'.

@bovine Your comment and associated decision are acknowledged.

[d-hoke]

one more variant, and use/licensing info for those who might find these routines or derivatives as useful extensions to base tcl 'after' functionality. They are likely useable in any version of tcl, past or current or future, which supports the features (used by the routines) of 'clock' and 'after' APIs as those APIs operate in tcl 8.6.9.

This proc should probably have been the original at2, but... oh well. This form requires specification of a specific filter function, with a prototype matching that of proc atxwfilt posted earlier.

proc at2explicitfilterfunc {time explicitfiltfunc args} {
  # time - value matching domain of [clock seconds]
  # filtfunc - function matching prototype of atxwfilt
  # args - items that would be passed to 'after' command for execution
  # https://wiki.tcl-lang.org/page/Free%2Dformat+clock+scan
  # "...deprecate it if you like, but don't ever take the old code out."
  # free form scan does seem *really* useful
  set t1 [clock scan $time]
  set curtime [clock seconds]
  if { $t1 < $curtime } {
    error "time already past" "$t1 < $curtime" -1
  }
  if { $t1 == $curtime } {
    # time is now, execute in global context and return
    # TBD: Is this the correct way to accomplish that?
    uplevel #0 {*}$args
    return ok
  }

  set ival [expr $t1 - [clock seconds]]
  set cmd {after [expr $ival * 1000] $args}
  if {[string match  $args ""]} {
    #just a delay
    set cmd [concat after [expr $ival * 1000] $explicitfiltfunc \\\{ $time \\\} ]
  } else {
    set cmd [concat after [expr $ival * 1000] $explicitfiltfunc \\\{ $time \\\} $args ]
    #give us something to vwait on with vwait...
    #set cmd [concat after [expr $ival * 1000] $args \; set lastatcmd ""]
  }
  # puts $cmd
  eval $cmd
  # return ok
}

which might be exercised with

at2explicitfilterfunc [clock format [clock add [clock seconds] 10 seconds]] atxwfilt puts stdout \\\{ hello there, time is now \[clock format \[clock seconds\]\] \\\}

Use and Licensing information

The tcl code of the routines posted here of my creation, proc at proc atscanopts proc at2 proc atxwfilt proc at2explicitfilterfunc may be used freely as public domain -or- if your situation requires the presence of some license, may be used subject to the following bsd license. Copyright (c) 2019 David L. Hoke

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

[d-hoke]

one more variant, and use/licensing info for those who might find these routines or derivatives as useful extensions to base tcl 'after' functionality. They are likely useable in any version of tcl, past or current or future, which supports the features (used by the routines) of 'clock' and 'after' APIs as those APIs operate in tcl 8.6.9.

This proc should probably have been the original at2, but... oh well. This form requires specification of a specific filter function, with a prototype matching that of proc atxwfilt posted earlier.

proc at2explicitfilterfunc {time explicitfiltfunc args} {
  # time - value matching domain of [clock seconds]
  # filtfunc - function matching prototype of atxwfilt
  # args - items that would be passed to 'after' command for execution
  # https://wiki.tcl-lang.org/page/Free%2Dformat+clock+scan
  # "...deprecate it if you like, but don't ever take the old code out."
  # free form scan does seem *really* useful
  set t1 [clock scan $time]
  set curtime [clock seconds]
  if { $t1 < $curtime } {
    error "time already past" "$t1 < $curtime" -1
  }
  if { $t1 == $curtime } {
    # time is now, execute in global context and return
    # TBD: Is this the correct way to accomplish that?
    uplevel #0 {*}$args
    return ok
  }

  set ival [expr $t1 - [clock seconds]]
  set cmd {after [expr $ival * 1000] $args}
  if {[string match  $args ""]} {
    #just a delay
    set cmd [concat after [expr $ival * 1000] $explicitfiltfunc \\\{ $time \\\} ]
  } else {
    set cmd [concat after [expr $ival * 1000] $explicitfiltfunc \\\{ $time \\\} $args ]
    #give us something to vwait on with vwait...
    #set cmd [concat after [expr $ival * 1000] $args \; set lastatcmd ""]
  }
  # puts $cmd
  eval $cmd
  # return ok
}

which might be exercised with

at2explicitfilterfunc [clock format [clock add [clock seconds] 10 seconds]] atxwfilt puts stdout \\\{ hello there, time is now \[clock format \[clock seconds\]\] \\\}

Use and Licensing information

The tcl code of the routines posted here of my creation, proc at proc atscanopts proc at2 proc atxwfilt proc at2explicitfilterfunc may be used freely as public domain -or- if your situation requires the presence of some license, may be used subject to the following bsd license. Copyright (c) 2019 David L. Hoke

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.