search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.42k stars 4.76k forks source link

API to provide the current system time #36617

Closed YohDeadfall closed 1 year ago

YohDeadfall commented 4 years ago

This proposal is edited by @tarekgh

Proposal

The aim of this proposal is to introduce time abstraction. This abstraction will include the ability to retrieve the system date and time, either in UTC or local time, as well as timestamps for use in performance or tagging scenarios. Additionally, this abstraction can be used in the Task operations like WaitAsync and CancellationTokenSource CancelAfter. By introducing this new abstraction, it will become possible to replace other existing abstractions that are currently being used in a nonuniform way across various interfaces such as ISystemClock and ITimer. The following are some examples of such interfaces:

In addition, this new abstraction will enable the creation of tests that can mock time functionality, providing greater flexibility in testing through the ability to customize time operations.

Below are some design notes to consider:

APIs proposal

APIs for .NET 8.0 and Down-levels

namespace System
{
    /// <summary>Provides an abstraction for time. </summary>
    public abstract class TimeProvider
    {
        /// <summary>Initializes the instance. </summary>
        protected TimeProvider();

        /// <summary>
        /// Gets a <see cref="TimeProvider"/> that provides a clock based on <see cref="DateTimeOffset.UtcNow"/>,
        /// a time zone based on <see cref="TimeZoneInfo.Local"/>, a high-performance time stamp based on <see cref="Stopwatch"/>,
        /// and a timer based on <see cref="Timer"/>.
        /// </summary>
        public static TimeProvider System { get; }

        /// <summary>
        /// Creates a <see cref="TimeProvider"/> that provides a clock based on <see cref="DateTimeOffset.UtcNow"/>,
        /// a time zone based on <paramref name="timeZone"/>, a high-performance time stamp based on <see cref="Stopwatch"/>,
        /// and a timer based on <see cref="Timer"/>.
        /// </summary>
        public static TimeProvider FromLocalTimeZone(TimeZoneInfo timeZone);

        /// <summary>
        /// Gets a <see cref="DateTimeOffset"/> value whose date and time are set to the current
        /// Coordinated Universal Time (UTC) date and time and whose offset is Zero,
        /// all according to this <see cref="TimeProvider"/>'s notion of time.
        /// </summary>
        public abstract DateTimeOffset UtcNow { get; }

        /// <summary>
        /// Gets a <see cref="DateTimeOffset"/> value that is set to the current date and time according to this <see cref="TimeProvider"/>'s
        /// notion of time based on <see cref="UtcNow"/>, with the offset set to the <see cref="LocalTimeZone"/>'s offset from Coordinated Universal Time (UTC).
        /// </summary>
        public DateTimeOffset LocalNow { get; }

        /// <summary>Gets a <see cref="TimeZoneInfo"/> object that represents the local time zone according to this <see cref="TimeProvider"/>'s notion of time. </summary>
        public abstract TimeZoneInfo LocalTimeZone { get; }

        /// <summary>Gets the current high-frequency value designed to measure small time intervals with high accuracy in the timer mechanism. </summary>
        /// <returns>A long integer representing the high-frequency counter value of the underlying timer mechanism. </returns>
        public abstract long GetTimestamp();

        /// <summary>Gets the frequency of <see cref="GetTimestamp"/> of high-frequency value per second. </summary>
        public abstract long TimestampFrequency { get; }

        /// <summary>Gets the elapsed time between two timestamps retrieved using <see cref="GetTimestamp"/>. </summary>
        /// <param name="startingTimestamp">The timestamp marking the beginning of the time period. </param>
        /// <param name="endingTimestamp">The timestamp marking the end of the time period. </param>
        /// <returns>A <see cref="TimeSpan"/> for the elapsed time between the starting and ending timestamps. </returns>
        public TimeSpan GetElapsedTime(long startingTimestamp, long endingTimestamp);

        /// <summary>Creates a new <see cref="ITimer"/> instance, using <see cref="TimeSpan"/> values to measure time intervals. </summary>
        /// <param name="callback">
        /// A delegate representing a method to be executed when the timer fires.  The method specified for callback should be reentrant,
        /// as it may be invoked simultaneously on two threads if the timer fires again before or while a previous callback is still being handled.
        /// </param>
        /// <param name="state">An object to be passed to the <paramref name="callback"/>. This may be null. </param>
        /// <param name="dueTime">The amount of time to delay before <paramref name="callback"/> is invoked. Specify <see cref="Timeout.InfiniteTimeSpan"/> to prevent the timer from starting. Specify <see cref="TimeSpan.Zero"/> to start the timer immediately. </param>
        /// <param name="period">The time interval between invocations of <paramref name="callback"/>. Specify <see cref="Timeout.InfiniteTimeSpan"/> to disable periodic signaling. </param>
        /// <returns>The newly created <see cref="ITimer"/> instance. </returns>
        /// <exception cref="ArgumentNullException"><paramref name="callback"/> is null. </exception>
        /// <exception cref="ArgumentOutOfRangeException">The number of milliseconds in the value of <paramref name="dueTime"/> or <paramref name="period"/> is negative and not equal to <see cref="Timeout.Infinite"/>, or is greater than <see cref="int.MaxValue"/>. </exception>
        /// <remarks>
        /// <para>
        /// The delegate specified by the callback parameter is invoked once after <paramref name="dueTime"/> elapses, and thereafter each time the <paramref name="period"/> time interval elapses.
        /// </para>
        /// <para>
        /// If <paramref name="dueTime"/> is zero, the callback is invoked immediately. If <paramref name="dueTime"/> is -1 milliseconds, <paramref name="callback"/> is not invoked; the timer is disabled,
        /// but can be re-enabled by calling the <see cref="ITimer.Change"/> method.
        /// </para>
        /// <para>
        /// If <paramref name="period"/> is 0 or -1 milliseconds and <paramref name="dueTime"/> is positive, <paramref name="callback"/> is invoked once; the periodic behavior of the timer is disabled,
        /// but can be re-enabled using the <see cref="ITimer.Change"/> method.
        /// </para>
        /// <para>
        /// The return <see cref="ITimer"/> instance will be implicitly rooted while the timer is still scheduled.
        /// </para>
        /// <para>
        /// <see cref="CreateTimer"/> captures the <see cref="ExecutionContext"/> and stores that with the <see cref="ITimer"/> for use in invoking <paramref name="callback"/>
        /// each time it's called.  That capture can be suppressed with <see cref="ExecutionContext.SuppressFlow"/>.
        /// </para>
        /// </remarks>
        public abstract ITimer CreateTimer(TimerCallback callback, object? state, TimeSpan dueTime, TimeSpan period);
    }
}
namespace System.Threading
{
    /// <summary>Represents a timer that can have its due time and period changed. </summary>
    /// <remarks>
    /// Implementations of <see cref="Change"/>, <see cref="IDisposable.Dispose"/>, and <see cref="IAsyncDisposable.DisposeAsync"/>
    /// must all be thread-safe such that the timer instance may be accessed concurrently from multiple threads.
    /// </remarks>
    public interface ITimer : IDisposable, IAsyncDisposable
    {
        /// <summary>Changes the start time and the interval between method invocations for a timer, using <see cref="TimeSpan"/> values to measure time intervals. </summary>
        /// <param name="dueTime">
        /// A <see cref="TimeSpan"/> representing the amount of time to delay before invoking the callback method specified when the <see cref="ITimer"/> was constructed.
        /// Specify <see cref="Timeout.InfiniteTimeSpan"/> to prevent the timer from restarting. Specify <see cref="TimeSpan.Zero"/> to restart the timer immediately.
        /// </param>
        /// <param name="period">
        /// The time interval between invocations of the callback method specified when the Timer was constructed.
        /// Specify <see cref="Timeout.InfiniteTimeSpan"/> to disable periodic signaling.
        /// </param>
        /// <returns><see langword="true"/> if the timer was successfully updated; otherwise, <see langword="false"/>. </returns>
        /// <exception cref="ObjectDisposedException">The timer has already been disposed. </exception>
        /// <exception cref="ArgumentOutOfRangeException">The <paramref name="dueTime"/> or <paramref name="period"/> parameter, in milliseconds, is less than -1 or greater than 4294967294. </exception>
        bool Change(TimeSpan dueTime, TimeSpan period);
    }

APIs for .NET 8.0 Only

namespace System.Threading
{
-   public sealed class Timer : MarshalByRefObject, IDisposable, IAsyncDisposable
+  public sealed class Timer : MarshalByRefObject, IDisposable, IAsyncDisposable, ITimer
    {
    }

    public class CancellationTokenSource : IDisposable
    {
+        /// <summary>Initializes a new instance of the <see cref="CancellationTokenSource"/> class that will be canceled after the specified <see cref="TimeSpan"/>. </summary>
+        /// <param name="delay">The time interval to wait before canceling this <see cref="CancellationTokenSource"/>. </param>
+        /// <param name="timeProvider">The <see cref="TimeProvider"/> with which to interpret the <paramref name="delay"/>. </param>
+        /// <exception cref="ArgumentOutOfRangeException"><paramref name="delay"/>'s <see cref="TimeSpan.TotalMilliseconds"/> is less than -1 or greater than <see cref="uint.MaxValue"/> - 1. </exception>
+        /// <exception cref="ArgumentNullException"><paramref name="timeProvider"/> is null. </exception>
+        /// <remarks>
+        /// The countdown for the delay starts during the call to the constructor.  When the delay expires,
+        /// the constructed <see cref="CancellationTokenSource"/> is canceled, if it has
+        /// not been canceled already. Subsequent calls to CancelAfter will reset the delay for the constructed
+        /// <see cref="CancellationTokenSource"/>, if it has not been canceled already.
+        /// </remarks>
+        public CancellationTokenSource(TimeSpan delay, TimeProvider timeProvider);
    }

    public sealed class PeriodicTimer : IDisposable
    {
+        /// <summary>Initializes the timer. </summary>
+        /// <param name="period">The time interval between returning the next enumerated value.</param>
+        /// <param name="timeProvider">The <see cref="TimeProvider"/> used to interpret <paramref name="period"/>. </param>
+        /// <exception cref="ArgumentOutOfRangeException"><paramref name="period"/> must be <see cref="Timeout.InfiniteTimeSpan"/> or represent a number of milliseconds equal to or larger than 1 and smaller than <see cref="uint.MaxValue"/>. </exception>
+        /// <exception cref="ArgumentNullException"><paramref name="timeProvider"/> is null</exception>
+        public PeriodicTimer(TimeSpan period, TimeProvider timeProvider);
    }
}

namespace System.Threading.Tasks
{
    public class Task<TResult> : Task
    {
+        /// <summary>Gets a <see cref="Task{TResult}"/> that will complete when this <see cref="Task{TResult}"/> completes or when the specified timeout expires. </summary>
+        /// <param name="timeout">The timeout after which the <see cref="Task"/> should be faulted with a <see cref="TimeoutException"/> if it hasn't otherwise completed. </param>
+        /// <param name="timeProvider">The <see cref="TimeProvider"/> with which to interpret <paramref name="timeout"/>. </param>
+        /// <returns>The <see cref="Task{TResult}"/> representing the asynchronous wait.  It may or may not be the same instance as the current instance. </returns>
+        public new Task<TResult> WaitAsync(TimeSpan timeout, TimeProvider timeProvider);

+        /// <summary>Gets a <see cref="Task{TResult}"/> that will complete when this <see cref="Task{TResult}"/> completes, when the specified timeout expires, or when the specified <see cref="CancellationToken"/> has cancellation requested. </summary>
+        /// <param name="timeout">The timeout after which the <see cref="Task"/> should be faulted with a <see cref="TimeoutException"/> if it hasn't otherwise completed. </param>
+        /// <param name="timeProvider">The <see cref="TimeProvider"/> with which to interpret <paramref name="timeout"/>. </param>
+        /// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for a cancellation request. </param>
+        /// <returns>The <see cref="Task{TResult}"/> representing the asynchronous wait.  It may or may not be the same instance as the current instance. </returns>
+        public new Task<TResult> WaitAsync(TimeSpan timeout, TimeProvider timeProvider, CancellationToken cancellationToken);
    }

    public class Task : IAsyncResult, IDisposable
    {
+        /// <summary>Gets a <see cref="Task"/> that will complete when this <see cref="Task"/> completes or when the specified timeout expires. </summary>
+        /// <param name="timeout">The timeout after which the <see cref="Task"/> should be faulted with a <see cref="TimeoutException"/> if it hasn't otherwise completed. </param>
+        /// <param name="timeProvider">The <see cref="TimeProvider"/> with which to interpret <paramref name="timeout"/>. </param>
+        /// <returns>The <see cref="Task"/> representing the asynchronous wait.  It may or may not be the same instance as the current instance. </returns>
+        public Task WaitAsync(TimeSpan timeout, TimeProvider timeProvider);

+        /// <summary>Gets a <see cref="Task"/> that will complete when this <see cref="Task"/> completes, when the specified timeout expires, or when the specified <see cref="CancellationToken"/> has cancellation requested. </summary>
+        /// <param name="timeout">The timeout after which the <see cref="Task"/> should be faulted with a <see cref="TimeoutException"/> if it hasn't otherwise completed. </param>
+        /// <param name="timeProvider">The <see cref="TimeProvider"/> with which to interpret <paramref name="timeout"/>. </param>
+        /// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for a cancellation request. </param>
+        /// <returns>The <see cref="Task"/> representing the asynchronous wait.  It may or may not be the same instance as the current instance. </returns>
+        public Task WaitAsync(TimeSpan timeout, TimeProvider timeProvider, CancellationToken cancellationToken);
    }
}

APIs for down-level Only

namespace System.Threading.Tasks
{
    public static class TimeProviderTaskExtensions
    {
        public static async Task<TResult> WaitAsync<TResult>(this Task<TResult> task, TimeSpan timeout, TimeProvider timeProvider);
        public static async Task<TResult> WaitAsync<TResult>(this Task<TResult> task, TimeSpan timeout, TimeProvider timeProvider, CancellationToken cancellationToken);
        public static async Task WaitAsync(this Task task, TimeSpan timeout, TimeProvider timeProvider);
        public static async Task WaitAsync(this Task task, TimeSpan timeout, TimeProvider timeProvider, CancellationToken cancellationToken);

    }

Possible APIs addition for .NET 8.0 and down-level

namespace System.Threading.Tasks
{
    public static class TimeProviderTaskExtensions
    {

        /// <summary>Initializes a new instance of the <see cref="CancellationTokenSource"/> class that will be canceled after the specified <see cref="TimeSpan"/>. </summary>
        /// <param name="timeProvider">The <see cref="TimeProvider"/> with which to interpret the <paramref name="delay"/>. </param>
        /// <param name="delay">The time interval to wait before canceling this <see cref="CancellationTokenSource"/>. </param>
        /// <exception cref="ArgumentOutOfRangeException"><paramref name="delay"/>'s <see cref="TimeSpan.TotalMilliseconds"/> is less than -1 or greater than <see cref="uint.MaxValue"/> - 1. </exception>
        /// <remarks>
        /// The countdown for the delay starts during the call to the constructor.  When the delay expires,
        /// the constructed <see cref="CancellationTokenSource"/> is canceled, if it has
        /// not been canceled already. Subsequent calls to CancelAfter will reset the delay for the constructed
        /// <see cref="CancellationTokenSource"/>, if it has not been canceled already.
        /// </remarks>
        public static CancellationTokenSource CreateCancellationTokenSource(this TimeProvider timeProvider, TimeSpan delay) ;
    }
}

End of the @tarekgh edit

The Original Proposal

Motivation

The ISystemClock interface exists in:

There is a small difference exists between implementations, but in most cases it can be moved out. In case of Kestrel's clock which has a specific logic inside, there could be made a new interface IScopedSystemClock which will provide the scope start time as UtcNow does now. Therefore, looks like all of them could be merged into a single class/interface and put into Microsoft.Extensions.Primitives.

The same interface often implemented by developers themselves to be used by microservices and applications utilizing dependency injection.

Having a common implementation of the data provider pattern will free users from repeating the same simple code many times and will allow to test apps in conjunction with ASP.NET internals without changing an environment.

Proposed API

The ISystemClock defines a way to get the current time in UTC timezone, and it has a simple implementation:

public interface ISystemClock
{
    DateTimeOffset UtcNow { get; }
}

public class SystemClock : ISystemClock
{
    public DateTimeOffset UtcNow => DateTimeOffset.UtcNow;
}

Originally proposed in dotnet/aspnetcore#16844.

BrennanConroy commented 3 years ago

We recently changed the internal ISystemClock in SignalR to use a long tick instead of DateTime/DateTimeOffset and have plans to do a similar thing in Kestrel. This is because the system clock can change unexpectedly on certain machines and we want to have a monotonically increasing view of the duration between certain activities.

This might not be interesting to this issue, but since Kestrel was used as a data point for this issue I'm pointing out that it may be moving away from DateTime in the future.

Clockwork-Muse commented 3 years ago

@BrennanConroy - Maybe so (and that's another potential thing to consider, returning a proxy for Environment.TickCount64), but for certain activities I'd imagine even for Kestrel you still want "current instant" for some activities.

AraHaan commented 3 years ago

Also isn’t DateTime and DateTimeOffset prone to the January 2038 problem (where it can overflow into thinking that it is 1901 again)

stephentoub commented 3 years ago

DateTime and DateTimeOffset prone to the January 2038 problem

They're 64-bit, not 32-bit.

scottsauber commented 2 years ago

Any update on this? It’d be nice if these things were in the BCL. I can’t tell you how many times I’ve wrapped the clock… but it’s a lot.

rsr-maersk commented 2 years ago

Could it not be extended to include e.g.: 

  public DateTimeOffset Now => DateTime.Now;
    public DateTimeOffset Now => DateTime.UtcNow;
    public DateTimeOffset Today => DateTime.Today;
scottsauber commented 2 years ago

Bumping again, curious if there's any interest in doing this before .NET 7. I just wrote yet another IClock interface.

tarekgh commented 2 years ago

We didn't have a chance to address this in .NET 7.0 because of other higher priority work. Hopefully, we get into this in the next cycle.

scottsauber commented 2 years ago

We didn't have a chance to address this in .NET 7.0 because of other higher priority work. Hopefully, we get into this in the next cycle.

Appreciate the update! Was curious where this stood.

geeknoid commented 2 years ago

Hi,

Although I like the idea of a interface for reading the clock, I believe it is insufficient for the task at hand.

Invariably, abstractions that use real-time in some form are difficult to test against. Dependency on real-time is in fact the most common cause of flaky tests in service code. You end up with tests that mostly work, or tests that have extremely large timeouts in order to cope with the randomness of execution time in the CI pipeline. Additionally, the use of real time makes it extremely difficult to write tests that exercise all code paths in a component.

With the goal of creating 100% deterministic tests, I think we need to abstract more than merely getting time. We also need to abstract waiting for time, and weave that concept throughout the platform.

I did a survey through many of our service code bases and found several instances of an IClock interface, each with different features. I reduced this to a common abstraction which we've been using successfully for over a year now. It's entirely done outside of the code framework, so it's a patchwork, but it shows the kind of functionality I think we need.

Here's the type as we have it today. I'm not proposing adding this to .NET, but I am proposing adding equivalent functionality throughout the stack (which would require passing an IClock instance to any function that deals with time).

/// <summary>
/// Abstracts the notion of time to increase testability.
/// </summary>
/// <remarks>
/// Flaky tests slow down development by introducing non-determinism to the development process.
/// A big source of non-deterministic behavior in tests is having a dependency on wall-clock time measurements.
/// This can be eliminated systematically by controlling the passage of time in tests.
///
/// Applications that adopt the IClock interface rather than directly reading from the system clock
/// (using methods like <see cref="System.DateTime.Now"/>) enable tests to substitute a custom
/// implementation of the interface which is under the control of the tests and thus can be deterministically
/// controlled.
/// </remarks>
public interface IClock
{
    /// <summary>
    /// Gets a <see cref="DateTimeOffset" /> object whose date and time are set to the current Coordinated Universal Time (UTC)
    /// date and time and whose offset is Zero.
    /// </summary>
    /// <seealso cref="System.DateTimeOffset.UtcNow"/>.
    DateTimeOffset UtcNow { get; }

    /// <summary>
    /// Gets current performance time.
    /// </summary>
    PerfTime PerfNow { get; }

    /// <summary>
    /// Creates a cancellable task that completes after a specified time interval.
    /// </summary>
    /// <param name="delay">The time span to wait before completing the returned task, or <c>TimeSpan.FromMilliseconds(-1)</c> to wait indefinitely.</param>
    /// <param name="cancellationToken">A cancellation token to observe while waiting for the task to complete.</param>
    /// <returns>A task that represents the time delay.</returns>
    /// <exception cref="ArgumentOutOfRangeException">
    /// The <paramref name="delay"/> is less than -1 or greater than the maximum allowed timer duration.
    /// </exception>
    /// <exception cref="ObjectDisposedException">
    /// The <see cref="CancellationTokenSource"/> associated
    /// with <paramref name="cancellationToken"/> has already been disposed.
    /// </exception>
    /// <remarks>
    /// If the cancellation token is signaled before the specified time delay, then the <see cref="Task"/> is completed in
    /// <see cref="TaskStatus.Canceled"/> state. Otherwise, the <see cref="Task"/> is completed in
    /// <see cref="TaskStatus.RanToCompletion"/> state once the specified time
    /// delay has expired.
    /// </remarks>
    /// <seealso cref="Task.Delay(TimeSpan, CancellationToken)"/>
    Task DelayAsync(TimeSpan delay, CancellationToken cancellationToken);

    /// <summary>
    /// Creates a periodic timer that enables waiting asynchronously for timer ticks.
    /// </summary>
    /// <param name="period">The time interval between invocations of the callback.</param>
    /// <returns>The new timer, call its <c>Dispose</c> method when you're done with it.</returns>
    ITimer CreatePeriodicTimer(TimeSpan period);

    /// <summary>
    /// Initializes a new instance of the <see cref="CancellationTokenSource" /> class that will be canceled after the specified time span.
    /// </summary>
    /// <param name="delay">The time interval to wait before canceling the source.</param>
    /// <returns>The new source.</returns>
    CancellationTokenSource CreateCancellationTokenSource(TimeSpan delay);

    /// <summary>
    /// Schedules a cancel operation on a <see cref="CancellationTokenSource" />.
    /// </summary>
    /// <param name="source">The source to affect.</param>
    /// <param name="delay">The time span to wait before canceling this.</param>
    void CancelAfter(CancellationTokenSource source, TimeSpan delay);

    /// <summary>
    /// Waits for the task to complete execution within a specified time interval.
    /// </summary>
    /// <param name="task">The task to wait for.</param>
    /// <param name="timeout">The time to wait for, or -1 to wait indefinitely.</param>
    /// <param name="cancellationToken">A cancellation token to observe while waiting for the task to complete.</param>
    /// <returns><see langword="true" /> if the task completed execution within the allotted time; otherwise, <see langword="false"/>.</returns>
    bool Wait(Task task, TimeSpan timeout, CancellationToken cancellationToken);

    /// <summary>
    /// Waits for all of the provided task objects to complete execution within a specified time or until the wait is cancelled.
    /// </summary>
    /// <param name="tasks">An array of task instances on which to wait.</param>
    /// <param name="timeout">The amount of time to wait, or -1 to wait indefinitely.</param>
    /// <param name="cancellationToken">A cancellation token to observe while waiting for the tasks to complete.</param>
    /// <returns><see langword="true" /> if all of the task instances completed execution within the allotted time; otherwise, <see langword="false"/>.</returns>
    bool WaitAll(Task[] tasks, TimeSpan timeout, CancellationToken cancellationToken);

    /// <summary>
    /// Waits for any of the provided task objects to complete execution within a specified time or until a cancellation token is cancelled.
    /// </summary>
    /// <param name="tasks">An array of task instances on which to wait.</param>
    /// <param name="timeout">The amount of time to wait, of -1 to wait indefinitely.</param>
    /// <param name="cancellationToken">A cancellation token to observe while waiting for the task to complete.</param>
    /// <returns>The index of the completed task in the <paramref name="tasks"/> array argument, or -1 if the timeout occurred.</returns>
    int WaitAny(Task[] tasks, TimeSpan timeout, CancellationToken cancellationToken);

    /// <summary>
    /// Gets a task that will complete when the input task completes, when the specified timeout expires, or when the specified cancellation token has cancellation requested.
    /// </summary>
    /// <param name="task">The task of interest.</param>
    /// <param name="timeout">The timeout after which the Task should be faulted with a <see cref="TimeoutException" /> if it hasn't otherwise completed.</param>
    /// <param name="cancellationToken">The cancellation token to monitor for a cancellation request.</param>
    /// <returns>The task representing the asynchronous wait. It may or may not be the same instance as the input task.</returns>
    Task WaitAsync(Task task, TimeSpan timeout, CancellationToken cancellationToken);
}

The above abstraction is supported by a SystemClock instance which implements the above using the wall-clock time, along with a FakeClock type which is used in tests. The FakeClock requires the test to explicitly advance time through the test.

This model has worked very well for us and we'll love to see this adopted as a first class concept in .NET.

mattjohnsonpint commented 2 years ago

@geeknoid - I think that most of those methods have to do with elapsed time, not the clock time. I genuinely don't understand why they would be part of an IClock or ISystemClock interface.

geeknoid commented 2 years ago

@geeknoid - I think that most of those methods have to do with elapsed time, not the clock time. I genuinely don't understand why they would be part of an IClock or ISystemClock interface.

Having a virtualized clock has limited value without having virtualized waiting as well. I want to be able to virtualize the passage of time in my code in order to enable higher quality testing of this code. Just virtualizing DateTime.Now isn't sufficient to that goal.

ChadNedzlek commented 2 years ago

I don't disagree that waiting times might benefit from being virtualized, but I don't agree that one is useless without the other. There are many applications for a "clock" that don't need timer systems. Any sort of logging or other timestamping system, or anything that does date/time math (is this cert expired) benefits hugely from a very simple "when is now" type interface and are difficult to test at all without one. And it also means that API doesn't get lost in the weeds trying to come to an agreement on the various forms of the timer stuff.

mattjohnsonpint commented 2 years ago

Measuring the passage of time isn't dependent on DateTime.Now or any of the underlying infrastructure that powers it.

In .NET, we have a few different ways to measure the passage of time, the main one being System.Diagnostics.Stopwatch, but also System.Threading.Timer, and others. None of them are based on clock time. Wrapping an interface around those would be much more difficult, and confusing from both an API and implementation perspective, IMHO.

Even if it were to be done, it shouldn't be mixed with this proposal, but proposed separately. The use cases are far too different.

GSPP commented 2 years ago

The name UtcNow is curious since this property contains UTC plus a local offset. So it's precisely not UTC, it's mixed with local information.

To be frank, I do not quite see why using DateTimeOffset everywhere is now the recommended practice. What has happened to the old principle of using UTC internally, and using local time for user-facing interaction? This always seemed sane and rather simple to me. Now, there is a superfluous offset value in play that is not used for anything 99% of the time.

I can see that this new interface must offer DateTimeOffset because of this new guidance. But it probably also should offer a plain UTC DateTime value for those codebases that are still following the old strategy (which is not obsolete by any means).

julealgon commented 2 years ago

I can see that this new interface must offer DateTimeOffset because of this new guidance. But it probably also should offer a plain UTC DateTime value for those codebases that are still following the old strategy (which is not obsolete by any means).

It is not "obsolete", but definitely discouraged in most cases:

These uses for DateTimeOffset values are much more common than those for DateTime values. As a result, consider DateTimeOffset as the default date and time type for application development.

I don't agree with providing a DateTime API with this since it is easy to convert a DateTimeOffset into a DateTime, and since it could also end up encouraging people to rely on a type that is not the recommended default anymore.

stephentoub commented 2 years ago

So it's precisely not UTC, it's mixed with local information.

You mean the _offsetMinutes? That'll be 0 for UtcNow. DateTimeOffset.UtcNow is literally just wrapping DateTime.UtcNow + 0 offset.

GSPP commented 2 years ago

For amusement, I'll post this:

image

(https://xkcd.com/927/)

We now have system clock interface number 15 🙃

But seriously, this one might actually be the last one we need.

mika76 commented 2 years ago

The blog post at https://codeblog.jonskeet.uk/2019/03/27/storing-utc-is-not-a-silver-bullet/ might give you an idea when just utc is not enough...

tarekgh commented 2 years ago

Some thoughts:

I want to focus the discussion here on what the exact scenario this interface is going to be used for and what we are missing in this proposal. As indicated in the beginning of this discussion, the main scenario is this interface is super useful for testing. Let's focus on whatever scenarios are used for and then look at what functionality is missing from the proposal.

stephentoub commented 2 years ago

First, I agree IClock should wrap the clock functionality only and shouldn't include other things like timers for instance. If we need more abstractions for timers or anything else, we should do that in other types.

I want to focus the discussion here on what the exact scenario this interface is going to be used for and what we are missing in this proposal.

I think we should use this issue to discuss all of the required functionality. If we decide it needs to be split into multiple interfaces, that can be done as part of this issue. I'm not currently convinced it actually needs to be separated or is beneficial to do so, and I'd like us to make forward progress on both aspects.

It's worth noting that various popular libraries have combined both the absolute and relative time aspects into a single abstraction. For example, Rx has a Scheduler abstraction, which is focused largely on enabling testing. It has both a "Now" for getting the current time as well as TimeSpan-based APIs for asking for callbacks/notifications when a certain period of time has elapsed. Martin's proposal effectively does the same thing, with ways of receiving such callbacks/notifications, and he's deployed his widely in various services. I think the relative time support is the primary "functionality missing from the proposal" ;-)

julealgon commented 2 years ago

I think we should use this issue to discuss all of the required functionality.

That's a very different requirement than what this issue describes though. It very clearly focuses on unifying all existing interfaces. Even the name would have to change as is specifically talks about current system time only.

stephentoub commented 2 years ago

That's a very different requirement than what this issue describes though

I'm suggesting this issue needs to account for both. I don't want to design one abstraction in isolation from the other. If it ends up with two abstractions, fine. If it ends up with one abstraction, fine. But they're highly related and should be designed together.

davidfowl commented 2 years ago

So we should put the timer APIs in here as well?

julealgon commented 2 years ago

So we should put the timer APIs in here as well?

The more stuff we try to solve at the same time with a single abstraction, the longer (exponentially) this will take to be done.

I'd rather see this one done as-is, then a possible scheduler abstraction (which I've personally never needed) rely on it later on.

If timer concerns are included here, I think this will just invariably become an endless discussion and it will just never be implemented.

Hopefully I'm wrong though.

tarekgh commented 2 years ago

@davidfowl

So we should put the timer APIs in here as well?

are you referring to what @geeknoid suggested or you are talking about other timer proposals?

ChadNedzlek commented 2 years ago

It feels like, since the primary motivation here appears to be testing, keeping the abstraction(s) smaller is better. If there is an interface with 20 methods on it, creating a fake/mock/whatever is 20 times harder than if it's just a single "UtcNow" member, which can be implemented trivially, while many of the other proposed timer/delay style things can't be implemented quite as simply, especially if they all need to be implemented together in a single object to implement the larger interface.

stephentoub commented 2 years ago

If there is an interface with 20 methods on it, creating a fake/mock/whatever is 20 times harder than if it's just a single "UtcNow" member

What are you faking it to do? If a concrete FakeClock : IClock were shipped as part of the same Microsoft.Extensions.* library that contained SystemClock : IClock, where FakeClock supported having its time set, would you still need to implement your own? And if so, why?

ChadNedzlek commented 2 years ago

Hrm... this got longer than I expected:

Sever tests I've written needed something like a clock that just went up with each call, or changed in the middle... I don't know that I've ever written a test that used some existing fake. Maybe that's some pattern that other people use, but it's not common in the code bases I'm used to, usually one or two methods are mocked out, or the whole interface implemented with a complete fake (and configurable in the test, so the fake behavior can be controlled). If it was just the simple ISystemClock with UtcNow, there really isn't a need for the framework to provide one publicly... it's a trivially simple interface for my tests to fake however they need to.

And presumably if that fake clock also needed to make all the delay/callback methods configurable in order to really support them all, it would have to a lot of knobs and parameters and such to flip, which might make it a complicate object to setup for a given test correctly. I don't have to configure every knob, but then my test might get some very weird behavior in the future when some piece of code depended on the part of the fake that I didn't configure as expected. With separate interfaces, I can just not provide that dependency, and the tests will fail very quickly with "Hey, someone started depending on a new interface you haven't accounted for", which I can catch really early, when validating the DI container. But if it's just one giant interface, it's going to be more difficult to catch those unless a test happens to catch that particular code path and notice whatever the "default" behavior of the framework's implementation isn't what we expected/wanted.

To me, at least, "I need to know what right now is" and "I need to delay work into the future" are very different concepts for code, and pretty different levels of complication in what code using them is really doing. Any code that has delays in it will be a mess to test (presumably you want to not have tests just waiting around doing nothing), so it would be nice to be able to easily identify those pieces of code via that a notable dependency that can be validated without having to execute all the code (it's possible to validate the containers without having to run any code in them, which has saved us from a lot of bugs).

I'm not saying that a testable delay interface isn't desirable: I definitely think it's as important, if not more so, to pull out for testing as the now concept. It's just a much more complicated API, and it will make me sad if the simple "now" interface gets lost in the weeds, as @julealgon mentions, because of that added complexity. I guess I don't understand the value in tying them together. The now interface is valuable all on its own, and could reduce duplication in multiple, publicly facing packages we already ship that have their own, also publicly visible, implementations of it (meaning my tests can stop having to mock all 4 clocks, and my DI containers don't have to inject all 4 of them in every service I create). And given that this issue for what is, more or less, a 1 line interface with a 1 line system implementation, but has been around for more than two years, that seems like real worry. :-( I don't know if we have a lot of publicly visible version of the delays stuff, but it's at least rare enough that I haven't encountered them yet.

tarekgh commented 2 years ago

In general, in .NET we don't provide any API mocking, but we provide interfaces. I am inclining to expose the interface + the actual SystemClock but not providing the FakeClock. Mocking libraries/end users will be free to implement that the way that fit. I don't think we need to provide a global FakeClock and make it work for any needed mocking scenarios even if we think we are covering all.

stephentoub commented 2 years ago

Thanks for the write-up.

I guess I don't understand the value in tying them together.

Anything we add here wouldn't ship until .NET 8. That's a year from now. Trying to rush in one thing doesn't really help.

The value from my perspective, at least in discussing them together (if not actually shipping everything as part of the same abstraction), is that we need to collect all the scenarios for what it means to abstract time. Getting the current time is part of that. Being able to be notified when it becomes a certain time is part of that. If you ship the former without the latter, then you very quickly find yourself with methods that do things like:

// All I have is a IClock.UtcNow.  I guess I'll make do...
DateTimeOffset start = clock.UtcNow;
DateTimeOffset end = start + timeout;
while (clock.UtcNow < end)
{
    ...
}

trying to build the latter on top of the former. And then you find yourself having to pass around two notions of time:

public void M(IAbsoluteClock absoluteClock, IRelativeClock relativeClock) { ... }

due to the fine-grained nature of it. But if they're all to do with time, why force that pain onto the developers using the interface? Why not just have one that supports both?

Again, I'm not saying we must have one. But I think there are valid reasons to, and I don't want to sweep a set of the requirements under the rug because it complicates another set of requirements.

in .NET we don't provide any API mocking, but we provide interfaces

There's no reason we couldn't provide a ManualClock implementation to go along with an interface. If the interface is simple, with just UtcNow, then sure, there's little benefit to doing so. But the moment you add some kind of notification for it becoming a certain time, which is essentially what all of the methods on Martin's interface boil down to, the complexity goes way up and it becomes much harder to throw together a simple implementation. At that point, having a helper that does it is valuable. If you don't want to think of it as mocking, think of it as a building block which someone can use to do their own mocking if desired.

ChadNedzlek commented 2 years ago

From my experience, DI stuff does end up being really fine grained, which I sort of like, since it keeps my dependency surface narrower to keep my stuff behaving as expected. Also, we could have all three. IAbsoluteClock, IRelativeClock, and then just "IAllTheTimeConcepts" if you code, for some reason, needs to do both.

I guess as long as we think we can resolve this whole conversation in time for 8.0, I'm happy to do it all together, but if it ends up dragging out, and looking like it's just never going to get resolved, I'd like to consider splitting them out if we could?

tarekgh commented 2 years ago

I recall mocking libraries using something simple like the ISystemClock. We need to expose IClock as that simple to be used there. For other functionality like timers ITimer or anything else, we should have different abstraction for that and not be incapsulated inside IClock. Of course, we need to ensure the whole required scenario works fully. I don't mind building helper methods/classes for that functionality (naming will be important I guess).

Anyway, we need to work on a proposal to focus the discussion on and then we move from there.

Clockwork-Muse commented 2 years ago

Questions to go along with this:

FiniteReality commented 2 years ago

If the proposal is moving in the direction of also including APIs for measuring time and waiting, then maybe something like this would work:

interface IStopClock
{
    // methods/properties for waiting (e.g. WaitAsync(TimeSpan))
}
interface ISystemClock
{
    // methods/properties for measuring time (e.g. Now, Ticks, etc.)
}
interface IClock : IStopClock, ISystemClock
{
    // additional methods/properties on clocks able to measure time and wait (e.g. WaitUntil(DateTimeOffset))
}

Then a standard implementation would be provided, based on UTC. Users would be able to implement any interface based on their needs, including if they need a localised clock. Any methods or properties that return time would preferably return DateTimeOffset, as that includes timezone information, and users would be able to convert that to DateTime if necessary.

As an additional question, should this proposal also include concepts such as named clocks and clock providers? I could imagine an API doing something like this to retrieve clocks located in specific timezones in order to handle localisation:

public void M(INamedClockProvider clockProvider, string timezone)
{

    var fiveHoursFromNow = clockProvider.GetClock("UTC").Now + TimeSpan.FromHours(5);
    var localised = clockProvider.GetClock(timezone).GetTimeAt(fiveHoursFromNow);
    SomeMessageQueue.Schedule(new MessageContents($"Due date: {localised}");
    SomeMessageQueue.Schedule(fiveHoursFromNow, new MessageContents("Stuff's here!"));
}
ChadNedzlek commented 2 years ago

I'm wary of the timezone being ambiguous for a clock, it's incredibly common to need to be in UTC to do basically anything in code, so if you just accepted a IClock that might be in UTC means a lot of ToUniversalTime() calls all over the place. Or, more likely, incorrect code, that only works when run on a server that already happens to be in UTC time but fails in weird places.

mika76 commented 2 years ago

At the risk of sounding stupid, but trying to think a little out the box here - would not all these problems be solved just by adding a IDateTime interface to the actual DateTime type and make all the methods and properties return that instead of a basic DateTime? That way IDateTime could be mocked directly if necessary. The same for DateTimeOffset and any related classes/structs...

julealgon commented 2 years ago

To me, at least, "I need to know what right now is" and "I need to delay work into the future" are very different concepts for code, and pretty different levels of complication in what code using them is really doing.

This is a very important aspect as well. The more is added to the abstraction, the further away from a "role interface" and closer to a "header interface" it becomes. This brings not only complications to mock the object, but exposes too much functionality to the consumer: now suddenly code that only needed to deal with current time, has its dependency widened to include other concerns it doesn't need to use.

Considering how often we only need to deal with current time vs relative time, it would feel wrong to mix both concerns together into the same abstraction as this just increases the surface area of what is accessible from any consumer without the consumer wanting that. "Partially leveraging" abstractions like this is usually a bad practice.

@stephentoub

If the interface is simple, with just UtcNow, then sure, there's little benefit to doing so. But the moment you add some kind of notification for it becoming a certain time, which is essentially what all of the methods on Martin's interface boil down to, the complexity goes way up and it becomes much harder to throw together a simple implementation. At that point, having a helper that does it is valuable.

From my experience, if the abstraction is so complex that a manual fake implementation is required like this, I think there is a clear problem with such abstraction. Ideally, you always want a very focused abstraction that has a single core functionality, ideally centralized in a single method, which all subsequent behavior is based upon. If said method is a bit too complex for "normal" use, you can then provide helper overloads and extensions via a wrapper class (like HttpClient is to HttpMessageHandler), or extension methods in the interface (like ILogger and LoggerExtensions). This approach keeps the core of the functionality as part of the interface, and usability helpers and overloads separate, so that when mocking you only need to be concerned with that single core aspect and be sure that no matter how you call into the instance, that mocked implementation will definitely be used.

If the abstraction now has many different entry points and mandatory state management concerns that force a manual fake to be provided, it should be split up IMHO.

@mika76

At the risk of sounding stupid, but trying to think a little out the box here - would not all these problems be solved just by adding a IDateTime interface to the actual DateTime type and make all the methods and properties return that instead of a basic DateTime? That way IDateTime could be mocked directly if necessary. The same for DateTimeOffset and any related classes/structs...

This would create an abstraction that is just a header interface, meaning it has too many concerns, each one required for different purposes in different situations. Such interfaces are not ideal as they broaden the surface area of what you really need, making everything more complicated.

As per the above, you want to, as much as possible, keep interfaces to a minimum in terms of functionality so that you can be sure that your consumer is only doing what it is supposed to be doing, and so that testing is also more straightforward.

dazinator commented 2 years ago

It might help just to agree on the use cases that need to be addressed, agree on those, then work out what abstractions are necessary. I think @stephentoub was hinting at just wanting to understand the various cases before designing anything, like how the abstractions might look. I was looking at this issue with a very specific and simple use case in mind:

  1. What is the current date & time (I need to set a last updated or created date on a new object / record for example) - encouraging users of the dotnet framework to obtain this in a way in their code base that will support mocking, would be useful. e.g warn against accessing DateTime.UtcNow and friends. This can be important in scenarios where you need to test / simulate things happening at a "specific" time (mocked) to verify the behaviour of the system.

I understand there are other use cases, it has been mentioned "notify me after this duration is reached" may be one. For me, if I need to delay or schedule an action, I tend to use Task.Delay(TimeSpan) and I find that as long as the TimeSpan is configurable (replacable at test time) - if I want to shorten the delay during a test, I don't need to amend the time at all - I just need to use a shorter TimeSpan in the test. Then there is a more complicated notify me when this specific time is reached - this one takes into account the current time in the timezone and works out the duration that needs to be waited in this timezone so it can Task.Delay for the required period of time and basically allow the caller to be notified once the time is reached. I have used third party libraries like HangFire to fulfill this scheduling requirement, given all the complication -and often given how schedules are more conveniently expressed and configured at the application level as CRON expressions.

stephentoub commented 2 years ago

I think @stephentoub was hinting at just wanting to understand the various cases before designing anything, like how the abstractions might look.

Exactly. Thanks.

dazinator commented 2 years ago

It has both a "Now" for getting the current time as well as TimeSpan-based APIs for asking for callbacks/notifications when a certain period of time has elapsed.

It should be mentioned that the framework already provides a pretty flexible solution for notifications/callbacks that supports mocking. I am thinking of Change Tokens - In my view change tokens better protect developers from evolving requirements - sure you want your callback to happen according to a clock time today, but tomorrow when someone says they also need to be able to trigger it via a button click (or api call etc) then suddenly your code being coupled directly to a clock / timer for a callback makes less sense - as now to keep it the same, you'd have to implement a hybrid "clock / timer plus button click" implementation of a clock, so that callers still think they are working with a clock but will be notified on both occasions. Being able to replace the implementation of the clock, is not the best way to cater for a button click requirement. I think change tokens have solved this problem by seperating the concerns of "just wanting to be notified", from the firing of the notification callback. So I'd encourage them to be used for these situations, and then your implementation of a change token can be triggered today based on a clock timer, or tomorrow based on whatever logic you need by evolving the publisher and without touching the subscribers.

One negative I found with change tokens is they don't seem to be asynchronous in nature, so if you need an async callback it's not as easy to wire them up as it should be, but still possible,

julealgon commented 2 years ago

One negative I found with change tokens is they don't seem to be asynchronous in nature, so if you need an async callback it's not as easy to wire them up as it should be, but still possible,

That seems like such an obvious gap. Do you know if there is an issue already tracking adding async support for it? Might be worth it linking here if so.

mattjohnsonpint commented 2 years ago

Important use cases (IMHO) are more along the lines of:

dazinator commented 2 years ago

One negative I found with change tokens is they don't seem to be asynchronous in nature, so if you need an async callback it's not as easy to wire them up as it should be, but still possible,

That seems like such an obvious gap. Do you know if there is an issue already tracking adding async support for it? Might be worth it linking here if so.

Yeah there is this: https://github.com/dotnet/runtime/issues/69099 You may also find this useful:

geeknoid commented 2 years ago

@dazinator Flaky tests are the bane of any large engineering organization. A substantial source of flakiness in tests is any dependency on real-time. Even large timeout values eventually hit in the CI system because of the oddball things that happen when in that environment. Using large timeout values to mitigate flakes now makes your test suite take potentially much longer wall-clock time to execute.

In our organization, we have a requirement of having 100% unit test coverage. Achieving this level of coverage in an environment where you don't fully control the passage of time is often impossible to do reliably.

Looking through the source base of dozens of services within our organization, I've found dozens of ad hoc implementations of the IClock I showed up, all created in order to stabilize tests and enable deeper testing. Since the underlying platform doesn't support IClock systematically, these ad hoc solutions aren't perfect, but they sure do raise the bar.

In our organization, we've introduced a FakeClock implementation of IClock. It lets the unit test move time forward explicitly so you can test all cases in your state machines. FakeClock also supports an 'auto-advance on read' mode which is also commonly used to make core behave as though time is moving forward normally, except that it is 100% deterministic and reproducible in tests.

dazinator commented 2 years ago

@geeknoid

In our organization, we've introduced a FakeClock implementation of IClock. It lets the unit test move time forward explicitly so you can test all cases in your state machines

It sounds like the simple IClock api that jjust returns the current time would suit your case then as you could mock this and return whatever time you like at test time, including incrementing it so time "flows".

Using large timeout values to mitigate flakes now makes your test suite take potentially much longer wall-clock time to execute.

Not sure who has argued for using large timeout values, but yes that does sound like one of those quick and dirty solutions that developers would use when there is a sensitive test, and they have too many problems and not enough time!

I noticed though that you have proposed more api's other than the expected method to just return the current time. For example take this one:


    /// <seealso cref="Task.Delay(TimeSpan, CancellationToken)"/>
    Task DelayAsync(TimeSpan delay, CancellationToken cancellationToken);

I concur that Task.Delay is a common thing that crops up, similar to DateTime.UtcNow - you end up having to refactor the code so you can eliminate or take control of the delay at test time. However whether this method should be part of any proposed IClock - I am not sure.. It feels to me that a clock is about reporting the time, and task based stuff would be best satisfied by something seperate.. that's where I'll leave it to the experts ;-)

geeknoid commented 2 years ago

@dazinator Replacing just the "get time" function is completely insufficient to ensure robust tests. Code does things like waiting for tasks to complete with a timeout. The timeout that is used follows wall-clock time so it needs to be virtualized.

The API I proposed is strictly exemplary, showing what we've needed in our systems given that we weren't modifying the core libraries. What I'm really arguing for is:

Clockwork-Muse commented 2 years ago

Task.Delay would take an IClock

As near as I'm aware this isn't possible, because the part that actually "waits" is a hardware/os scheduler solely taking expected elapsed time, and isn't going to be making a trip back up the stack to check any clock (wall-clock or elapsed clock, even ignoring testing mocks).

stephentoub commented 2 years ago

Think of all the methods I showed in my interface being exposed as first-class methods in their existing types. So Task.Delay would take an IClock as parameter. Existing overloads would simply assume a concrete implementation of IClock that uses wall-clock time as today.

I'm not excited about this. It's not possible to control the low-level timeout in many situations, e.g. where we just delegate to the OS, and in other situations it'll make the implementations more expensive. There are also hundreds of methods today in the core libraries that take timeouts. Rather, I'm happy to consider one or more abstractions above the core libraries, e.g. in Microsoft.Extensions, focused on DI. The abstraction can also be general, e.g. an interface callback mechanism, along with more specific APIs to allow for primary use cases to be optimized, e.g. a DIM on that interface for producing a delay task, implemented in terms of the callback but overridable in particular by a SystemClock to delegate to Task.Delay.

mattjohnsonpint commented 2 years ago

... the part that actually "waits" is a hardware/os scheduler ...

Yes - there are different hardware features for different functions.

Great article on this from a Windows perspective here. Other OSs have similar features, and .NET uses them accordingly.