Closed ChristopherHaws closed 2 years ago
Is the common use case really to have separate values for seconds, milliseconds, microseconds and nanoseconds? Wouldn't it be better to instead expose a floating point number that's the number of seconds, including the fractional part?
I realize this would be inconsistent (seconds and milliseconds are already separate), but I think usefulness beats consistency.
Though some of the proposed members (like AddMicroseconds
or TotalMicroseconds
) would still make sense.
(Also, I'm not sure what exactly would be the right API to expose that.)
Is the common use case really to have separate values for seconds, milliseconds, microseconds and nanoseconds? Wouldn't it be better to instead expose a floating point number that's the number of seconds, including the fractional part?
If the caller has the values of seconds, milliseconds, microseconds, nanoseconds, would need to do some calculations to convert these values into the second fraction to be able to call the API. I think the request here is to help users not do the calculations themselves. users can achieve the result today by calculating the ticks from the second fractions. so I am seeing the proposed APIs just a helper methods.
@ChristopherHaws what is the need for the To/From Unix APIs? could you please clarify how these exactly get used?
@tarekgh Correct, there is no functional changes. It is just additional properties and methods for making it easier to work with microseconds and nanoseconds, which already exists in the Ticks
. Here is a recent PR where I made some helper methods for doing just this: https://github.com/fluentassertions/fluentassertions/pull/669/files
I wasnt sure if unix has nanoseconds, but they do have the concept of microseconds, which is stored as a 64-bit unsigned integer. Here is a thread for the GO language where they are discussing use cases: https://github.com/golang/go/issues/18935#issuecomment-277884761. One of the example mentioned (https://github.com/golang/go/issues/18935#issuecomment-278378765) is that there are many system calls in unix that now support microseconds and the standard protobuf timestamp uses nanoseconds.
Specificly TimeSpanConversionExtensions.cs
and FluentDateTimeExtensions.cs
@tarekgh
If the caller has the values of seconds, milliseconds, microseconds, nanoseconds, would need to do some calculations to convert these values into the second fraction to be able to call the API.
I get that. My question is: how often does that happen? As an example, what's more likely, that the input is "3 minutes 14 seconds 159 milliseconds" or "3 minutes 14.159 seconds"? I think it's the latter and the current API does not handle that case well and the API proposed here does not improve it much either.
@svick I think that both ways are valid for different reasons. The way I proposed in the issue are simply extending the existing API's standard. If DateTime.Seconds
had been a method instead, I would say there should be an overload that allows you to decide if it includes the higher precision values or not. I cant think of a good name, but something like might work better for what you are talking about:
namespace System {
public struct DateTime {
public double GetPart(DateTimePart.Seconds, DateTimePartOptions.IncludeHigherPrecision);
}
}
@ChristopherHaws
please let me know what you think.
I am not seeing a big value to have From/To Unix proposed APIs. if you agree, we can remove them from the proposal.
Um, but what about dotnet/runtime#23747 ?
@Clockwork-Muse for dotnet/runtime#23747, I am seeing a lot of people is using
new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)
this why dotnet/runtime#23747 would have a value. but for the proposed Unix APIs here, I am not seeing many people using it.
@tarekgh
Nanoseconds { get; }
but remove FromNanoseconds(double value);
and the constructors containing nanoseconds. I would be ok with this, but I still don't like that as a consumer of DateTime
, if I want to add nanoseconds, I need to know how to convert between ticks and nanoseconds. For this reason, I would recommend we keep the TicksPerNanosecond
const.DateTime.Microsecond
and DateTime.Nanosecond
return int now since the values will always be between 0 and 999.DateTime.AddMicroseconds
takes an int now since the value must be between 0 and 999.DateTimeOffset.Microsecond
and DateTimeOffset.Nanosecond
return int now since the values will always be between 0 and 999.DateTimeOffset.AddMicroseconds
takes an int now since the value must be between 0 and 999.TimeSpan.Microseconds
and TimeSpan.Nanoseconds
return int now since the values will always be between 0 and 999.TimeSpan.TotalMicroseconds
and TimeSpan.TotalNanoseconds
return double still since that is what the other total properties return. This could be long but I kept it as double for consistency. Let me know if I should change this.TimeSpan.FromMicroseconds
takes a double still to keep the api consistent with the other From
methods.TimeSpan.TicksPerNanosecond
is a double since it is equal to 0.01d
.Removed from Proposal
namespace System {
public struct DateTime {
public DateTime(int year, int month, int day, int hour, int minute, int second, int millisecond, int microseconds, int nanoseconds);
public DateTime(int year, int month, int day, int hour, int minute, int second, int millisecond, int microseconds, int nanoseconds, System.DateTimeKind kind);
public DateTime(int year, int month, int day, int hour, int minute, int second, int millisecond, int microseconds, int nanoseconds, System.Globalization.Calendar calendar);
public System.DateTime AddNanoseconds(int nanoseconds);
}
public struct DateTimeOffset {
public DateTimeOffset(int year, int month, int day, int hour, int minute, int second, int millisecond, int microseconds, int nanoseconds, System.TimeSpan offset);
public DateTimeOffset(int year, int month, int day, int hour, int minute, int second, int millisecond, int microseconds, int nanoseconds, System.TimeSpan offset, System.Globalization.Calendar calendar);
public static System.DateTimeOffset FromUnixTimeMicroseconds(long microseconds);
public static System.DateTimeOffset FromUnixTimeNanoseconds(long nanoseconds);
public System.DateTimeOffset AddNanoseconds(int nanoseconds);
public static long ToUnixTimeMicroseconds();
public static long ToUnixTimeNanoseconds();
}
public struct TimeSpan {
public static System.TimeSpan FromNanoseconds(double value);
public TimeSpan(int days, int hours, int minutes, int seconds, int milliseconds, int microseconds, int nanoseconds);
}
}
@ChristopherHaws thanks for your updates. I'll try to look more get back to you if I have any other comment. could you please move the allowed values of the millisecond, Microseconds...etc. to the proposal, section to be clear about how the APIs will work and when we throw?
@tarekgh Added in the form of xml docs to the proposal. Thanks!
@iamcarbon I am seeing your reaction to the proposal. do you have any specific feedback to understand why you don't like the proposal? Thanks.
@tarekgh DateTime is only accurate to 100 nanoseconds. It's confusing to have a property that cannot represent its full range.
It's confusing to have a property that cannot represent its full range.
I am not sure I fully understand this but DateTime has Ticks property that can reflect the ticks (in hundreds of nanoseconds). Also, it has MinValue and MaxValue properties which can tell the min and max of the supported range. if you are referring to a specific property in the proposal, please let me know so we can evaluate your feedback.
@tarekgh I believe he is referring to the fact that the Nanoseconds
property would only ever return one of 10 values (0, 100, 200, 300, 400, 500, 600, 700, 800, 900).
I personally don't have any issue with this, as the alternative is that anyone who needs to know the amount of nanoseconds would have to calculate it from the ticks, but it should definitely be pointed out in the documentation. IMO, since DateTime
does have nanoseconds in it's precision, a consumer should be able to access it as a property.
This is a limitation of DateTime. even if we were supporting nanoseconds we could get cases asking for femtoseconds :-)
At any case, we have to have a minimum unit we can support considering we want to keep DateTime size not exceeding long size for perf reasons.
@tarekgh I agree. I think he was just pointing out that the signature states that it returns an int between 0 and 900, but the values are not contiguous which might confuse some people that don't realize that DateTime's precision is 100ns. I think asking users to convert ticks manually is much more confusing and potentially error prone.
@ChristopherHaws of course a hypothetical future runtime/platform could support a finer resolution with the same API, and I don't believe we would consider that a breaking change.
@danmosemsft True. That's another good reason for these helper methods/properties. If someone is making an assumption about the resolution of ticks, their code would break if that resolution changed from underneath them. With these helper methods, consumers can be sure that TimeSpan.Microseconds
returns microseconds and not nanoseconds or picoseconds. :)
@joperezr this proposal looks good to me. if you agree, please mark the issue as ready for design review. Thanks.
Be very careful with this. Exposing micros and nanos directly can easily lead to overflow exceptions or floating point errors.
For example, TimeSpan.MaxValue.TotalNanoseconds
would give a double
of 9.22337203685477E+20
, which cannot be converted to a long
without overflowing. Essentially this is the same as:
(long) (TimeSpan.MaxValue.TotalSeconds * 1000 * 1000 * 1000) // -9223372036854775808
There's also a loss of precision with such extreme floating point values - which is counter-intuitive when working with APIs that promise nanoseconds.
I advise caution. We don't want people to think DateTime
, DateTimeOffset
and TimeSpan
now have increased precision.
@mj1856 Agreed. I was thinking about this as well. From what I can see, TotalMicroseconds shouldn't have any issue, but TotalNanoseconds could have issues when you start approaching TimeSpan.Max
and TimeSpan.Min
. Please correct me if I am wrong about this. I will add an overflow exception to the original post so that it is not overlooked. If you have any thoughts on a better way to handle this, I am all ears! :)
@tarekgh Do you mean long
? The only reason it returns double
in the current proposal is because the other Total
properties return double
's.
@ChristopherHaws ignore my comment. I was looking at different place :-(
@mj1856 I was wrong, there is a potential for loss of precision with double
even with microseconds. One alternative could be to use decimal
since it's 128-bit. Thoughts?
void Main()
{
TimeSpan.MaxValue.TotalMicroseconds(); // 9.22337203685478E+17
TimeSpan.MaxValue.TotalMicrosecondsAsDecimal(); // 922337203685477580.7
TimeSpan.MaxValue.TotalNanoseconds(); // 9.22337203685478E+20
TimeSpan.MaxValue.TotalNanosecondsAsDecimal(); // 922337203685477580700
}
public static class TimeSpanEx
{
public const Int64 TicksPerMicrosecond = TimeSpan.TicksPerMillisecond / 1000;
public const Double TicksPerNanosecond = TicksPerMicrosecond / 1000d;
public static Double TotalMicroseconds(this TimeSpan self) => self.Ticks * (1d / TicksPerMicrosecond);
public static Decimal TotalMicrosecondsAsDecimal(this TimeSpan self) => self.Ticks * (1m / TicksPerMicrosecond);
public static Double TotalNanoseconds(this TimeSpan self) => self.Ticks * (1d / TicksPerNanosecond);
public static Decimal TotalNanosecondsAsDecimal(this TimeSpan self) => self.Ticks * (1m / (Decimal)TicksPerNanosecond);
}
Well, the fact is that all three types in question already have their range locked in, and over that range one cannot represent nanoseconds in 64 bits without losing some of them. Floating point type allow for loss of precision, but one needs System.Numerics.BigInteger
to truly work with nanoseconds without any loss.
We're also touching on why ticks go to 7 decimals and no further. 8 decimals and you start losing information over this range.
I'm not saying don't do this, I'm just saying check that the APIs can handle everything at all boundaries, especially with nanoseconds.
Given the above concerns about overflow and precision, let's discuss a bit more on what would the right return type should be.
My take here:
@tarekgh Here are the results using G17 formatting:
void Main()
{
TimeSpan.MaxValue.TotalMicroseconds().ToString("G17", CultureInfo.InvariantCulture); // 9.2233720368547763E+17
TimeSpan.MaxValue.TotalMicrosecondsAsDecimal().ToString("G17", CultureInfo.InvariantCulture); // 9.2233720368547758E+17
TimeSpan.MaxValue.TotalNanoseconds().ToString("G17", CultureInfo.InvariantCulture); // 9.2233720368547758E+20
TimeSpan.MaxValue.TotalNanosecondsAsDecimal().ToString("G17", CultureInfo.InvariantCulture); // 9.2233720368547758E+20
}
public static class TimeSpanEx
{
public const Int64 TicksPerMicrosecond = TimeSpan.TicksPerMillisecond / 1000;
public const Double TicksPerNanosecond = TicksPerMicrosecond / 1000d;
public static Double TotalMicroseconds(this TimeSpan self) => self.Ticks * (1d / TicksPerMicrosecond);
public static Decimal TotalMicrosecondsAsDecimal(this TimeSpan self) => self.Ticks * (1m / TicksPerMicrosecond);
public static Double TotalNanoseconds(this TimeSpan self) => self.Ticks * (1d / TicksPerNanosecond);
public static Decimal TotalNanosecondsAsDecimal(this TimeSpan self) => self.Ticks * (1m / (Decimal)TicksPerNanosecond);
}
I believe this is accepted precision.
Is there any particular reason not to use decimal
? If we have the ability to have no lose of precision using a built in type, I think we should consider using it.
I am not seeing the precision here is the issue. the more concern I have is what @mj1856 raised which is the possible overflow.
for example, imagine someone wants to get the total microseconds of TimeSpan.MaxValue which will produce 9.2233720368547763E+17
(using double). then want to round trip this value back to ticks.
doing
(long) (TimeSpan.MaxValue.TotalMicroseconds() * TimeSpanEx.TicksPerMicrosecond)
will result in overflow. and if did
(ulong) (TimeSpan.MaxValue.TotalMicroseconds() * TimeSpanEx.TicksPerMicrosecond)
will produce wrong ticks which would be a negative number if you convert it to long.
@tarekgh FromUnixTimeMicroseconds
and FromUnixTimeNanoseconds
was in needed in firestore. Please stop forcing people to use that ugly new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)
anywhere
https://github.com/googleapis/google-cloud-dotnet/issues/2546#issuecomment-450796398
Thanks @Thaina for pointing at that. reading the comments there, it looks they have some limitation to run on versions which doesn't support FromUnixTimeSeconds. I think this is a matter of time to get into the point always running on versions support FromUnixTimeSeconds and then we should force using it. please keep pointing to such cases and we can help pushing if there is no good reason not to use it.
@tarekgh Versioning is one problem but the main point of discussion there is, firestore system use a number for time in unix time value but more precise than millisecond. Even they would use latest version they still couldn't use FromUnixTimeMilliSeconds
I just pointed out that there are really a system that keep unix timestamp in very precise format. It also somewhat popular system in very large and firm company like google. So FromUnixTimeMicroSeconds
FromUnixTimeNanoSeconds
and maybe FromUnixTimeTicks
should be available. It really have usage
Cc @anipik
Hi I'm a new to this. Working with an API where I get now data but with a for my unclear time stamp (format) e.g.: "636926976000000000" & "636927840000000000" The time difference is 24h ,I verified that. How can I this converted to a date/time that make any sense ?
do the following:
Console.WriteLine(new DateTime(636926976000000000)); // or new DateTime(636926976000000000).ToString() if you want to get it as string
Console.WriteLine(new DateTime(636927840000000000));
This will print it as:
5/6/2019 12:00:00 AM
5/7/2019 12:00:00 AM
Regarding use-cases, I'd like to further point out that anytime you measure time in a hot-spinning loop, nanosecond timestamps are very useful. This a pattern that's at the core of two industries at the very least (gaming and finance), and I'm sure there are many others that also use hot-spinning loops.
As a little anecdote, we used timestamps to order processed work-items in a hot spinning loop, and it turned out that we had timestamp collisions, due to lack of resolution of the 100ns. I don't think nanoseconds are an edge-case anymore.
@jundl77 thanks for the feedback. Just to understand more regarding the use-case. in the scenarios you mentioned, can you acquire the time to measure it in nanoseconds? I imagine measuring the time will require getting the current time which usually in 100-nanoseconds and not nanoseconds. no?
@tarekgh Exactly, we would like to measure time with nanosecond precision (and by extension, you might be handed a time with nanosecond precision from another server running somewhere), and right now this is only possible with 100ns precision using .NET from my understanding.
Note that no "standard" computer is going to be able to do (much, if at all) better than ~100ns Ticks on a clock at all.
For hot loops in games, they usually use QPCs (on windows, anyways, dunno about Linux) which are practically limited to ~333ns Ticks.
High performance trading computers in theory might have dedicated clock boards, but you're not going to be using C# to run those programs...
DateTime
and TimeSpan
have an internal limit of 100ns
ticks
@Clockwork-Muse sure, but if you do collect nano-second timestamps somewhere, then they are suddenly in your system. If you now pass them on to a windows server, then you do suddenly have them there, and it would be nice if they can be supported out of the box.
It can be supported already, DateTime
and TimeSpan
have constructors that take ticks
where 1 tick
is 100 ns
. It incurs no conversion cost and is just stored directly (with minimal checks for DateTime
to ensure the ticks is in range):
So if you have nanosecond timestamps, you just need to divide by 100 and store (accounting for any rounding if that is desired).
Yes DateTime/DateTimeOffset/TimeSpan cannot support less than 100ns unit. If need something to support less than that, means cannot be done in such types. but anyway, the proposed APIs here are not about that. The proposed APIs are just to allow create time by specifying microseconds or 100ns. which still can be done.
@mj1856 I think the proposed APIs are good enough. The only open issue we need to take care with before we proceed is handling the overflow in the following TimeSpan APIs:
public double TotalMicroseconds { get; }
public double TotalNanoseconds { get; }
public static TimeSpan FromMicroseconds(double value);
I think we can limit the double values we are returning to some max/min which will guarantee not overflow when reconstructing the TimeSpan with such values. Also for FromMicroseconds
we can limit the input to that max/min values too. Of course, we'll have to document the behavior as we'll truncate in some edge cases.
will you have any more concerns if we do that?
Should we also have double DateTimeOffset.ToUnixTimeSecondsDouble()
?
Suppose we have DateTimeOffset
today 12:30:15.1244
and want to get unix time seconds but with milliseconds as decimal
I would like to have FromUnixTime***
in all precision that receive double
too
@tarekgh @mattjohnsonpint it seems from above this API proposal is almost ready for review with just one question?
Just came across it as @deeprobin would have used it in one of his changes.
Currently, the value of a single
Tick
in .NET is 100ns, however there is no API surface for accessing microseconds or nanoseconds other than usingTicks
to manually calculate these numbers. In order to allow for working with these higher precision times, I am proposing the following new APIs:Approved API
Original API Proposal
Example Usage
Notes
TimeSpan.TotalMicroseconds
andTimeSpan.TotalNanoseconds
return double still since that is what the otherTotal
properties return. This could be long but I kept it as double for consistency.TimeSpan.FromMicroseconds
takes a double to keep the API consistent with the otherFrom
methods.