JSON Serializer settings not honored?

[Mortana89]

We're seeing behavior locally, that our object's datetimeoffset properties are saved as local datetimeoffsets (with an offset) to the database, instead of UTC datetimeoffsets (which is the behavior on our servers in Azure, as that is their 'local' timezone)

We are specifying the following on a cosmossettings object:

DefaultDatabaseThroughput = 400;
            OnDatabaseThroughput = Cosmonaut.Configuration.ThroughputBehaviour.UseDatabaseThroughput;
            JsonSerializerSettings = new Newtonsoft.Json.JsonSerializerSettings()
            {
                DateParseHandling = Newtonsoft.Json.DateParseHandling.DateTimeOffset,
                DateTimeZoneHandling = Newtonsoft.Json.DateTimeZoneHandling.Utc
            };

This should make sure that any object that is being serialized/deserialized, will have DateTimeOffsets converted to UTC, at least, that's what I'm expecting.

When I test in a simple console application of the Cosmos .NET Samples, I can clearly see that it works as expected. DateTimeOffsets are saved correctly.

[Elfocrash]

Just tested this locally. Your problem is cased because you set the DateParseHandling to DateTimeOffset. Cosmonaut's ToCosmonautDocument actually does the right thing and adds the +0:00 suffix because it's honoring your offset setting. If you wanna get that as expected you need to change the DateParseHandling to None and leave the DateTimeZoneHandling to Utc. If the SDK is performing differently then that's a bug on the SDK not Cosmonaut.

[Elfocrash]

Basically

var jsonSerializerSettings = new JsonSerializerSettings()
{
    DateParseHandling = DateParseHandling.None,
    DateTimeZoneHandling = DateTimeZoneHandling.Utc
};

Gives you: 2019-07-24T09:51:26Z

And

var jsonSerializerSettings = new JsonSerializerSettings()
{
    DateParseHandling = DateParseHandling.DateTimeOffset,
    DateTimeZoneHandling = DateTimeZoneHandling.Utc
};

Gives you: 2019-07-24T09:51:26+00:00.

This is how the serializer is meant to work. They are both UTC ISO 8601 DateTimes

[Mortana89]

Ow okey, didn't know that adding the DateTimeOffset handling would cause the datetimeoffset to be saved as +xx:xx. However, would the timezone handling then not make sure the offset is 00:00? Because that's not what I'm seeing here. In any case I will modify the code to remove the parse handling

[Mortana89]

And thanks for your prompt reply!

[Elfocrash]

However, would the timezone handling then not make sure the offset is 00:00? Because that's not what I'm seeing here. In any case I will modify the code to remove the parse handling

The offset will always be +00:00 if you set DateParseHandling = DateParseHandling.DateTimeOffset and DateTimeZoneHandling = DateTimeZoneHandling.Utc. The DateTimeOffset value just tells the serializer that "I prefer +00:00 than Z at the end" which indicate the same thing. The base of the date time will be UTCfied.

[Mortana89]

And that’s what’s not happening, I get +02:00 while I expected +00:00 with these settings…

Van: Nick Chapsas notifications@github.com Verzonden: woensdag 24 juli 2019 13:04 Aan: Elfocrash/Cosmonaut Cosmonaut@noreply.github.com CC: Yoni Nijs yoni.nijs@zerofriction.co; Author author@noreply.github.com Onderwerp: Re: [Elfocrash/Cosmonaut] JSON Serializer settings not honored? (#102)

However, would the timezone handling then not make sure the offset is 00:00? Because that's not what I'm seeing here. In any case I will modify the code to remove the parse handling

The offset will always be +00:00 if you set DateParseHandling = DateParseHandling.DateTimeOffset and DateTimeZoneHandling = DateTimeZoneHandling.Utc. The DateTimeOffset value just tells the serializer that "I prefer +00:00 than Z at the end" which indicate the same thing. The base of the date time will be UTCfied.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/Elfocrash/Cosmonaut/issues/102?email_source=notifications&email_token=AK4YUUNGE2MKBFSCURSLEFTQBAZLFA5CNFSM4IGOJCSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2V7THA#issuecomment-514587036, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AK4YUUL4SECE5HCVBI5G42DQBAZLFANCNFSM4IGOJCSA.

[Elfocrash]

Ok here is my code

serviceCollection.AddCosmosStore<Book>("localtest", "https://localhost:8081",
    "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==",
    settings =>
{
    settings.ConnectionPolicy = connectionPolicy;
    settings.DefaultCollectionThroughput = 5000;
    settings.JsonSerializerSettings = new JsonSerializerSettings()
    {
        DateParseHandling = DateParseHandling.DateTimeOffset,
        DateTimeZoneHandling = DateTimeZoneHandling.Utc
    };
});

Here is my object (and I'm even specifying that it is local. I am in London so our time now is UTC +1)

new Car
{
    Id = Guid.NewGuid().ToString(),
    Name = "Car " + i,
    DateTime = DateTime.SpecifyKind(DateTime.Now.AddHours(3), DateTimeKind.Local)
}

and here is the object created

{
    "Name": "Car 0",
    "CosmosEntityName": "Cosmonaut.Console.Car",
    "DateTime": "2019-07-24T14:03:45.848812+00:00",
    "id": "2ffcd662-713e-4b70-a3a0-0b386a7bcd52",
    "_rid": "H-0RAPvfAGfLAAAAAAAAAA==",
    "_self": "dbs/H-0RAA==/colls/H-0RAPvfAGc=/docs/H-0RAPvfAGfLAAAAAAAAAA==/",
    "_etag": "\"00000000-0000-0000-420f-7d9de4b601d5\"",
    "_attachments": "attachments/",
    "_ts": 1563966237
}

It correctly goes one hour backwards to UTC and then adds 3 on top of that making the time 14:03

[Mortana89]

Can you make your datetime a datetimeoffset property and try again?

Van: Nick Chapsas notifications@github.com Verzonden: woensdag 24 juli 2019 13:10 Aan: Elfocrash/Cosmonaut Cosmonaut@noreply.github.com CC: Yoni Nijs yoni.nijs@zerofriction.co; Author author@noreply.github.com Onderwerp: Re: [Elfocrash/Cosmonaut] JSON Serializer settings not honored? (#102)

Ok here is my code

serviceCollection.AddCosmosStore("localtest", "https://localhost:8081",

"C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==",

    settings =>

{

    settings.ConnectionPolicy = connectionPolicy;

    settings.DefaultCollectionThroughput = 5000;

    settings.JsonSerializerSettings = new JsonSerializerSettings()

    {

           DateParseHandling = DateParseHandling.DateTimeOffset,

           DateTimeZoneHandling = DateTimeZoneHandling.Utc

    };

});

Here is my object (and I'm even specifying that it is local. I am in London so our time now is UTC +1)

new Car

{

    Id = Guid.NewGuid().ToString(),

    Name = "Car " + i,

    DateTime = DateTime.SpecifyKind(DateTime.Now.AddHours(3), DateTimeKind.Local)

}

and here is the object created

{

"Name": "Car 0",

"CosmosEntityName": "Cosmonaut.Console.Car",

"DateTime": "2019-07-24T14:03:45.848812+00:00",

"id": "2ffcd662-713e-4b70-a3a0-0b386a7bcd52",

"_rid": "H-0RAPvfAGfLAAAAAAAAAA==",

"_self": "dbs/H-0RAA==/colls/H-0RAPvfAGc=/docs/H-0RAPvfAGfLAAAAAAAAAA==/",

"_etag": "\"00000000-0000-0000-420f-7d9de4b601d5\"",

"_attachments": "attachments/",

"_ts": 1563966237

}

It correctly goes one hour backwards to UTC and then adds 3 on top of that making the time 14:03

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/Elfocrash/Cosmonaut/issues/102?email_source=notifications&email_token=AK4YUUJJO4X7Z6HRQEXHXJLQBA2AVA5CNFSM4IGOJCSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2V772I#issuecomment-514588649, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AK4YUUI7SQ22ERMQROK7VYDQBA2AVANCNFSM4IGOJCSA.

[Elfocrash]

Well yeah if i change it to a DateTimeOffset then ofc it would have the offset value to represent the offset. That's the whole point. Now the same thing will return a value like this: 2019-07-24T15:14:32.4911889+01:00. It's preserving the timezone as it should be. DateTimeOffset in UTC doesn't mean that you will have always 00:00 offset. Am I missing something?

[Mortana89]

I must be blind then or something 😃 Basically what I’m trying to achieve is to have all our dates in the database to be in UTC. My thought would be that, if I set the serializer settings as provided, that, if I have a local datetimeoffset, the serializer would convert it to a UTC datetimeoffset, setting the offset to 00:00 and thus removing (in this case) 1h from the time.

Van: Nick Chapsas notifications@github.com Verzonden: woensdag 24 juli 2019 13:19 Aan: Elfocrash/Cosmonaut Cosmonaut@noreply.github.com CC: Yoni Nijs yoni.nijs@zerofriction.co; Author author@noreply.github.com Onderwerp: Re: [Elfocrash/Cosmonaut] JSON Serializer settings not honored? (#102)

Well yeah if i change it to a DateTimeOffset then ofc it would have the offset value to represent the offset. That's the whole point. Now the same thing will return a value like this: 2019-07-24T15:14:32.4911889+01:00. It's preserving the timezone as it should be. DateTimeOffset in UTC doesn't mean that you will have always 00:00 offset. Am I missing something?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/Elfocrash/Cosmonaut/issues/102?email_source=notifications&email_token=AK4YUUJ5527RC2UHIQBK7ULQBA3CZA5CNFSM4IGOJCSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2WATOQ#issuecomment-514591162, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AK4YUUL4WTLUUPYA34WCYODQBA3CZANCNFSM4IGOJCSA.

[Elfocrash]

Can you ensure the behaviour with just plain Json.Net?

[Mortana89]

See:

I've uploaded it to a seperate repo: https://github.com/Mortana89/azure-cosmos-dotnet-v2/blob/master/samples/code-samples/DocumentManagement/Program.cs, line 78 I've specified UTC. Line 305 I'm populating a local DateTimeOffset :/

[Elfocrash]

The example you linked works because the DateParseHandling value isn't set to DateTimeOffset. If I do the same thing with Cosmonaut and don't set the DateParseHandling then I get the same result:

This is all expected behavior from both parties.

[Mortana89]

Hi @Elfocrash ,

Thanks for the prompt reply! I can confirm that removing the datetimeoffset handling option, causes the DateTimeOffsets to be serialized to a UTC datetime. I was in the assumption that I could have both :) Thanks!