DarkWanderer / ClickHouse.Client

.NET client for ClickHouse
MIT License
315 stars 62 forks source link

Inserting DateTime64(3) into ClickHouse #298

Closed plewam closed 1 year ago

plewam commented 1 year ago

Hi @DarkWanderer,

First of all thanks for creating this ClickHouse client library. I have the requirement for storing datetime values in millisecond precision. In order to archieve that I choose DateTime64(3) data type within ClickHouse. However I cannot get it to work with the library. It always throws me below exception:

I already tried the following conversions:

row[data.Name] = ((DateTime)value).ToString("yyyy-MM-dd HH:mm:ss.fff"); //Convert to string
row[data.Name] = ((DateTimeOffset)((DateTime)value)).ToUnixTimeMilliseconds(); //Convert to unix milliseconds
row[data.Name] = value; // C# DateTime (no conversion)
ClickHouse.Client.Copy.ClickHouseBulkCopySerializationException
  HResult=0x80131500
  Nachricht = Error when serializing data
  Quelle = ClickHouse.Client
  Stapelüberwachung:
   bei ClickHouse.Client.Copy.ClickHouseBulkCopy.<WriteToServerAsync>d__25.MoveNext()
   bei Dft.Inside.DataAgent.Sinks.Classes.DataAgentSinkClickHouse.<PushEntitiesToSinkInner>d__0.MoveNext() in D:\DftInternalCollection\Inside\trunk\source\Dft.Inside.DataAgent.Sinks\Classes\DataAgentSinkClickHouse.cs: Zeile95
   bei Dft.Inside.DataAgent.Sinks.Base.DataAgentSinkBase.<PushEntitiesToSink>d__19.MoveNext() in D:\DftInternalCollection\Inside\trunk\source\Dft.Inside.DataAgent.Sinks\Base\DataAgentSinkBase.cs: Zeile86

  Diese Ausnahme wurde ursprünglich von dieser Aufrufliste ausgelöst:
    [Externer Code]

Innere Ausnahme 1:
ArgumentException: Cannot convert value to datetime
DarkWanderer commented 1 year ago

Hi,

It is a bit strange as both DateTime and DateTimeOffset values should be accepted - the code which throws this exception supports both:

    public override void Write(ExtendedBinaryWriter writer, object value)
    {
        var instant = value switch
        {
            DateTimeOffset dto => Instant.FromDateTimeOffset(dto),
            DateTime dt => ToZonedDateTime(dt).ToInstant(),
            _ => throw new ArgumentException("Cannot convert value to datetime"),
        };
        writer.Write(ToClickHouseTicks(instant));
    }

ClickHouseBulkCopySerializationException has Row and Index values pointing at specific value which caused an error - can you check what is the value of exception.Row[exception.Index] in your case?