Parquet export trims SqlDateTime time

[TheSpy]

Hello, When I try to export data to parquet from SQLServer table with a column type SqlDateTime, the column is converted to string which does not contain time data anymore, but only date.

I tried setting TypeConverterFormatSpec, but it didn't help. What am I missing?

        ChoETLFrxBootstrap.IsSandboxEnvironment = true;

        var cmd = new SqlCommand($"sql cmd", conn);

        var dr = await cmd.ExecuteReaderAsync();

        using var w = new ChoParquetWriter(fileName)
            .Configure(c => c.Culture = CultureInfo.InvariantCulture)
            .Configure(c => c.TypeConverterFormatSpec = new ChoTypeConverterFormatSpec { DateTimeFormat = "o" })
            .Configure(c => c.LiteParsing = true)
            .Configure(c => c.RowGroupSize = 5000)
            .NotifyAfter(100000)
            .OnRowsWritten((o, e) => $"Rows Loaded: {e.RowsWritten} <-- {DateTime.Now}".Print());

        w.Write(dr);

[Cinchoo]

currently configuration TypeConverterFormatSpec is not respected by writer. I'll fix it.

In the mean time, use the global one

ChoTypeConverterFormatSpec.Instance.DateTimeFormat = "o";

[TheSpy]

I have tried the global one too - the outcome is the same.

Thank you for looking into it!

[Cinchoo]

pls stop using 'LiteParsing', see if it works.

[TheSpy]

Unfortunately, it didn't help.

[Cinchoo]

Got it, type spec is respected only in POCO model reader.

Let me fix it. If you want to load them successfully, define POCO class and use the writer.

Will keep you posted with the fix.

[Cinchoo]

applied fix, released v1.0.1.26. try it and let me know.

[TheSpy]

It works now, thank you very much! 👍🏽