k2v-academy / K2View-Academy

Other
0 stars 0 forks source link

Changing DATETIME_FORMAT in Fabric #999

Closed yBqdo2VLaCdftea1MqgSdtEhrPZtV5oJRr4eIUo closed 7 months ago

yBqdo2VLaCdftea1MqgSdtEhrPZtV5oJRr4eIUo commented 8 months ago

Hello, is there any special step required to change DATETIME_FORMAT for Fabric?

W/o changing it - keeping commented in config.ini as #DATETIME_FORMAT=yyyy-MM-dd HH:mm:ss.SSS I'm getting my Datetime values stored in Fabric tables as:

image

After changing it as following:

image

Restarting Fabric, delete my instance and resync - I still see the same format..

image

what do I miss?


Thank you, Andrey

157aq9Ud3GJlkIVikTS1SzhhiP3x1Vrj0jAjZ3X commented 8 months ago

Hi

Can you share how it looks like in the source DB? is this with ms? What is the source DB? (oracle? other?)

yBqdo2VLaCdftea1MqgSdtEhrPZtV5oJRr4eIUo commented 7 months ago

In this case I tested with local Postgres DB (our demo CRM_DB ) - and it has no ms in DB:

image
157aq9Ud3GJlkIVikTS1SzhhiP3x1Vrj0jAjZ3X commented 7 months ago

So can you provide the concrete example where it did not work?

yBqdo2VLaCdftea1MqgSdtEhrPZtV5oJRr4eIUo commented 7 months ago

Hello - yes, please see one of these.

This is my config:

image

This is my get and data in fabric (with .0 in the end):

image

This is the same data in source DB:

image

and when it is published to ElasticSearch via CDC for this table - it is failed on ES side as datetime format is unexpected (it is coming with .0, while ES expecting it only w/o ms or with .sss for millis)


Best regards, Andrey

157aq9Ud3GJlkIVikTS1SzhhiP3x1Vrj0jAjZ3X commented 7 months ago

Hi Are you using Date as the format inside Fabric? If so, the config.ini is irrelevant. This is used to previous versions, where Fabric did not support Date and holds dates as text. For this, you could control on the format using the config definition (instead user had to convert it in implementation). When using Date format, then you shall use the date-to-text SQL function, each time according to the required format,

yBqdo2VLaCdftea1MqgSdtEhrPZtV5oJRr4eIUo commented 7 months ago

Yes, in this case I have it as 'Datetime' in fabric table