search

microsoft / delta-kusto

Engine able to compute delta between ADX clusters (and/or Kusto scripts) and generate update scripts
MIT License
47 stars 19 forks source link

Delta on Materialized View #91

Open GowthamDisys opened 2 years ago

margelatu commented 2 years ago

+1 to this request. We are using a materialised view for the main queries against the ADX database and at this time do not have a proper mechanism for managing the evolution of the schema. If delta-kusto would support materialized views and external tables, it would help tremendously.

vplauzon commented 2 years ago

Hi. Thank you for the feedback!

The reason we didn't include it at first is the unknown surrounding backfill. Delta Kusto is able to push and pull schema from a Kusto DB. In its current form, the backfill = true / false isn't returned on .show materialized-views but we can deduce it from the effectiveDateTime.

So how should Delta Kusto behave with that? Should it take the effectiveDate from its source?

What do you think?

vplauzon commented 2 years ago

This is being worked on right now. The solution is that we will not support the backfill mode.

dnavas77 commented 2 years ago

Hello @vplauzon, just wondering if this feature implemented?

pottsork commented 1 year ago

Hi @vplauzon! Any updates on materialized views support?

vplauzon commented 1 year ago

Awaiting new version of Kusto.Language to make progress on that front. They should release the new version in a few weeks.

vvelghe-iba commented 1 year ago

Hello @vplauzon. Did the latest update on Kusto allow you to move forward on materialized views support?

delta-kusto is super great btw, thx!

sandupetrasco commented 9 months ago

Hello @vplauzon & team! Any updates here?

faketepopaj commented 8 months ago

Hi @vplauzon Also with the latest update I am receiving Error: 'DeltaKustoLib.DeltaException ; Can't handle CommandKind 'CreateMaterializedView''.

Is there any update on that or any plan when will it be implemented?

thomjak commented 1 month ago

Hey @vplauzon, any updates here?

Does anybody have suggestions for workarounds?