microsoft / delta-kusto

Engine able to compute delta between ADX clusters (and/or Kusto scripts) and generate update scripts
MIT License
47 stars 20 forks source link

Partitioning policy behaviour #176

Open matipos2 opened 5 months ago

matipos2 commented 5 months ago

I'm creating new table. Create statement and partitioning policy is in separate kql file. Both file are read by delta-kusto. Partitioning policy is not being recognized to be executed by the tool. In the result no partitioning policy will be applied.

But, when the table exists, partitioning policy is executed each time, even when no change is made.

Status on database (after finally it's created):

.show table MyTable policy partitioning

"PolicyName": DataPartitioning,
"EntityName": [MyDB].[MyTable],
"Policy": {
  "PartitionKeys": [
    {
      "ColumnName": "Timestamp",
      "Kind": "UniformRange",
      "Properties": {
        "Reference": "1970-01-01T00:00:00",
        "RangeSize": "7.00:00:00",
        "OverrideCreationTime": true
      }
    }
  ],
  "EffectiveDateTime": "2024-06-21T19:20:11.3013261Z",
  "MinRowCountPerOperation": 0,
  "MaxRowCountPerOperation": 0,
  "MaxOriginalSizePerOperation": 0
},
"ChildEntities": ,
"EntityType": Table

Result by .show database schema as csl script:

.alter table MyTable policy partitioning "{\"PartitionKeys\":[{\"ColumnName\":\"Timestamp\",\"Kind\":2,\"Properties\":{\"Reference\":\"1970-01-01T00:00:00\",\"RangeSize\":\"7.00:00:00\",\"OverrideCreationTime\":true}}],\"EffectiveDateTime\":\"2024-06-21T19:20:11.5044466Z\",\"MinRowCountPerOperation\":0,\"MaxRowCountPerOperation\":0,\"MaxOriginalSizePerOperation\":0}"

My script:

.alter table MyTable policy partitioning
```{
        "PartitionKeys": [
            {
                "ColumnName": "Timestamp",
                "Kind": "UniformRange",
                "Properties": {
                    "Reference": "1970-01-01T00:00:00",
                    "RangeSize": "7.00:00:00",
                    "OverrideCreationTime": true
                }
            }
        ]
    }