search

Azure / azure-kusto-dotnet

Azure Data Explorer (Kusto) SDK for .NET
MIT License
8 stars 5 forks source link

Is there some limit in azure data explorer to difference between columns mapping and table schema? #17

Closed nicolaslattuada closed 1 year ago

nicolaslattuada commented 1 year ago

I am experiencing some very strange behavior when ingesting data to kusto using dotnet client.

In one table, we decided that there were some columns that we don't want to store anymore.

So I removed them from the mapping, in my dotnet application (29 out of 94 columns), and suddenly the data stopped being stored in some other column for which I hadn't removed the mapping.

And to my surprise when testing, if I restore one column from mapping (only some work, not any of them) from the 29 removed, then the data into this other column appears to be stored again.

I also noticed that if I drop the same columns from the schema that I removed from mapping, then the data is stored correctly again.

So my question is, what is the rule for how much different the mapping can be from the schema?

(if needed I can provide sample code for reproducing the issue)

Thanks!

yogilad commented 1 year ago

It would best if you could provide the table schema, the mapping and a sample data. Also, please check if there are update policies on your table which might calculate the value of some columns.

nicolaslattuada commented 1 year ago

Hi @yogilad Thank you very much for taking time to reply, I added you and David to a private repo containing minimum code that allows to reproduce the issue. I added table schema in comments inside the Program.cs file. Any doubts please let me know !

nicolaslattuada commented 1 year ago

Closing this issue here, will communicate using internal Kusto Discussions mailing list