MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.28k stars 21.47k forks source link

Synapse Lake Database documentation: table schema and metadata enforcement behaviour not documented #87106

Closed pSciortino closed 5 months ago

pSciortino commented 2 years ago

In this subsection of the Synapse Lake Database documentation, it is explained how one can edit the metadata at the column level. To me, this naturally raises the question of whether and, if so, how this metadata (e.g. setting a column as non-null) is enforced. I would assume that no constraint on the source file(s) in the storage is enforced and that an error message is simply raised at read time if the underlying data does not comply with the provided metadata information, but this is not explicitly explained. Explicitely setting expectations in this regard will become even more relevant once support for the apache delta format is added, as the latter does feature schema on write which would make metadata enforcement possible in the data source.

On a similar note, what does happen when deleting a column? Does it reflect on the source data or just in the "select statement" of the external table?

I think it would make sense to set those expectations as part of the lake database documentation.


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

YutongTie-MSFT commented 2 years ago

Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

bandersmsft commented 5 months ago

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. We are closing this issue for now, but if you feel that it's still a concern, please respond and let us know. If you determine another possible update to our documentation, please don't hesitate to reach out again. #please-close