MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.25k stars 21.42k forks source link

Hive metastore #104599

Closed LauraDamianTNA closed 1 year ago

LauraDamianTNA commented 1 year ago

Regarding this sentence "Every Azure Databricks deployment has a central Hive metastore accessible by all clusters to persist table metadata." does it still hold for Databricks clusters that use a runtime 11.1 or above which offer a Unity Catalog by default?

Document Details

Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

SaibabaBalapur-MSFT commented 1 year ago

@LauraDamianTNA Thanks for your feedback! We will investigate and update as appropriate.

Naveenommi-MSFT commented 1 year ago

Hi @LauraDamianTNA Could you please go through this below link for your reference. It might be helpful.

https://learn.microsoft.com/en-us/azure/databricks/data/metastores/external-hive-metastore#:~:text=Hive%202.3.7%20(Databricks,to%20them.

Here Information about objects across all catalogs within the metastore. https://learn.microsoft.com/en-us/azure/databricks/release-notes/runtime/11.1#unity-catalog-is-ga

If there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.

Naveenommi-MSFT commented 1 year ago

@LauraDamianTNA We are going to close this thread, if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.

LauraDamianTNA commented 1 year ago

Thanks @Naveenommi-MSFT. My understanding is that after Catalog Unity was released, the hive metastore is not provided anymore, but I could not find anywhere this clearly explained and I can not use it without attaching an external one as you showed in the links. This link https://docs.databricks.com/data-governance/unity-catalog/hive-metastore.html says only this: "If your workspace was in service before it was enabled for Unity Catalog, it likely has a Hive metastore that contains data that you want to continue to use." In the meantime, I am using an older runtime to work with the Hive metastore.

Naveenommi-MSFT commented 1 year ago

Hi @LauraDamianTNA Thank you for your feedback!

Since this issue isn't directly related to improving our docs, and to gain a better understanding of your issue, I'd recommend working closer with our support team via an [Azure support request] (https://docs.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request). Or you can leverage our Q&A forum by posting your issue there so our community, and MVPs can further assist you in troubleshooting this issue or finding potential workarounds.

[Teams Q&A forum] (https://docs.microsoft.com/en-us/answers/topics/46488/office-teams-windows-itpro.html) for technical questions about the configuration and administration of Microsoft Teams on Windows. [Microsoft Teams Community forum] (https://answers.microsoft.com/en-us/msteams/forum?sort=LastReplyDate&dir=Desc&tab=All&status=all&mod=&modAge=&advFil=&postedAfter=&postedBefore=&threadType=All&isFilterExpanded=false&page=1

It might be helpful.