The example code results in Event Hubs authentication failure if access key name and the secret name are different

[msdotnetclr]

The example code results in Event Hubs authentication failure if access key name and the secret name are different.

The following line retrieves the name of the access key (shared access policy):

EH_CONN_SHARED_ACCESS_KEY_NAME  = spark.conf.get("iot.ingestion.eh.accessKeyName")

Then the example proceeds to get the actual access key value from the secret scope:

EH_CONN_SHARED_ACCESS_KEY_VALUE = dbutils.secrets.get(scope = SECRET_SCOPE, key = EH_CONN_SHARED_ACCESS_KEY_NAME)

Then EH_CONN_SHARED_ACCESS_KEY_NAME is referenced in the connection string below:

EH_CONN_STR                     = f"Endpoint=sb://{EH_NAMESPACE}.servicebus.windows.net/;SharedAccessKeyName={EH_CONN_SHARED_ACCESS_KEY_NAME};SharedAccessKey={EH_CONN_SHARED_ACCESS_KEY_VALUE}"

This only works when the name of the secret is exactly the same as the name of the Shared Access Policy, but if the two names are different, authentication will fail due to an invalid SharedAccessKeyName value in the connection string.

A new spark conf option like "iot.ingestion.eh.accessKeySecretName" is needed to store the name of the secret, keep "iot.ingestion.eh.accessKeyName" to store the actual Shared Access Policy name. Then you can use both to construct the connection string correctly.

I submitted an issue (and a companion PR) for a similar issue in the dlt-meta library maintained by Databricks Labs.

---

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

• ID: af237509-ff09-7f48-5d13-49f0118d4ca0
• Version Independent ID: af237509-ff09-7f48-5d13-49f0118d4ca0
• Content: Use Azure Event Hubs as a Delta Live Tables data source - Azure Databricks
• Content Source: databricks/delta-live-tables/event-hubs.md
• Service: azure-databricks
• GitHub Login: @mssaperla
• Microsoft Alias: saperla

[SaibabaBalapur-MSFT]

@msdotnetclr Thanks for your feedback! We will investigate and update as appropriate.

[RamanathanChinnappan-MSFT]

@msdotnetclr I've delegated this to @mssaperla, a content author, to review and share their valuable insights.

[kateglee-db]

Thanks for providing feedback that helps improve our documentation. We've created an internal work item (DOC-11128) to address your feedback. The timeline for resolution varies based on resourcing.

please-close