search

Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.59k stars 2.8k forks source link

ModuleNotFound Error when using eventhub-azure in 5.12.2 on Azure Databricks 12.2.x with azure-eventhub-checkpointstoreblob-aio==1.1.4 when using BlobCheckpointStore in Pyhton #38013

Open defaultUser3214 opened 6 hours ago

defaultUser3214 commented 6 hours ago

Describe the bug I install on a Azure databricks notebook the packages: !pip install azure-eventhub==5.12.2 azure-eventhub-checkpointstoreblob-aio==1.1.4 Then, I get the issue that {pth}/envs/myenv/lib/python3.9/site-packages/ on the usual path for site-packages only a part of the azure-eventhub-checkpointstoreblob-aio package is installed, i.e., only until azure/eventhub/extensions, however the subfolders in the azure/eventhub/extensions are not there. Surprisingly, the package is completely installed with all subfolders of azure/eventhub/extension at {pth}/cluster_libraries/python/lib/python3.9/site-packages/. Somehow, it seems as if the package has a bug when getting installed on the latest Azure with Databricks 12.2.x. If you instead use azure-eventhub=5.12.1, the problem does not appear and everything works fine.

To Reproduce Steps to reproduce the behavior:

  1. Install !pip install azure-eventhub==5.12.2 azure-eventhub-checkpointstoreblob-aio==1.1.4 on a Databricks 12.2.x notebook on Azure.
  2. From azure.eventhub.extensions.checkpointstoreblobaio import BlobCheckpointStore in your notebook.
  3. You will get an ModuleNotFound-Error.
  4. If you instead install azure-eventhub==5.12.1 and azure-eventhub-checkpointstoreblob-aio==1.1.4 no issue occurs when importing the BlobCheckpointStore

Expected behavior I expect that I can use the BlobCheckPointStore and that all the files including subfolders of the package azure-eventhub-checkpointstoreblob-aio are installed at {pth}/envs/myenv/lib/python3.9/site-packages/ and not suddenly at {pth}/cluster_libraries/python/lib/python3.9/site-packages/ where my Databricks cannot access them. Screenshots None

Additional context The packages azure-eventhub==5.12.2 azure-eventhub-checkpointstoreblob-aio==1.1.4 work on a normal Ubuntu without problems as a side-note.

github-actions[bot] commented 6 hours ago

Thank you for your feedback. Tagging and routing to the team member best able to assist.

kashifkhan commented 4 hours ago

Thank you for the feedback. We will investigate this and get back to you asap.