search

Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.61k stars 2.82k forks source link

Unable to list blob files when all other actions work #36597

Closed derekahuang closed 2 months ago

derekahuang commented 3 months ago

Describe the bug I have setup my BlobServiceClient and I am able to check that it exists, upload files, and download files via exists() and get_blob_client. This verifies that my credentials are correct and the container_client is working. However, when I attempt to list files, I always get the same error:

   File "/app/src/python/engine/py_image.binary.runfiles/strap/src/python/engine/jobs/static_asset.py", line 173, in download_files_from_remote
    for blob in blobs_list:
  File "/app/src/python/engine/py_image.binary.runfiles/pip_deps_azure_core/azure/core/paging.py", line 128, in __next__
    return next(self._page_iterator)
  File "/app/src/python/engine/py_image.binary.runfiles/pip_deps_azure_core/azure/core/paging.py", line 84, in __next__
    self.continuation_token, self._current_page = self._extract_data(self._response)
  File "/app/src/python/engine/py_image.binary.runfiles/pip_deps_azure_storage_blob/azure/storage/blob/_list_blobs_helper.py", line 92, in _extract_data_cb
    self.current_page = [self._build_item(item) for item in self._response.segment.blob_items]
AttributeError: 'NoneType' object has no attribute 'blob_items'

To Reproduce I am running this inside a Docker container 

        blob_service_client = BlobServiceClient.from_connection_string(credentials.connection_string)
        container_client = blob_service_client.get_container_client('static-asset-test')
        blobs_list = container_client.list_blobs()
        for blob in blobs_list:
            print(blob)

Expected behavior List to work

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Output of pip list

accelerate==0.32.1
adlfs==2024.4.1
aiofiles==23.2.1
aiohttp==3.9.5
aiohttp-cors==0.7.0
aiosignal==1.2.0
alabaster==0.7.12
alembic==1.10.4
allennlp @ https://snorkel-python-wheels.s3.us-west-2.amazonaws.com/allennlp-2.10.2-py3-none-any.whl
aniso8601==9.0.1
anyio==4.4.0
appdirs==1.4.4
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asn1crypto==1.5.1
astroid==3.2.2
asttokens==2.2.1
astunparse==1.6.3
async-exit-stack==1.0.1
async-generator==1.10
async-lru==2.0.4
async-timeout==4.0.2
attrs==23.2.0
autopep8==2.0.4
azure-ai-formrecognizer==3.2.0
azure-common==1.1.28
azure-core==1.24.2
azure-datalake-store==0.0.53
azure-identity==1.17.1
azure-storage-blob==12.12.0
babel==2.10.3
backoff==2.2.1
baron==0.10.1
base58==2.1.1
bayesian-optimization==1.2.0
bcrypt==3.1.7
beautifulsoup4==4.8.2
bleach==4.1.0
blessed==1.19.1
blis==0.7.9
boto3==1.34.106
boto3-stubs[ec2,efs,route53,s3,sagemaker,sagemaker-runtime,secretsmanager]==1.34.106
botocore==1.34.106
botocore-stubs==1.34.94
boxdetect==1.0.2
cached-path==1.6.3
cachetools==5.3.3
catalogue==2.0.8
certifi==2024.6.2
certipy==0.1.3
cffi==1.16.0
cgroupspy==0.2.2
chardet==5.0.0
charset-normalizer==2.0.6
checksumdir==1.2.0
click==8.1.7
clip @ https://snorkel-python-wheels.s3.us-west-2.amazonaws.com/clip-1.0-py3-none-any.whl
cloudpickle==2.2.1
colorama==0.4.6
colorful==0.5.4
comm==0.2.2
confection==0.0.3
contextlib2==21.6.0
contourpy==1.1.1
cramjam==2.7.0
cryptography==42.0.8
cupy-cuda11x==13.1.0
cycler==0.10.0
cymem==2.0.8
dask[array,dataframe]==2022.4.0
databricks-sql-connector==2.9.3
datasets==2.14.4
debugpy==1.8.1
decompyle3==3.9.0
decorator==4.4.2
defusedxml==0.6.0
deprecated==1.2.10
deprecation==2.1.0
deslib==0.3.5
dill==0.3.5.1
distlib==0.3.8
distributed==2022.4.0
distro==1.9.0
docker==4.1.0
docopt==0.6.2
docstring-parser==0.13
docstring-to-markdown==0.15
docutils==0.19
emoji==2.8.0
en-core-web-md @ https://github.com/explosion/spacy-models/releases/download/en_core_web_md-3.4.1/en_core_web_md-3.4.1-py3-none-any.whl
en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.4.1/en_core_web_sm-3.4.1-py3-none-any.whl
entrypoints==0.3
escapism==1.0.1
et-xmlfile==1.1.0
evaluate==0.4.0
exceptiongroup==1.1.0
execnet==2.1.1
executing==1.2.0
fairscale==0.4.6
faiss-cpu==1.7.4
faiss-gpu==1.7.2
fakeredis==2.18.0
fastapi==0.110.0
fastcluster==1.2.6
fastjsonschema==2.15.3
fastparquet==0.8.3
fastrlock==0.8.1
filelock==3.7.0
flake8==7.0.0
flask==2.2.5
fonttools==4.43.0
fqdn==1.5.1
freezegun==1.3.1
frozendict==2.3.4
frozenlist==1.4.0
fsspec[http]==2024.6.0
ftfy==5.8
future==0.18.3
gcsfs==2024.6.0
gitdb==4.0.11
gitpython==3.1.43
google-api-core[grpc]==2.11.0
google-auth==2.17.3
google-auth-oauthlib==0.4.6
google-cloud-aiplatform==1.43.0
google-cloud-bigquery==3.11.1
google-cloud-bigquery-storage==2.14.2
google-cloud-core==2.3.2
google-cloud-resource-manager==1.10.2
google-cloud-storage==2.10.0
google-crc32c==1.3.0
google-pasta==0.2.0
google-resumable-media==2.3.3
googleapis-common-protos[grpc]==1.56.4
gpustat==1.0.0
grafanalib==0.6.3
graphene==3.3
graphql-core==3.2.3
graphql-relay==3.2.0
greenlet==1.1.3
grpc-google-iam-v1==0.12.6
grpcio==1.56.2
grpcio-status==1.43.0
gunicorn==22.0.0
h11==0.14.0
h5py==3.7.0
httpcore==1.0.5
httptools==0.6.1
httpx==0.27.0
huggingface-hub==0.23.4
hyperopt==0.2.4
idna==3.7
imagesize==1.4.1
importlib-metadata==6.11.0
imutils==0.5.4
influxdb-client==1.10.0
iniconfig==1.1.1
ipykernel==6.29.4
ipython==8.24.0
ipywidgets==8.1.2
isodate==0.6.1
isoduration==20.11.0
isort==5.13.2
itsdangerous==2.1.2
jedi==0.19.1
jinja2==3.1.4
jira==3.4.1
jmespath==0.9.5
joblib==1.2.0
json5==0.9.24
jsonnet==0.20.0
jsonpointer==2.4
jsonschema[format-nongpl]==4.20.0
jsonschema-specifications==2023.11.2
jupyter-client==8.6.2
jupyter-core==5.7.2
jupyter-events==0.10.0
jupyter-lsp==2.2.5
jupyter-resource-usage==1.0.2
jupyter-server==2.14.1
jupyter-server-mathjax==0.2.6
jupyter-server-proxy==4.2.0
jupyter-server-terminals==0.5.3
jupyter-telemetry==0.1.0
jupyterhub==4.1.5
jupyterlab==4.2.1
jupyterlab-git==0.50.1
jupyterlab-lsp==5.1.0
jupyterlab-pygments==0.3.0
jupyterlab-server==2.27.2
jupyterlab-snorkel-auth @ https://snorkel-python-wheels.s3.us-west-2.amazonaws.com/jupyterlab_snorkel_auth-0.1.2-py3-none-any.whl
jupyterlab-widgets==3.0.11
jupytext==1.16.2
kiwisolver==1.1.0
kubernetes==25.3.0
langcodes==3.3.0
line-profiler==3.5.1
llvmlite==0.41.1
lmdb==1.4.1
locket==1.0.0
lupa==1.14.1
lxml==4.9.3
lz4==4.3.2
mako==1.2.2
markdown==3.3.4
markdown-it-py==3.0.0
markupsafe==2.1.5
matplotlib==3.7.3
matplotlib-inline==0.1.6
mccabe==0.7.0
mdit-py-plugins==0.4.0
mdurl==0.1.2
memory-profiler==0.57.0
memray==1.10.0
milksnakex==0.1.6
minio==7.0.3
mirakuru==2.2.0
mistune==3.0.2
mlflow==2.14.1
mlflow-skinny==2.14.1
mock==4.0.1
more-itertools==10.2.0
moto[s3]==5.0.2
mpld3==0.5.9
mpmath==1.3.0
msal==1.29.0
msal-extensions==1.2.0
msgpack==1.0.0
msrest==0.7.1
multidict==4.7.6
multiprocess==0.70.13
munkres==1.1.2
murmurhash==1.0.10
mypy-boto3-ec2==1.34.114
mypy-boto3-efs==1.34.0
mypy-boto3-route53==1.34.31
mypy-boto3-s3==1.34.105
mypy-boto3-sagemaker==1.34.116
mypy-boto3-sagemaker-runtime==1.34.0
mypy-boto3-secretsmanager==1.34.109
nbclient==0.10.0
nbconvert==7.16.3
nbdime==4.0.1
nbformat==5.10.4
nest-asyncio==1.6.0
networkx==2.8.8
nltk==3.8.1
notebook @ https://snorkel-python-wheels.s3.us-west-2.amazonaws.com/notebook-7.2.0-py3-none-any.whl
notebook-shim==0.2.4
num2words==0.5.10
numba==0.58.1
numpy==1.22.2
numpydoc==1.4.0
nvidia-cublas-cu11==11.11.3.6
nvidia-cuda-cupti-cu11==11.8.87
nvidia-cuda-nvrtc-cu11==11.8.89
nvidia-cuda-runtime-cu11==11.8.89
nvidia-cudnn-cu11==8.7.0.84
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.3.0.86
nvidia-cusolver-cu11==11.4.1.48
nvidia-cusparse-cu11==11.7.5.86
nvidia-ml-py==11.495.46
nvidia-nccl-cu11==2.19.3
nvidia-nvtx-cu11==11.8.86
oauthlib==3.2.2
objsize==0.6.1
oic==1.5.0
openai==1.30.4
opencensus==0.11.4
opencensus-context==0.1.3
opencv-python==4.8.1.78
opencv-python-headless==4.8.1.78
openpyxl==3.1.2
opentelemetry-api==1.25.0
opentelemetry-exporter-otlp==1.25.0
opentelemetry-exporter-otlp-proto-common==1.25.0
opentelemetry-exporter-otlp-proto-grpc==1.25.0
opentelemetry-exporter-otlp-proto-http==1.25.0
opentelemetry-proto==1.25.0
opentelemetry-sdk==1.25.0
opentelemetry-semantic-conventions==0.46b0
orjson==3.10.2
overrides==7.7.0
packaging==23.2
pamela==1.0.0
pandas==1.3.5
pandasql==0.7.3
pandocfilters==1.4.2
parso==0.8.4
partd==1.1.0
passlib[bcrypt]==1.7.2
pathos==0.2.9
pathy==0.10.1
pdfminer-six==20200517
pdfplumber==0.5.28
pexpect==4.8.0
pgmpy==0.1.9
pikepdf==8.4.1
pillow==10.3.0
pkgconfig==1.5.2
platformdirs==3.11.0
plotly==5.1.0
pluggy==1.4.0
port-for==0.7.1
portalocker==2.3.2
pox==0.3.4
ppft==1.7.6.8
preshed==3.0.8
prometheus-client==0.13.1
prompt-toolkit==3.0.45
proto-plus==1.22.0
protobuf==3.19.6
psutil==5.9.8
psycopg2-binary==2.9.9
ptyprocess==0.6.0
pure-eval==0.2.2
py-partiql-parser==0.5.1
py-spy==0.3.14
pyarrow==14.0.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycodestyle==2.11.1
pycparser==2.22
pycryptodome==3.20.0
pycryptodomex==3.20.0
pydantic==1.10.14
pydocstyle==6.3.0
pyflakes==3.2.0
pygithub==1.53
pygments==2.17.2
pyjwkest==1.4.2
pyjwt[crypto]==2.8.0
pylint==3.2.2
pympler==1.0.1
pyop==3.3.1
pyopenssl==24.1.0
pyparsing==3.1.1
pypdf[crypto]==4.1.0
pyre2-updated==0.3.8
pyroscope-io==0.8.0
pytesseract==0.3.10
pytest==7.2.0
pytest-memray==1.4.0
pytest-mock==3.1.1
pytest-postgresql==2.2.1
pytest-redis==3.0.2
pytest-timeout==1.4.2
pytest-xdist==3.6.1
python-crfsuite==0.9.9
python-dateutil==2.9.0
python-dotenv==0.19.2
python-gflags==3.1.2
python-json-logger==2.0.7
python-lsp-jsonrpc==1.1.2
python-lsp-server[all]==1.11.0
python-multipart==0.0.9
python-redis-lock==3.7.0
python-snappy==0.6.1
python3-saml==1.10.1
pytoolconfig[global]==1.3.0
pytz==2019.3
pyyaml==6.0.1
pyzmq==25.1.2
querystring-parser==1.2.4
ray[default,observability,train] @ https://snorkel-python-wheels.s3.us-west-2.amazonaws.com/ray-2.8.1-cp310-cp310-manylinux2014_x86_64.whl
redbaron==0.9.2
redis==5.0.0
referencing==0.34.0
regex==2022.1.18
requests==2.32.3
requests-oauthlib==1.3.0
requests-toolbelt==1.0.0
responses==0.18.0
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.7.0
rope==1.13.0
rpds-py==0.13.2
rply==0.7.8
rq==1.15.1
rsa==4.7
ruamel-yaml==0.17.20
ruamel-yaml-clib==0.2.6
rx==3.0.1
s3fs==0.4.2
s3transfer==0.10.1
sacremoses==0.1.1
safetensors==0.4.3
sagemaker==2.223.0
schema==0.7.5
scikit-learn==1.2.0
scipy==1.10.1
seaborn==0.11.2
send2trash==1.8.2
sentence-transformers==2.7.0
sentencepiece==0.2.0
sentry-sdk==1.26.0
setfit==1.0.3
setproctitle==1.2.2
setuptools==69.5.1
shap==0.40.0
shapely==1.8.5.post1
simpervisor==1.0.0
six==1.16.0
sklearn-crfsuite==0.3.6
slicer==0.0.7
smart-open==6.3.0
smdebug-rulesconfig==1.0.1
smmap==3.0.5
sniffio==1.2.0
snowballstemmer==2.2.0
snowflake-connector-python[pandas]==3.7.1
snowflake-sqlalchemy==1.5.1
sortedcontainers==2.4.0
soupsieve==2.0
spacy[lookups]==3.4.4
spacy-legacy==3.0.10
spacy-loggers==1.0.4
spacy-lookups-data==1.0.3
spark-parser==1.8.9
sparse==0.13.0
sphinx==5.1.1
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
sqlalchemy==1.4.44
sqlparse==0.5.0
srsly==2.4.5
stack-data==0.6.2
stanza==1.5.1
starlette==0.36.3
statsd==3.3.0
stdlib-list==0.8.0
stream-zip==0.0.71
sympy==1.12
tabulate==0.9.0
tblib==3.0.0
tenacity==6.3.1
tensorboardx==2.6
termcolor==1.1.0
terminado==0.18.1
textblob==0.15.3
thinc==8.1.5
threadpoolctl==3.1.0
thrift==0.16.0
tiktoken==0.3.2
timm==0.9.16
tinycss2==1.1.1
tokenizers==0.19.1
toml==0.10.2
tomli==2.0.1
tomlkit==0.12.3
toolz==0.12.0
torch @ https://download.pytorch.org/whl/cu118/torch-2.2.2%2Bcu118-cp310-cp310-linux_x86_64.whl
torchvision @ https://download.pytorch.org/whl/cu118/torchvision-0.17.2%2Bcu118-cp310-cp310-linux_x86_64.whl
tornado==6.4.1
tqdm==4.66.4
traitlets==5.14.2
transformers[torch]==4.42.4
triton==2.2.0
typer==0.7.0
types-awscrt==0.20.9
types-python-dateutil==2.9.0.20240316
types-s3transfer==0.10.1
typing-extensions==4.11.0
ujson==5.4.0
uri-template==1.3.0
urllib3==2.2.2
uvicorn[standard]==0.28.1
uvloop==0.19.0
virtualenv==20.21.0
wand==0.6.7
wasabi==0.10.1
watchfiles==0.21.0
wcwidth==0.1.8
webcolors==1.13
webencodings==0.5.1
websocket-client==1.7.0
websockets==12.0
werkzeug==3.0.3
whatthepatch==1.0.5
wheel==0.43.0
widgetsnbextension==4.0.10
wrapt==1.12.1
xdis==6.0.5
xgboost==1.1.0
xlsxwriter==1.2.7
xmlsec==1.3.3
xmltodict==0.12.0
xxhash==3.0.0
yapf==0.40.2
yarl==1.9.4
zict==3.0.0
zipp==3.7.0
derekahuang commented 3 months ago

https://github.com/Azure/azure-sdk-for-python/issues/29719 suggests that 2.13.1 should work and that azure-common-storage might be the culprit. However, I have tested 2.13.1 and I do not have that package installed

github-actions[bot] commented 3 months ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jalauzon-msft @vincenttran-msft.

swathipil commented 3 months ago

Hi @derekahuang - Thanks for opening an issue. We'll take a look asap!

jalauzon-msft commented 3 months ago

Hi @derekahuang, we are unable to reproduce your issue. Could you try enabling HTTP Debug logging and sharing the output (redacting sensitive information) so we can better debug what may be happening? See here for more information on enabling logging.

Another idea would be attempting this in a clean pip environment where you only install azure-storage-blob to ensure one of your many other dependencies are not interfering.

github-actions[bot] commented 3 months ago

Hi @derekahuang. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

github-actions[bot] commented 3 months ago

Hi @derekahuang, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!