ihsaan-ullah / meta-album

Meta-Album meta-dataset for few-shot image classification
24 stars 7 forks source link

Problem downloading dataset **Bucket name does not follow S3 standards.** #11

Closed sebastianpinedaar closed 8 months ago

sebastianpinedaar commented 8 months ago

Hi! I am trying to download a dataset, using openml 0.13.0, with the following code:

openml.datasets.get_dataset(dataset_id=44238, download_data=True, download_all_files=True)

But I get the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pineda/anaconda3/envs/quick_tune/lib/python3.9/site-packages/openml/datasets/functions.py", line 449, in get_dataset
    parquet_file = _get_dataset_parquet(
  File "/home/pineda/anaconda3/envs/quick_tune/lib/python3.9/site-packages/openml/datasets/functions.py", line 1041, in _get_dataset_parquet
    openml._api_calls._download_minio_bucket(source=cast(str, url), destination=cache_directory)
  File "/home/pineda/anaconda3/envs/quick_tune/lib/python3.9/site-packages/openml/_api_calls.py", line 187, in _download_minio_bucket
    for file_object in client.list_objects(bucket, recursive=True):
  File "/home/pineda/anaconda3/envs/quick_tune/lib/python3.9/site-packages/minio/api.py", line 2702, in _list_objects
    check_bucket_name(bucket_name)
  File "/home/pineda/anaconda3/envs/quick_tune/lib/python3.9/site-packages/minio/helpers.py", line 211, in check_bucket_name
    raise ValueError(
ValueError: Bucket name does not follow S3 standards. Bucket: datasets/0004/44238

Do you maybe know what is causing this?

Regards,

Sebastian

PGijsbers commented 8 months ago

Please upgrade to the latest version of openml-python (0.14.2) by running python -m pip install --upgrade openml. The server has changed the location/organisation of their buckets on MinIO, and the 0.13.0 release was not forward compatible with this change.