voxel51 / eta

ETA: Extensible Toolkit for Analytics
https://voxel51.com
Apache License 2.0
29 stars 13 forks source link

Fix address for gdrive download content #610

Closed swheaton closed 10 months ago

swheaton commented 10 months ago

There has previously been some tricks and attempts to confirm downloading large files from google drive without requiring user action. It seems that somewhat recently these stopped working.

Instead, if we fetch the file from drive.usercontent.google.com/download rather than drive.google.com/uc, we can bypass the confirm business.

Testing from within fiftyone because this module doesn't work on its own really.

import fiftyone as fo

# Set to new dir, or ensure quickstart-groups isn't downloaded yet
# fo.config.dataset_zoo_dir=''

fo.zoo.load_zoo_dataset('quickstart-groups')

Before change

Downloading dataset to '/Users/stuart/fiftyone2/quickstart-groups'
Downloading dataset...
 100% |█████████████████████████████████████████████████████████████████████████████████████████|   19.0Kb/19.0Kb [635.0us elapsed, 0s remaining, 40.0Mb/s] 
Extracting dataset...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/stuart/dev/fiftyone_venv2/lib/python3.8/site-packages/fiftyone/zoo/datasets/__init__.py", line 251, in load_zoo_dataset
    info, dataset_dir = download_zoo_dataset(
  File "/Users/stuart/dev/fiftyone_venv2/lib/python3.8/site-packages/fiftyone/zoo/datasets/__init__.py", line 170, in download_zoo_dataset
    return zoo_dataset.download_and_prepare(
  File "/Users/stuart/dev/fiftyone_venv2/lib/python3.8/site-packages/fiftyone/zoo/datasets/__init__.py", line 1119, in download_and_prepare
    ) = self._download_and_prepare(dataset_dir, scratch_dir, None)
  File "/Users/stuart/dev/fiftyone_venv2/lib/python3.8/site-packages/fiftyone/zoo/datasets/base.py", line 3102, in _download_and_prepare
    _download_and_extract_archive(
  File "/Users/stuart/dev/fiftyone_venv2/lib/python3.8/site-packages/fiftyone/zoo/datasets/base.py", line 3261, in _download_and_extract_archive
    etau.extract_archive(archive_path)
  File "/Users/stuart/dev/fiftyone_venv2/lib/python3.8/site-packages/eta/core/utils.py", line 3578, in extract_archive
    extract_zip(archive_path, outdir=outdir, delete_zip=delete_archive)
  File "/Users/stuart/dev/fiftyone_venv2/lib/python3.8/site-packages/eta/core/utils.py", line 3604, in extract_zip
    with zf.ZipFile(zip_path, "r", allowZip64=True) as f:
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/zipfile.py", line 1269, in __init__
    self._RealGetContents()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/zipfile.py", line 1336, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file