search

google-research / meta-dataset

A dataset of datasets for learning to learn from few examples
Apache License 2.0
762 stars 139 forks source link

Meta-Dataset in TFDS: can't download&convert quickdraw #114

Open ruili-pml opened 1 year ago

ruili-pml commented 1 year ago

Hi,

I'm having trouble downloading and converting quickdraw dataset:

Traceback (most recent call last):
  File "/scratch/work/lir3/.conda_envs/fsc/bin/tfds", line 8, in <module>
    sys.exit(launch_cli())
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/main.py", line 109, in launch_cli
    app.run(main, flags_parser=_parse_flags)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/main.py", line 104, in main
    args.subparser_fn(args)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/build.py", line 274, in _build_datasets
    _download_and_prepare(args, builder)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/scripts/cli/build.py", line 510, in _download_and_prepare
    builder.download_and_prepare(
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/core/logging/__init__.py", line 169, in __call__
    return function(*args, **kwargs)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/core/dataset_builder.py", line 628, in download_and_prepare
    self._download_and_prepare(
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/core/dataset_builder.py", line 1436, in _download_and_prepare
    split_generators = self._split_generators(  # pylint: disable=unexpected-keyword-arg
  File "/scratch/work/lir3/FSC/meta-dataset/meta_dataset/data/tfds/md_tfds.py", line 287, in _split_generators
    paths = dl_manager.download_and_extract(self.builder_config.filenames)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/core/download/download_manager.py", line 686, in download_and_extract
    return _map_promise(self._download_extract, url_or_urls)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/core/download/download_manager.py", line 829, in _map_promise
    res = tree_utils.map_structure(
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tree/__init__.py", line 435, in map_structure
    [func(*args) for args in zip(*map(flatten, structures))])
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tree/__init__.py", line 435, in <listcomp>
    [func(*args) for args in zip(*map(flatten, structures))])
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/core/download/download_manager.py", line 830, in <lambda>
    lambda p: p.get(), all_promises
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/promise/promise.py", line 512, in get
    return self._target_settled_value(_raise=True)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/promise/promise.py", line 516, in _target_settled_value
    return self._target()._settled_value(_raise)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/promise/promise.py", line 226, in _settled_value
    reraise(type(raise_val), raise_val, self._traceback)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/six.py", line 719, in reraise
    raise value
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/promise/promise.py", line 844, in handle_future_result
    resolve(future.result())
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/concurrent/futures/_base.py", line 433, in result
    return self.__get_result()
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/core/download/downloader.py", line 225, in _sync_download
    return self._sync_file_copy(url, destination_path)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow_datasets/core/download/downloader.py", line 191, in _sync_file_copy
    tf.io.gfile.copy(filepath, out_path)
  File "/scratch/work/lir3/.conda_envs/fsc/lib/python3.9/site-packages/tensorflow/python/lib/io/file_io.py", line 581, in copy_v2
    _pywrap_file_io.CopyFile(
tensorflow.python.framework.errors_impl.FailedPreconditionError: Error executing an HTTP request: libcurl code 77 meaning 'Problem with the SSL CA cert (path? access rights?)', error details: error setting certificate verify locations:  CAfile: /etc/ssl/certs/ca-certificates.crt CApath: none
     when reading gs://quickdraw_dataset/full/numpy_bitmap/The Eiffel Tower.npy

I was wondering do you know how to fix it?

Thanks, Rui