Closed NikhilBartwal closed 3 years ago
1) Have you tried with tfds-nightly ? You need to replace any_dataset
by a valid dataset name
2) The community datasets are installed at runtime. They are not shipped with TFDS
@Conchylicultor Community datasets seem to be working successfully with tfds-nightly
on colab. However, the same does not work on local environment.
Both the TFDS versions were 4.2.0+nightly
```
Python 3.8.6 (tags/v3.8.6:db45529, Sep 23 2020, 15:52:53) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow_datasets as tfds
2021-03-16 17:52:41.327784: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
>>> tfds.__version__
'4.2.0+nightly'
>>> ds = tfds.load('huggingface:art')
Traceback (most recent call last):
File "
Make sure to update your tfds-nightly with pip install --upgrade tfds-nightly
@Conchylicultor I'm currently using tfds-nightly==4.2.0.dev202103160106
on my local machine and community datasets does not seem to work on it.
I have tried uninstalling TFDS completely and then reinstalling using pip install tfds-nightly
but it didn't work.
I have also tried installing TFDS from the freshly cloned repo as well.
Do you have any idea why this could be happending?
@Conchylicultor I think that I have pinpointed the issue. Since the original question has been answered, I think I should open a new issue so as to avoid further confusion for others.
For this year's GSOC, TFDS is going forward with community datasets. However, I had some queries regarding it as I would like to contribute to TFDS as part of this year's GSOC.
The community datasets index currently contains only
huggingface
as part of registered namespaces. https://github.com/tensorflow/datasets/blob/fcd51122d639b6406648912f27ed9a8446986a84/tensorflow_datasets/community-datasets.toml#L4 However, when usingtfds.load('huggingface: any_dataset')
results inDataset not found
. So, are the community datasets not currently operational?Also, I was thinking whether the community datasets scripts would be part of TFDS or they will be downloaded on runtime when the user loads it?
@Conchylicultor @vijayphoenix It would be great if the project could be discussed a bit more into detail. Thanks!