[Datasets] Create Dataset download component.

google / nitroml

NitroML is a modular, portable, and scalable model-quality benchmarking framework for Machine Learning and Automated Machine Learning (AutoML) pipelines.

Apache License 2.0

41 stars 6 forks source link

[Datasets] Create Dataset download component. #17

Open cweill opened 4 years ago

cweill commented 4 years ago

We need a way for TFDS to lazily download and prepare datasets, so that it only runs at pipeline execution time instead of during DAG construction time.

nikhil-dce commented 4 years ago

Do you think we still need this? We can change the TFDS data_dir (e.g. https://github.com/googleinterns/nitroml/blob/nitroml_cloud/examples/titanic_benchmark.py#L50)