Yale-LILY / SummerTime

An open-source text summarization toolkit for non-experts. EMNLP'2021 Demo
https://arxiv.org/abs/2108.12738
Apache License 2.0
264 stars 30 forks source link

Can i specify dataset path in datasetclass? #74

Closed epsilon-deltta closed 3 years ago

epsilon-deltta commented 3 years ago
import dataset
arxdt = dataset.ArxivDataset()

In this, this default saved path is /home/$username/.cache/huggingface/datasets/summertime_arxiv. Can I specify a certain path when downloading and loading the dataset?

troyfeng116 commented 3 years ago

Can I specify a certain path when downloading and loading the dataset?

@epsilon-deltta You can create an environment variable HF_HOME specifying where you want .cache. i.e. export HF_HOME=path/to/somewhere/.cache

MuroriM commented 3 years ago

@epsilon-deltta Do you want to change the path for only a certain dataset or for all datasets?

epsilon-deltta commented 3 years ago

@troyfeng116 thank you. I solved my problem cuz of you 😁😁. But, i hoped to change the path in the programming level, not the system level. I mean like this

import dataset
arxdt = dataset.ArxivDataset(path = path/to/somewhere) # default is ~/.cache/huggingface/datasets