search

facebookresearch / ELI5

Scripts and links to recreate the ELI5 dataset.
Other
316 stars 40 forks source link

Data size #9

Closed AmosHua closed 4 years ago

AmosHua commented 5 years ago

Hi, can you teel me the exact support doc files' total size?

yjernite commented 4 years ago

Here's what we have after finishing everything:

ELI5$ du -sh data_creation/processed_data/collected_docs/explainlikeimfive/*.json
13G data_creation/processed_data/collected_docs/explainlikeimfive/0.json
14G data_creation/processed_data/collected_docs/explainlikeimfive/1.json
15G data_creation/processed_data/collected_docs/explainlikeimfive/2.json
16G data_creation/processed_data/collected_docs/explainlikeimfive/3.json
17G data_creation/processed_data/collected_docs/explainlikeimfive/4.json
19G data_creation/processed_data/collected_docs/explainlikeimfive/5.json
19G data_creation/processed_data/collected_docs/explainlikeimfive/6.json
21G data_creation/processed_data/collected_docs/explainlikeimfive/7.json
24G data_creation/processed_data/collected_docs/explainlikeimfive/8.json
31G data_creation/processed_data/collected_docs/explainlikeimfive/9.json