Unify dataset, upload it into Cloud instead of github, download it automatically

COOL-cohort / COOL

the source code of the COOL system

https://www.comp.nus.edu.sg/~dbsystem/cool/

Apache License 2.0

45 stars 16 forks source link

Unify dataset, upload it into Cloud instead of github, download it automatically #41

Closed Zrealshadow closed 2 years ago

Zrealshadow commented 2 years ago

TODO

Unify three dataset and add README.md for each one. the content in three dataset directory is mess.
Seperate static data from source code. upload dataset to cloud storage. Using github to store data is not a good idea.
add a script which can automatically download datasets from cloud storage and place them in right position.

KimballCai commented 2 years ago

What do you mean by the content in the first point? As for the second and the third points, we only store small samples as protoFile for testing the COOL system which is also acceptable. You can also check other apache projects, such as Apache Druid and so on. In the future, we will store test files of a huge size in cloud storage.