search

snap-stanford / relbench

RelBench: Relational Deep Learning Benchmark
https://relbench.stanford.edu
MIT License
223 stars 41 forks source link

Speed up downloading datasets via multiple connections #265

Closed akihironitta closed 2 months ago

akihironitta commented 2 months ago

Since the default way is too slow (at 5~15 MB/s from an AWS us-west-2 instance), I manually downloaded files using an external tool (https://github.com/aria2/aria2):

$ aria2c -x 16 -d ~/.cache/relbench/rel-amazon/ https://relbench.stanford.edu/download/rel-amazon/db.zip

Haven't done research on other options, it'd be nice to consider supporting something like this.

rishabh-ranjan commented 2 months ago

Thanks. Added to the README.